Wed Apr 19 17:24:16 2017 UTC ()
Updated py-chardet to 3.0.2.

chardet 3.0.2

Fixes an issue where detect would sometimes return None instead of a dict with the keys encoding, language, and confidence (Issue #113, PR #114).

chardet 3.0.1

This bugfix release fixes a crash in the EUC-TW prober when it encountered certain strings (Issue #67).

chardet 3.0.0

This release is long overdue, but still mostly serves as a placeholder
for the impending 4.0.0 release, which will have retrained models
for better accuracy. For now, this release will get the following
improvements up on PyPI:

    Added support for Turkish ISO-8859-9 detection (PR #41, thanks @queeup)
    Commented out large unused sections of Big5 and EUC-KR tables to save memory (8bc4b89)
    Removed Python 3.2 from testing, but add 3.4 - 3.6
    Ensure that stdin is open with mode 'rb' for chardetect CLI. (PR #38, thanks @lpsinger)
    Fixed chardetect crash with non-ascii file names (PR #39, thanks @nkanaev)
    Made naming conventions more Pythonic throughout (no more mTypicalPositiveRatio, and instead typical_positive_ratio)
    Modernized test scripts and infrastructure so we've got Travis testing and all that stuff
    Rename filter_without_english_words to filter_international_words and make it match current Mozilla implementation (PR #44, thanks @rsnair2)
    Updated filter_english_letters to match C implementation (c665459)
    Temporarily disabled Hungarian ISO-8859-2 and Windows-1250 detection because it is very inaccurate (da6c0a0)
    Allow CLI sub-package to be importable (PR #55)
    Add a hypotheis-based test (PR #66, thanks @DRMacIver)
    Strip endianness from UTF with BOM predictions so that the encoding can be passed directly to bytes.decode() (PR #73, thanks @snoack)
    Fixed broken links in docs (PR #90, thanks @roskakori)
    Added early exit to chardetect when encoding is detected instead of looping through entire file (PR #103, thanks @jpz)
    Use bytearray objects internally instead of wrap_ord calls, which provides a nice performance boost across the board (PR #106)
    Add language property to probers and UniversalDetector results (PR #180)
    Mark the 5 known test failures as such so we can have more useful Travis build results in the meantime (d588407)


(wiz)
diff -r1.16 -r1.17 pkgsrc/converters/py-chardet/Makefile
diff -r1.8 -r1.9 pkgsrc/converters/py-chardet/PLIST
diff -r1.6 -r1.7 pkgsrc/converters/py-chardet/distinfo
cvs diff -r1.16 -r1.17 pkgsrc/converters/py-chardet/Makefile (expand / switch to unified diff)

--- pkgsrc/converters/py-chardet/Makefile 2017/01/03 13:23:01 1.16
+++ pkgsrc/converters/py-chardet/Makefile 2017/04/19 17:24:16 1.17
@@ -1,21 +1,23 @@ @@ -1,21 +1,23 @@
1# $NetBSD: Makefile,v 1.16 2017/01/03 13:23:01 jperkin Exp $ 1# $NetBSD: Makefile,v 1.17 2017/04/19 17:24:16 wiz Exp $
2 2
3DISTNAME= chardet-2.3.0 3DISTNAME= chardet-3.0.2
4PKGREVISION= 1 
5PKGNAME= ${PYPKGPREFIX}-${DISTNAME} 4PKGNAME= ${PYPKGPREFIX}-${DISTNAME}
6CATEGORIES= converters python 5CATEGORIES= converters python
7MASTER_SITES= ${MASTER_SITE_PYPI:=c/chardet/} 6MASTER_SITES= ${MASTER_SITE_PYPI:=c/chardet/}
8 7
9MAINTAINER= bartosz.kuzma@gmail.com 8MAINTAINER= bartosz.kuzma@gmail.com
10HOMEPAGE= http://pypi.python.org/pypi/chardet 9HOMEPAGE= http://pypi.python.org/pypi/chardet
11COMMENT= Character encoding auto-detection in Python 10COMMENT= Character encoding auto-detection in Python
12LICENSE= gnu-lgpl-v2.1 11LICENSE= gnu-lgpl-v2.1
13 12
14REPLACE_PYTHON= chardet/chardetect.py 13# TEST_DEPENDS; however, no tests found
 14BUILD_DEPENDS+= ${PYPKGPREFIX}-test-[0-9]*:../../devel/py-test
 15BUILD_DEPENDS+= ${PYPKGPREFIX}-hypothesis-[0-9]*:../../devel/py-hypothesis
 16BUILD_DEPENDS+= ${PYPKGPREFIX}-py-[0-9]*:../../devel/py-py
15 17
16post-install: 18post-install:
17 ${MV} ${DESTDIR}${PREFIX}/bin/chardetect ${DESTDIR}${PREFIX}/bin/chardetect-${PYVERSSUFFIX} || ${TRUE} 19 ${MV} ${DESTDIR}${PREFIX}/bin/chardetect ${DESTDIR}${PREFIX}/bin/chardetect-${PYVERSSUFFIX} || ${TRUE}
18 20
19.include "../../lang/python/application.mk" 21.include "../../lang/python/application.mk"
20.include "../../lang/python/egg.mk" 22.include "../../lang/python/egg.mk"
21.include "../../mk/bsd.pkg.mk" 23.include "../../mk/bsd.pkg.mk"
cvs diff -r1.8 -r1.9 pkgsrc/converters/py-chardet/PLIST (expand / switch to unified diff)

--- pkgsrc/converters/py-chardet/PLIST 2016/02/05 12:40:56 1.8
+++ pkgsrc/converters/py-chardet/PLIST 2017/04/19 17:24:16 1.9
@@ -1,53 +1,56 @@ @@ -1,53 +1,56 @@
1@comment $NetBSD: PLIST,v 1.8 2016/02/05 12:40:56 wiz Exp $ 1@comment $NetBSD: PLIST,v 1.9 2017/04/19 17:24:16 wiz Exp $
2bin/chardetect-${PYVERSSUFFIX} 2bin/chardetect-${PYVERSSUFFIX}
3${PYSITELIB}/${EGG_INFODIR}/PKG-INFO 3${PYSITELIB}/${EGG_INFODIR}/PKG-INFO
4${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt 4${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt
5${PYSITELIB}/${EGG_INFODIR}/dependency_links.txt 5${PYSITELIB}/${EGG_INFODIR}/dependency_links.txt
6${PYSITELIB}/${EGG_INFODIR}/entry_points.txt 6${PYSITELIB}/${EGG_INFODIR}/entry_points.txt
7${PYSITELIB}/${EGG_INFODIR}/top_level.txt 7${PYSITELIB}/${EGG_INFODIR}/top_level.txt
8${PYSITELIB}/chardet/__init__.py 8${PYSITELIB}/chardet/__init__.py
9${PYSITELIB}/chardet/__init__.pyc 9${PYSITELIB}/chardet/__init__.pyc
10${PYSITELIB}/chardet/__init__.pyo 10${PYSITELIB}/chardet/__init__.pyo
11${PYSITELIB}/chardet/big5freq.py 11${PYSITELIB}/chardet/big5freq.py
12${PYSITELIB}/chardet/big5freq.pyc 12${PYSITELIB}/chardet/big5freq.pyc
13${PYSITELIB}/chardet/big5freq.pyo 13${PYSITELIB}/chardet/big5freq.pyo
14${PYSITELIB}/chardet/big5prober.py 14${PYSITELIB}/chardet/big5prober.py
15${PYSITELIB}/chardet/big5prober.pyc 15${PYSITELIB}/chardet/big5prober.pyc
16${PYSITELIB}/chardet/big5prober.pyo 16${PYSITELIB}/chardet/big5prober.pyo
17${PYSITELIB}/chardet/chardetect.py 
18${PYSITELIB}/chardet/chardetect.pyc 
19${PYSITELIB}/chardet/chardetect.pyo 
20${PYSITELIB}/chardet/chardistribution.py 17${PYSITELIB}/chardet/chardistribution.py
21${PYSITELIB}/chardet/chardistribution.pyc 18${PYSITELIB}/chardet/chardistribution.pyc
22${PYSITELIB}/chardet/chardistribution.pyo 19${PYSITELIB}/chardet/chardistribution.pyo
23${PYSITELIB}/chardet/charsetgroupprober.py 20${PYSITELIB}/chardet/charsetgroupprober.py
24${PYSITELIB}/chardet/charsetgroupprober.pyc 21${PYSITELIB}/chardet/charsetgroupprober.pyc
25${PYSITELIB}/chardet/charsetgroupprober.pyo 22${PYSITELIB}/chardet/charsetgroupprober.pyo
26${PYSITELIB}/chardet/charsetprober.py 23${PYSITELIB}/chardet/charsetprober.py
27${PYSITELIB}/chardet/charsetprober.pyc 24${PYSITELIB}/chardet/charsetprober.pyc
28${PYSITELIB}/chardet/charsetprober.pyo 25${PYSITELIB}/chardet/charsetprober.pyo
 26${PYSITELIB}/chardet/cli/__init__.py
 27${PYSITELIB}/chardet/cli/__init__.pyc
 28${PYSITELIB}/chardet/cli/__init__.pyo
 29${PYSITELIB}/chardet/cli/chardetect.py
 30${PYSITELIB}/chardet/cli/chardetect.pyc
 31${PYSITELIB}/chardet/cli/chardetect.pyo
29${PYSITELIB}/chardet/codingstatemachine.py 32${PYSITELIB}/chardet/codingstatemachine.py
30${PYSITELIB}/chardet/codingstatemachine.pyc 33${PYSITELIB}/chardet/codingstatemachine.pyc
31${PYSITELIB}/chardet/codingstatemachine.pyo 34${PYSITELIB}/chardet/codingstatemachine.pyo
32${PYSITELIB}/chardet/compat.py 35${PYSITELIB}/chardet/compat.py
33${PYSITELIB}/chardet/compat.pyc 36${PYSITELIB}/chardet/compat.pyc
34${PYSITELIB}/chardet/compat.pyo 37${PYSITELIB}/chardet/compat.pyo
35${PYSITELIB}/chardet/constants.py 
36${PYSITELIB}/chardet/constants.pyc 
37${PYSITELIB}/chardet/constants.pyo 
38${PYSITELIB}/chardet/cp949prober.py 38${PYSITELIB}/chardet/cp949prober.py
39${PYSITELIB}/chardet/cp949prober.pyc 39${PYSITELIB}/chardet/cp949prober.pyc
40${PYSITELIB}/chardet/cp949prober.pyo 40${PYSITELIB}/chardet/cp949prober.pyo
 41${PYSITELIB}/chardet/enums.py
 42${PYSITELIB}/chardet/enums.pyc
 43${PYSITELIB}/chardet/enums.pyo
41${PYSITELIB}/chardet/escprober.py 44${PYSITELIB}/chardet/escprober.py
42${PYSITELIB}/chardet/escprober.pyc 45${PYSITELIB}/chardet/escprober.pyc
43${PYSITELIB}/chardet/escprober.pyo 46${PYSITELIB}/chardet/escprober.pyo
44${PYSITELIB}/chardet/escsm.py 47${PYSITELIB}/chardet/escsm.py
45${PYSITELIB}/chardet/escsm.pyc 48${PYSITELIB}/chardet/escsm.pyc
46${PYSITELIB}/chardet/escsm.pyo 49${PYSITELIB}/chardet/escsm.pyo
47${PYSITELIB}/chardet/eucjpprober.py 50${PYSITELIB}/chardet/eucjpprober.py
48${PYSITELIB}/chardet/eucjpprober.pyc 51${PYSITELIB}/chardet/eucjpprober.pyc
49${PYSITELIB}/chardet/eucjpprober.pyo 52${PYSITELIB}/chardet/eucjpprober.pyo
50${PYSITELIB}/chardet/euckrfreq.py 53${PYSITELIB}/chardet/euckrfreq.py
51${PYSITELIB}/chardet/euckrfreq.pyc 54${PYSITELIB}/chardet/euckrfreq.pyc
52${PYSITELIB}/chardet/euckrfreq.pyo 55${PYSITELIB}/chardet/euckrfreq.pyo
53${PYSITELIB}/chardet/euckrprober.py 56${PYSITELIB}/chardet/euckrprober.py
@@ -82,26 +85,29 @@ ${PYSITELIB}/chardet/langcyrillicmodel.p @@ -82,26 +85,29 @@ ${PYSITELIB}/chardet/langcyrillicmodel.p
82${PYSITELIB}/chardet/langcyrillicmodel.pyo 85${PYSITELIB}/chardet/langcyrillicmodel.pyo
83${PYSITELIB}/chardet/langgreekmodel.py 86${PYSITELIB}/chardet/langgreekmodel.py
84${PYSITELIB}/chardet/langgreekmodel.pyc 87${PYSITELIB}/chardet/langgreekmodel.pyc
85${PYSITELIB}/chardet/langgreekmodel.pyo 88${PYSITELIB}/chardet/langgreekmodel.pyo
86${PYSITELIB}/chardet/langhebrewmodel.py 89${PYSITELIB}/chardet/langhebrewmodel.py
87${PYSITELIB}/chardet/langhebrewmodel.pyc 90${PYSITELIB}/chardet/langhebrewmodel.pyc
88${PYSITELIB}/chardet/langhebrewmodel.pyo 91${PYSITELIB}/chardet/langhebrewmodel.pyo
89${PYSITELIB}/chardet/langhungarianmodel.py 92${PYSITELIB}/chardet/langhungarianmodel.py
90${PYSITELIB}/chardet/langhungarianmodel.pyc 93${PYSITELIB}/chardet/langhungarianmodel.pyc
91${PYSITELIB}/chardet/langhungarianmodel.pyo 94${PYSITELIB}/chardet/langhungarianmodel.pyo
92${PYSITELIB}/chardet/langthaimodel.py 95${PYSITELIB}/chardet/langthaimodel.py
93${PYSITELIB}/chardet/langthaimodel.pyc 96${PYSITELIB}/chardet/langthaimodel.pyc
94${PYSITELIB}/chardet/langthaimodel.pyo 97${PYSITELIB}/chardet/langthaimodel.pyo
 98${PYSITELIB}/chardet/langturkishmodel.py
 99${PYSITELIB}/chardet/langturkishmodel.pyc
 100${PYSITELIB}/chardet/langturkishmodel.pyo
95${PYSITELIB}/chardet/latin1prober.py 101${PYSITELIB}/chardet/latin1prober.py
96${PYSITELIB}/chardet/latin1prober.pyc 102${PYSITELIB}/chardet/latin1prober.pyc
97${PYSITELIB}/chardet/latin1prober.pyo 103${PYSITELIB}/chardet/latin1prober.pyo
98${PYSITELIB}/chardet/mbcharsetprober.py 104${PYSITELIB}/chardet/mbcharsetprober.py
99${PYSITELIB}/chardet/mbcharsetprober.pyc 105${PYSITELIB}/chardet/mbcharsetprober.pyc
100${PYSITELIB}/chardet/mbcharsetprober.pyo 106${PYSITELIB}/chardet/mbcharsetprober.pyo
101${PYSITELIB}/chardet/mbcsgroupprober.py 107${PYSITELIB}/chardet/mbcsgroupprober.py
102${PYSITELIB}/chardet/mbcsgroupprober.pyc 108${PYSITELIB}/chardet/mbcsgroupprober.pyc
103${PYSITELIB}/chardet/mbcsgroupprober.pyo 109${PYSITELIB}/chardet/mbcsgroupprober.pyo
104${PYSITELIB}/chardet/mbcssm.py 110${PYSITELIB}/chardet/mbcssm.py
105${PYSITELIB}/chardet/mbcssm.pyc 111${PYSITELIB}/chardet/mbcssm.pyc
106${PYSITELIB}/chardet/mbcssm.pyo 112${PYSITELIB}/chardet/mbcssm.pyo
107${PYSITELIB}/chardet/sbcharsetprober.py 113${PYSITELIB}/chardet/sbcharsetprober.py
@@ -109,13 +115,16 @@ ${PYSITELIB}/chardet/sbcharsetprober.pyc @@ -109,13 +115,16 @@ ${PYSITELIB}/chardet/sbcharsetprober.pyc
109${PYSITELIB}/chardet/sbcharsetprober.pyo 115${PYSITELIB}/chardet/sbcharsetprober.pyo
110${PYSITELIB}/chardet/sbcsgroupprober.py 116${PYSITELIB}/chardet/sbcsgroupprober.py
111${PYSITELIB}/chardet/sbcsgroupprober.pyc 117${PYSITELIB}/chardet/sbcsgroupprober.pyc
112${PYSITELIB}/chardet/sbcsgroupprober.pyo 118${PYSITELIB}/chardet/sbcsgroupprober.pyo
113${PYSITELIB}/chardet/sjisprober.py 119${PYSITELIB}/chardet/sjisprober.py
114${PYSITELIB}/chardet/sjisprober.pyc 120${PYSITELIB}/chardet/sjisprober.pyc
115${PYSITELIB}/chardet/sjisprober.pyo 121${PYSITELIB}/chardet/sjisprober.pyo
116${PYSITELIB}/chardet/universaldetector.py 122${PYSITELIB}/chardet/universaldetector.py
117${PYSITELIB}/chardet/universaldetector.pyc 123${PYSITELIB}/chardet/universaldetector.pyc
118${PYSITELIB}/chardet/universaldetector.pyo 124${PYSITELIB}/chardet/universaldetector.pyo
119${PYSITELIB}/chardet/utf8prober.py 125${PYSITELIB}/chardet/utf8prober.py
120${PYSITELIB}/chardet/utf8prober.pyc 126${PYSITELIB}/chardet/utf8prober.pyc
121${PYSITELIB}/chardet/utf8prober.pyo 127${PYSITELIB}/chardet/utf8prober.pyo
 128${PYSITELIB}/chardet/version.py
 129${PYSITELIB}/chardet/version.pyc
 130${PYSITELIB}/chardet/version.pyo
cvs diff -r1.6 -r1.7 pkgsrc/converters/py-chardet/distinfo (expand / switch to unified diff)

--- pkgsrc/converters/py-chardet/distinfo 2015/11/03 01:43:53 1.6
+++ pkgsrc/converters/py-chardet/distinfo 2017/04/19 17:24:16 1.7
@@ -1,6 +1,6 @@ @@ -1,6 +1,6 @@
1$NetBSD: distinfo,v 1.6 2015/11/03 01:43:53 agc Exp $ 1$NetBSD: distinfo,v 1.7 2017/04/19 17:24:16 wiz Exp $
2 2
3SHA1 (chardet-2.3.0.tar.gz) = 50af8f8771ecbeb7a22567129c6c281b8bec3b1c 3SHA1 (chardet-3.0.2.tar.gz) = 6234b04c99465f0661301088f0ab47a77ea97031
4RMD160 (chardet-2.3.0.tar.gz) = b68ed94bccfd97f23cc0a6288ee8c7914f4c7d04 4RMD160 (chardet-3.0.2.tar.gz) = a4f0bab40259ef8042ae562f78ac964eddcb2f4d
5SHA512 (chardet-2.3.0.tar.gz) = d554c9b4564905a2b6c6971aaea2a812174d86b58137c6a2f9d54e2abd2395e6529968b2ce3ca2d97c717d2bd0bd80ab5b98cae9afac1a4add9ea305145030dd 5SHA512 (chardet-3.0.2.tar.gz) = d7b0e5ebeb9a0d5830857decbe2b11309fbb421c31cd391c143bf6f00e2c3d9eec3640954478e71be88ea3dac158b1e1df177a5e2b8b9749b27d2a956542cbf9
6Size (chardet-2.3.0.tar.gz) = 164346 bytes 6Size (chardet-3.0.2.tar.gz) = 1866978 bytes