Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK)) by mollari.NetBSD.org (Postfix) with ESMTPS id 1A9707A167 for ; Wed, 19 Apr 2017 17:24:18 +0000 (UTC) Received: by mail.netbsd.org (Postfix, from userid 605) id 7BDE684DD2; Wed, 19 Apr 2017 17:24:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.netbsd.org (Postfix) with ESMTP id 0CAD984DCF for ; Wed, 19 Apr 2017 17:24:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at netbsd.org Received: from mail.netbsd.org ([IPv6:::1]) by localhost (mail.netbsd.org [IPv6:::1]) (amavisd-new, port 10025) with ESMTP id ueqZG8qGfyJu for ; Wed, 19 Apr 2017 17:24:16 +0000 (UTC) Received: from cvs.NetBSD.org (ivanova.netbsd.org [199.233.217.197]) by mail.netbsd.org (Postfix) with ESMTP id 507B184CE2 for ; Wed, 19 Apr 2017 17:24:16 +0000 (UTC) Received: by cvs.NetBSD.org (Postfix, from userid 500) id 4BF92FBE4; Wed, 19 Apr 2017 17:24:16 +0000 (UTC) Content-Transfer-Encoding: 7bit Content-Type: multipart/mixed; boundary="_----------=_1492622656136900" MIME-Version: 1.0 Date: Wed, 19 Apr 2017 17:24:16 +0000 From: "Thomas Klausner" Subject: CVS commit: pkgsrc/converters/py-chardet To: pkgsrc-changes@NetBSD.org Reply-To: wiz@netbsd.org X-Mailer: log_accum Message-Id: <20170419172416.4BF92FBE4@cvs.NetBSD.org> Sender: pkgsrc-changes-owner@NetBSD.org List-Id: pkgsrc-changes.NetBSD.org Precedence: bulk List-Unsubscribe: This is a multi-part message in MIME format. --_----------=_1492622656136900 Content-Disposition: inline Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII" Module Name: pkgsrc Committed By: wiz Date: Wed Apr 19 17:24:16 UTC 2017 Modified Files: pkgsrc/converters/py-chardet: Makefile PLIST distinfo Log Message: Updated py-chardet to 3.0.2. chardet 3.0.2 Fixes an issue where detect would sometimes return None instead of a dict with the keys encoding, language, and confidence (Issue #113, PR #114). chardet 3.0.1 This bugfix release fixes a crash in the EUC-TW prober when it encountered certain strings (Issue #67). chardet 3.0.0 This release is long overdue, but still mostly serves as a placeholder for the impending 4.0.0 release, which will have retrained models for better accuracy. For now, this release will get the following improvements up on PyPI: Added support for Turkish ISO-8859-9 detection (PR #41, thanks @queeup) Commented out large unused sections of Big5 and EUC-KR tables to save memory (8bc4b89) Removed Python 3.2 from testing, but add 3.4 - 3.6 Ensure that stdin is open with mode 'rb' for chardetect CLI. (PR #38, thanks @lpsinger) Fixed chardetect crash with non-ascii file names (PR #39, thanks @nkanaev) Made naming conventions more Pythonic throughout (no more mTypicalPositiveRatio, and instead typical_positive_ratio) Modernized test scripts and infrastructure so we've got Travis testing and all that stuff Rename filter_without_english_words to filter_international_words and make it match current Mozilla implementation (PR #44, thanks @rsnair2) Updated filter_english_letters to match C implementation (c665459) Temporarily disabled Hungarian ISO-8859-2 and Windows-1250 detection because it is very inaccurate (da6c0a0) Allow CLI sub-package to be importable (PR #55) Add a hypotheis-based test (PR #66, thanks @DRMacIver) Strip endianness from UTF with BOM predictions so that the encoding can be passed directly to bytes.decode() (PR #73, thanks @snoack) Fixed broken links in docs (PR #90, thanks @roskakori) Added early exit to chardetect when encoding is detected instead of looping through entire file (PR #103, thanks @jpz) Use bytearray objects internally instead of wrap_ord calls, which provides a nice performance boost across the board (PR #106) Add language property to probers and UniversalDetector results (PR #180) Mark the 5 known test failures as such so we can have more useful Travis build results in the meantime (d588407) To generate a diff of this commit: cvs rdiff -u -r1.16 -r1.17 pkgsrc/converters/py-chardet/Makefile cvs rdiff -u -r1.8 -r1.9 pkgsrc/converters/py-chardet/PLIST cvs rdiff -u -r1.6 -r1.7 pkgsrc/converters/py-chardet/distinfo Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files. --_----------=_1492622656136900 Content-Disposition: inline Content-Length: 4943 Content-Transfer-Encoding: binary Content-Type: text/x-diff; charset=us-ascii Modified files: Index: pkgsrc/converters/py-chardet/Makefile diff -u pkgsrc/converters/py-chardet/Makefile:1.16 pkgsrc/converters/py-chardet/Makefile:1.17 --- pkgsrc/converters/py-chardet/Makefile:1.16 Tue Jan 3 13:23:01 2017 +++ pkgsrc/converters/py-chardet/Makefile Wed Apr 19 17:24:16 2017 @@ -1,7 +1,6 @@ -# $NetBSD: Makefile,v 1.16 2017/01/03 13:23:01 jperkin Exp $ +# $NetBSD: Makefile,v 1.17 2017/04/19 17:24:16 wiz Exp $ -DISTNAME= chardet-2.3.0 -PKGREVISION= 1 +DISTNAME= chardet-3.0.2 PKGNAME= ${PYPKGPREFIX}-${DISTNAME} CATEGORIES= converters python MASTER_SITES= ${MASTER_SITE_PYPI:=c/chardet/} @@ -11,7 +10,10 @@ HOMEPAGE= http://pypi.python.org/pypi/ch COMMENT= Character encoding auto-detection in Python LICENSE= gnu-lgpl-v2.1 -REPLACE_PYTHON= chardet/chardetect.py +# TEST_DEPENDS; however, no tests found +BUILD_DEPENDS+= ${PYPKGPREFIX}-test-[0-9]*:../../devel/py-test +BUILD_DEPENDS+= ${PYPKGPREFIX}-hypothesis-[0-9]*:../../devel/py-hypothesis +BUILD_DEPENDS+= ${PYPKGPREFIX}-py-[0-9]*:../../devel/py-py post-install: ${MV} ${DESTDIR}${PREFIX}/bin/chardetect ${DESTDIR}${PREFIX}/bin/chardetect-${PYVERSSUFFIX} || ${TRUE} Index: pkgsrc/converters/py-chardet/PLIST diff -u pkgsrc/converters/py-chardet/PLIST:1.8 pkgsrc/converters/py-chardet/PLIST:1.9 --- pkgsrc/converters/py-chardet/PLIST:1.8 Fri Feb 5 12:40:56 2016 +++ pkgsrc/converters/py-chardet/PLIST Wed Apr 19 17:24:16 2017 @@ -1,4 +1,4 @@ -@comment $NetBSD: PLIST,v 1.8 2016/02/05 12:40:56 wiz Exp $ +@comment $NetBSD: PLIST,v 1.9 2017/04/19 17:24:16 wiz Exp $ bin/chardetect-${PYVERSSUFFIX} ${PYSITELIB}/${EGG_INFODIR}/PKG-INFO ${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt @@ -14,9 +14,6 @@ ${PYSITELIB}/chardet/big5freq.pyo ${PYSITELIB}/chardet/big5prober.py ${PYSITELIB}/chardet/big5prober.pyc ${PYSITELIB}/chardet/big5prober.pyo -${PYSITELIB}/chardet/chardetect.py -${PYSITELIB}/chardet/chardetect.pyc -${PYSITELIB}/chardet/chardetect.pyo ${PYSITELIB}/chardet/chardistribution.py ${PYSITELIB}/chardet/chardistribution.pyc ${PYSITELIB}/chardet/chardistribution.pyo @@ -26,18 +23,24 @@ ${PYSITELIB}/chardet/charsetgroupprober. ${PYSITELIB}/chardet/charsetprober.py ${PYSITELIB}/chardet/charsetprober.pyc ${PYSITELIB}/chardet/charsetprober.pyo +${PYSITELIB}/chardet/cli/__init__.py +${PYSITELIB}/chardet/cli/__init__.pyc +${PYSITELIB}/chardet/cli/__init__.pyo +${PYSITELIB}/chardet/cli/chardetect.py +${PYSITELIB}/chardet/cli/chardetect.pyc +${PYSITELIB}/chardet/cli/chardetect.pyo ${PYSITELIB}/chardet/codingstatemachine.py ${PYSITELIB}/chardet/codingstatemachine.pyc ${PYSITELIB}/chardet/codingstatemachine.pyo ${PYSITELIB}/chardet/compat.py ${PYSITELIB}/chardet/compat.pyc ${PYSITELIB}/chardet/compat.pyo -${PYSITELIB}/chardet/constants.py -${PYSITELIB}/chardet/constants.pyc -${PYSITELIB}/chardet/constants.pyo ${PYSITELIB}/chardet/cp949prober.py ${PYSITELIB}/chardet/cp949prober.pyc ${PYSITELIB}/chardet/cp949prober.pyo +${PYSITELIB}/chardet/enums.py +${PYSITELIB}/chardet/enums.pyc +${PYSITELIB}/chardet/enums.pyo ${PYSITELIB}/chardet/escprober.py ${PYSITELIB}/chardet/escprober.pyc ${PYSITELIB}/chardet/escprober.pyo @@ -92,6 +95,9 @@ ${PYSITELIB}/chardet/langhungarianmodel. ${PYSITELIB}/chardet/langthaimodel.py ${PYSITELIB}/chardet/langthaimodel.pyc ${PYSITELIB}/chardet/langthaimodel.pyo +${PYSITELIB}/chardet/langturkishmodel.py +${PYSITELIB}/chardet/langturkishmodel.pyc +${PYSITELIB}/chardet/langturkishmodel.pyo ${PYSITELIB}/chardet/latin1prober.py ${PYSITELIB}/chardet/latin1prober.pyc ${PYSITELIB}/chardet/latin1prober.pyo @@ -119,3 +125,6 @@ ${PYSITELIB}/chardet/universaldetector.p ${PYSITELIB}/chardet/utf8prober.py ${PYSITELIB}/chardet/utf8prober.pyc ${PYSITELIB}/chardet/utf8prober.pyo +${PYSITELIB}/chardet/version.py +${PYSITELIB}/chardet/version.pyc +${PYSITELIB}/chardet/version.pyo Index: pkgsrc/converters/py-chardet/distinfo diff -u pkgsrc/converters/py-chardet/distinfo:1.6 pkgsrc/converters/py-chardet/distinfo:1.7 --- pkgsrc/converters/py-chardet/distinfo:1.6 Tue Nov 3 01:43:53 2015 +++ pkgsrc/converters/py-chardet/distinfo Wed Apr 19 17:24:16 2017 @@ -1,6 +1,6 @@ -$NetBSD: distinfo,v 1.6 2015/11/03 01:43:53 agc Exp $ +$NetBSD: distinfo,v 1.7 2017/04/19 17:24:16 wiz Exp $ -SHA1 (chardet-2.3.0.tar.gz) = 50af8f8771ecbeb7a22567129c6c281b8bec3b1c -RMD160 (chardet-2.3.0.tar.gz) = b68ed94bccfd97f23cc0a6288ee8c7914f4c7d04 -SHA512 (chardet-2.3.0.tar.gz) = d554c9b4564905a2b6c6971aaea2a812174d86b58137c6a2f9d54e2abd2395e6529968b2ce3ca2d97c717d2bd0bd80ab5b98cae9afac1a4add9ea305145030dd -Size (chardet-2.3.0.tar.gz) = 164346 bytes +SHA1 (chardet-3.0.2.tar.gz) = 6234b04c99465f0661301088f0ab47a77ea97031 +RMD160 (chardet-3.0.2.tar.gz) = a4f0bab40259ef8042ae562f78ac964eddcb2f4d +SHA512 (chardet-3.0.2.tar.gz) = d7b0e5ebeb9a0d5830857decbe2b11309fbb421c31cd391c143bf6f00e2c3d9eec3640954478e71be88ea3dac158b1e1df177a5e2b8b9749b27d2a956542cbf9 +Size (chardet-3.0.2.tar.gz) = 1866978 bytes --_----------=_1492622656136900--