Received: by mail.netbsd.org (Postfix, from userid 605) id 2C88D84E77; Mon, 13 Mar 2023 14:18:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.netbsd.org (Postfix) with ESMTP id 5BDA384C2C for ; Mon, 13 Mar 2023 14:18:28 +0000 (UTC) X-Virus-Scanned: amavisd-new at netbsd.org Received: from mail.netbsd.org ([IPv6:::1]) by localhost (mail.netbsd.org [IPv6:::1]) (amavisd-new, port 10025) with ESMTP id 6hOY-G4jBw9b for ; Mon, 13 Mar 2023 14:18:27 +0000 (UTC) Received: from cvs.NetBSD.org (ivanova.NetBSD.org [IPv6:2001:470:a085:999:28c:faff:fe03:5984]) by mail.netbsd.org (Postfix) with ESMTP id 9A3F384E6C for ; Mon, 13 Mar 2023 14:18:27 +0000 (UTC) Received: by cvs.NetBSD.org (Postfix, from userid 500) id 95CBEFA90; Mon, 13 Mar 2023 14:18:27 +0000 (UTC) Content-Transfer-Encoding: 7bit Content-Type: multipart/mixed; boundary="_----------=_1678717107162730" MIME-Version: 1.0 Date: Mon, 13 Mar 2023 14:18:27 +0000 From: "Thomas Klausner" Subject: CVS commit: pkgsrc/textproc/py-sentencepiece To: pkgsrc-changes@NetBSD.org Reply-To: wiz@netbsd.org X-Mailer: log_accum Message-Id: <20230313141827.95CBEFA90@cvs.NetBSD.org> Sender: pkgsrc-changes-owner@NetBSD.org List-Id: Precedence: bulk List-Unsubscribe: This is a multi-part message in MIME format. --_----------=_1678717107162730 Content-Disposition: inline Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII" Module Name: pkgsrc Committed By: wiz Date: Mon Mar 13 14:18:27 UTC 2023 Added Files: pkgsrc/textproc/py-sentencepiece: DESCR Makefile PLIST distinfo Log Message: textproc/py-sentencepiece: import py-sentencepiece-0.1.97 SentencePiece is an unsupervised text tokenizer and detokenizer mainly for Neural Network-based text generation systems where the vocabulary size is predetermined prior to the neural model training. SentencePiece implements subword units (e.g., byte-pair-encoding (BPE)) and unigram language model with the extension of direct training from raw sentences. SentencePiece allows us to make a purely end-to-end system that does not depend on language-specific pre/postprocessing. This package contains the Python module. To generate a diff of this commit: cvs rdiff -u -r0 -r1.1 pkgsrc/textproc/py-sentencepiece/DESCR \ pkgsrc/textproc/py-sentencepiece/Makefile \ pkgsrc/textproc/py-sentencepiece/PLIST \ pkgsrc/textproc/py-sentencepiece/distinfo Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files. --_----------=_1678717107162730 Content-Disposition: inline Content-Length: 3055 Content-Transfer-Encoding: binary Content-Type: text/x-diff; charset=us-ascii Added files: Index: pkgsrc/textproc/py-sentencepiece/DESCR diff -u /dev/null pkgsrc/textproc/py-sentencepiece/DESCR:1.1 --- /dev/null Mon Mar 13 14:18:27 2023 +++ pkgsrc/textproc/py-sentencepiece/DESCR Mon Mar 13 14:18:27 2023 @@ -0,0 +1,10 @@ +SentencePiece is an unsupervised text tokenizer and detokenizer +mainly for Neural Network-based text generation systems where the +vocabulary size is predetermined prior to the neural model training. +SentencePiece implements subword units (e.g., byte-pair-encoding +(BPE)) and unigram language model with the extension of direct +training from raw sentences. SentencePiece allows us to make a +purely end-to-end system that does not depend on language-specific +pre/postprocessing. + +This package contains the Python module. Index: pkgsrc/textproc/py-sentencepiece/Makefile diff -u /dev/null pkgsrc/textproc/py-sentencepiece/Makefile:1.1 --- /dev/null Mon Mar 13 14:18:27 2023 +++ pkgsrc/textproc/py-sentencepiece/Makefile Mon Mar 13 14:18:27 2023 @@ -0,0 +1,12 @@ +# $NetBSD: Makefile,v 1.1 2023/03/13 14:18:27 wiz Exp $ + +PKGNAME= ${PYPKGPREFIX}-${DISTNAME} + +WRKSRC= ${WRKDIR}/${DISTNAME}/python + +USE_TOOLS+= pkg-config + +.include "../../textproc/sentencepiece/Makefile.common" +.include "../../textproc/sentencepiece/buildlink3.mk" +.include "../../lang/python/egg.mk" +.include "../../mk/bsd.pkg.mk" Index: pkgsrc/textproc/py-sentencepiece/PLIST diff -u /dev/null pkgsrc/textproc/py-sentencepiece/PLIST:1.1 --- /dev/null Mon Mar 13 14:18:27 2023 +++ pkgsrc/textproc/py-sentencepiece/PLIST Mon Mar 13 14:18:27 2023 @@ -0,0 +1,18 @@ +@comment $NetBSD: PLIST,v 1.1 2023/03/13 14:18:27 wiz Exp $ +${PYSITELIB}/${EGG_INFODIR}/PKG-INFO +${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt +${PYSITELIB}/${EGG_INFODIR}/dependency_links.txt +${PYSITELIB}/${EGG_INFODIR}/top_level.txt +${PYSITELIB}/sentencepiece/__init__.py +${PYSITELIB}/sentencepiece/__init__.pyc +${PYSITELIB}/sentencepiece/__init__.pyo +${PYSITELIB}/sentencepiece/_sentencepiece.so +${PYSITELIB}/sentencepiece/_version.py +${PYSITELIB}/sentencepiece/_version.pyc +${PYSITELIB}/sentencepiece/_version.pyo +${PYSITELIB}/sentencepiece/sentencepiece_model_pb2.py +${PYSITELIB}/sentencepiece/sentencepiece_model_pb2.pyc +${PYSITELIB}/sentencepiece/sentencepiece_model_pb2.pyo +${PYSITELIB}/sentencepiece/sentencepiece_pb2.py +${PYSITELIB}/sentencepiece/sentencepiece_pb2.pyc +${PYSITELIB}/sentencepiece/sentencepiece_pb2.pyo Index: pkgsrc/textproc/py-sentencepiece/distinfo diff -u /dev/null pkgsrc/textproc/py-sentencepiece/distinfo:1.1 --- /dev/null Mon Mar 13 14:18:27 2023 +++ pkgsrc/textproc/py-sentencepiece/distinfo Mon Mar 13 14:18:27 2023 @@ -0,0 +1,5 @@ +$NetBSD: distinfo,v 1.1 2023/03/13 14:18:27 wiz Exp $ + +BLAKE2s (sentencepiece-0.1.97.tar.gz) = 969788b6d87e8c992f6df4349f984fb2d6e80f978d4007127174222ec7fcb3ab +SHA512 (sentencepiece-0.1.97.tar.gz) = 4c35488e3661e45be677b04299c0d0b1f0d46421098f0b1625a1bb5e7725d175dfd55328a5a7bbf88badeb03c2ba087aef942b0d7520a29f6bf34eae211a99eb +Size (sentencepiece-0.1.97.tar.gz) = 11945436 bytes --_----------=_1678717107162730--