Received: by mail.netbsd.org (Postfix, from userid 605) id DE7BE850C1; Wed, 7 Apr 2021 11:57:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.netbsd.org (Postfix) with ESMTP id 1D06684DFF for ; Wed, 7 Apr 2021 11:57:32 +0000 (UTC) X-Virus-Scanned: amavisd-new at netbsd.org Received: from mail.netbsd.org ([IPv6:::1]) by localhost (mail.netbsd.org [IPv6:::1]) (amavisd-new, port 10025) with ESMTP id Fr7DYBMlW83y for ; Wed, 7 Apr 2021 11:57:31 +0000 (UTC) Received: from cvs.NetBSD.org (ivanova.netbsd.org [199.233.217.197]) by mail.netbsd.org (Postfix) with ESMTP id 261B284D73 for ; Wed, 7 Apr 2021 11:57:31 +0000 (UTC) Received: by cvs.NetBSD.org (Postfix, from userid 500) id 1F87DFA95; Wed, 7 Apr 2021 11:57:31 +0000 (UTC) Content-Transfer-Encoding: 7bit Content-Type: multipart/mixed; boundary="_----------=_1617796651156540" MIME-Version: 1.0 Date: Wed, 7 Apr 2021 11:57:31 +0000 From: "Dr. Thomas Orgis" Subject: CVS commit: pkgsrc/math/py-numpy To: pkgsrc-changes@NetBSD.org Reply-To: thor@netbsd.org X-Mailer: log_accum Message-Id: <20210407115731.1F87DFA95@cvs.NetBSD.org> Sender: pkgsrc-changes-owner@NetBSD.org List-Id: Precedence: bulk List-Unsubscribe: This is a multi-part message in MIME format. --_----------=_1617796651156540 Content-Disposition: inline Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="UTF-8" Module Name: pkgsrc Committed By: thor Date: Wed Apr 7 11:57:31 UTC 2021 Modified Files: pkgsrc/math/py-numpy: Makefile distinfo pkgsrc/math/py-numpy/patches: patch-numpy_distutils_system__info.py Log Message: py-numpy: updated the BLAS patch for more likely upstream inclusion This introduces a 'generic' variant for blas_opt_info() and lapack_opt_info() that looks into BLAS_LIBS and LAPACK_LIBS. The others stay alive, and you can choose via NPY_BLAS_ORDER and NPY_LAPACK_ORDER. Will post this upstream … hoping for the best. Other projects seem just to abuse the [openblas] entry in site.cfg, or whatever hacks to trick this into building with a generic BLAS. One hurdle is that using openblas seems to assume that CBLAS is present in there, which it is not in pkgsrc. We could think about introducing some alternatives stuff that uses stubs for all the related libs anyway, but that is not my application, which builds things from source. For binary packages, it would be interesting, though. See Debian and Genoot for prior art on that. To generate a diff of this commit: cvs rdiff -u -r1.76 -r1.77 pkgsrc/math/py-numpy/Makefile cvs rdiff -u -r1.58 -r1.59 pkgsrc/math/py-numpy/distinfo cvs rdiff -u -r1.5 -r1.6 \ pkgsrc/math/py-numpy/patches/patch-numpy_distutils_system__info.py Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files. --_----------=_1617796651156540 Content-Disposition: inline Content-Length: 10014 Content-Transfer-Encoding: binary Content-Type: text/x-diff; charset=us-ascii Modified files: Index: pkgsrc/math/py-numpy/Makefile diff -u pkgsrc/math/py-numpy/Makefile:1.76 pkgsrc/math/py-numpy/Makefile:1.77 --- pkgsrc/math/py-numpy/Makefile:1.76 Fri Mar 26 20:34:28 2021 +++ pkgsrc/math/py-numpy/Makefile Wed Apr 7 11:57:30 2021 @@ -1,7 +1,7 @@ -# $NetBSD: Makefile,v 1.76 2021/03/26 20:34:28 thor Exp $ +# $NetBSD: Makefile,v 1.77 2021/04/07 11:57:30 thor Exp $ DISTNAME= numpy-1.19.5 -PKGREVISION= 2 +PKGREVISION= 3 PKGNAME= ${PYPKGPREFIX}-${DISTNAME} CATEGORIES= math python MASTER_SITES= ${MASTER_SITE_PYPI:=n/numpy/} @@ -18,8 +18,11 @@ TEST_DEPENDS+= ${PYPKGPREFIX}-test-[0-9] # error: 'for' loop initial declarations are only allowed in C99 mode USE_LANGUAGES= c99 fortran -MAKE_ENV+= ATLAS=None +# Use our BLAS. The patch adds 'generic' as first entry anyway, but we +# enforce it to fail early and for the case that upstream accepts the +# patch, but with changed default order. MAKE_ENV+= BLAS_LIBS=${BLAS_LIBS:Q} LAPACK_LIBS=${LAPACK_LIBS:Q} +MAKE_ENV+= NPY_BLAS_ORDER=generic NPY_LAPACK_ORDER=generic REPLACE_PYTHON+= *.py */*.py */*/*.py */*/*/*.py */*/*/*/*.py Index: pkgsrc/math/py-numpy/distinfo diff -u pkgsrc/math/py-numpy/distinfo:1.58 pkgsrc/math/py-numpy/distinfo:1.59 --- pkgsrc/math/py-numpy/distinfo:1.58 Fri Mar 26 20:34:28 2021 +++ pkgsrc/math/py-numpy/distinfo Wed Apr 7 11:57:30 2021 @@ -1,4 +1,4 @@ -$NetBSD: distinfo,v 1.58 2021/03/26 20:34:28 thor Exp $ +$NetBSD: distinfo,v 1.59 2021/04/07 11:57:30 thor Exp $ SHA1 (numpy-1.19.5.zip) = 61f0b3dad58ce97b14da9dccbee0722d36f26937 RMD160 (numpy-1.19.5.zip) = 3317c98790e8c2d1d9c36279b3451f09b6776935 @@ -7,5 +7,5 @@ Size (numpy-1.19.5.zip) = 7318340 bytes SHA1 (patch-numpy_distutils_fcompiler_____init____.py) = 49d070da5b48bd9818b37ac3254341fa68503c53 SHA1 (patch-numpy_distutils_fcompiler_g95.py) = be73b64a3e551df998b6a904d6db762bf28a98ed SHA1 (patch-numpy_distutils_fcompiler_gnu.py) = 1d62e1c5e35de0f2cf975de38f62df7f10c71d74 -SHA1 (patch-numpy_distutils_system__info.py) = e01c0ca1a7151f77ac9cfbccd44da44a0d354931 +SHA1 (patch-numpy_distutils_system__info.py) = 8322a9f27f61626812509e26c80cb9f9b49ced7e SHA1 (patch-numpy_linalg_lapack__litemodule.c) = e97ec871c2f33c3121b3c8471a9e5a74c3c798c8 Index: pkgsrc/math/py-numpy/patches/patch-numpy_distutils_system__info.py diff -u pkgsrc/math/py-numpy/patches/patch-numpy_distutils_system__info.py:1.5 pkgsrc/math/py-numpy/patches/patch-numpy_distutils_system__info.py:1.6 --- pkgsrc/math/py-numpy/patches/patch-numpy_distutils_system__info.py:1.5 Fri Mar 26 20:34:28 2021 +++ pkgsrc/math/py-numpy/patches/patch-numpy_distutils_system__info.py Wed Apr 7 11:57:30 2021 @@ -1,102 +1,78 @@ -$NetBSD: patch-numpy_distutils_system__info.py,v 1.5 2021/03/26 20:34:28 thor Exp $ +$NetBSD: patch-numpy_distutils_system__info.py,v 1.6 2021/04/07 11:57:30 thor Exp $ -Disable openblas detection. In pkgsrc, use mk/blas.buildlink.mk. +Introduce new option 'generic' for BLAS and LAPACK +TODO: The same for 64 bit offset versions, but we'd need repspective BLAS builds +first in pkgsrc to test. ---- numpy/distutils/system_info.py.orig 2020-06-02 05:24:58.000000000 +0000 +--- numpy/distutils/system_info.py.orig 2021-01-04 14:16:38.000000000 +0000 +++ numpy/distutils/system_info.py -@@ -1730,34 +1722,18 @@ class lapack_opt_info(system_info): +@@ -114,6 +114,13 @@ Currently, the following classes are ava + x11_info:x11 + xft_info:xft + ++Note that blas_opt_info and lapack_opt_info honor the NPY_BLAS_ORDER and NPY_LAPACK_ORDER ++environment variables to select a specific implementation. One possible implementation ++is 'generic', which relies on the environment providing BLAS_LIBS and LAPACK_LIBS to ++link to the customary plain f77 interface, supporting any standard-conforming BLAS ++and LAPACK implementation (which might be different between build-time and run-time, ++even). ++ + Example: + ---------- + [DEFAULT] +@@ -1651,7 +1658,7 @@ def get_atlas_version(**config): + class lapack_opt_info(system_info): + notfounderror = LapackNotFoundError + # List of all known BLAS libraries, in the default order +- lapack_order = ['mkl', 'openblas', 'flame', 'atlas', 'accelerate', 'lapack'] ++ lapack_order = ['generic', 'mkl', 'openblas', 'flame', 'atlas', 'accelerate', 'lapack'] + order_env_var_name = 'NPY_LAPACK_ORDER' + + def _calc_info_mkl(self): +@@ -1744,6 +1751,18 @@ class lapack_opt_info(system_info): + return True + return False + ++ def _calc_info_generic(self): ++ if 'LAPACK_LIBS' in os.environ: ++ info = {} ++ info['language'] = 'f77' ++ info['libraries'] = [] ++ info['include_dirs'] = [] ++ info['define_macros'] = [] ++ info['extra_link_args'] = os.environ['LAPACK_LIBS'].split() ++ self.set_info(**info) ++ return True ++ return False ++ + def _calc_info(self, name): return getattr(self, '_calc_info_{}'.format(name))() - def calc_info(self): -- user_order = os.environ.get(self.order_env_var_name, None) -- if user_order is None: -- lapack_order = self.lapack_order -- else: -- # the user has requested the order of the -- # check they are all in the available list, a COMMA SEPARATED list -- user_order = user_order.lower().split(',') -- non_existing = [] -- lapack_order = [] -- for order in user_order: -- if order in self.lapack_order: -- lapack_order.append(order) -- elif len(order) > 0: -- non_existing.append(order) -- if len(non_existing) > 0: -- raise ValueError("lapack_opt_info user defined " -- "LAPACK order has unacceptable " -- "values: {}".format(non_existing)) -- -- for lapack in lapack_order: -- if self._calc_info(lapack): -- return -- -- if 'lapack' not in lapack_order: -- # Since the user may request *not* to use any library, we still need -- # to raise warnings to signal missing packages! -- warnings.warn(LapackNotFoundError.__doc__ or '', stacklevel=2) -- warnings.warn(LapackSrcNotFoundError.__doc__ or '', stacklevel=2) -+ # Fixing usage of LAPACK specified in LAPACK_LIBS. -+ # Existence of LAPACK_LIBS is mandatory. Things shall break early -+ # if it is not set. -+ info = {} -+ info['language'] = 'f77' -+ info['libraries'] = [] -+ info['include_dirs'] = [] -+ info['define_macros'] = [] -+ info['extra_link_args'] = os.environ['LAPACK_LIBS'].split() -+ -+ self.set_info(**info) -+ return - - - class _ilp64_opt_info_mixin: -@@ -1875,32 +1848,22 @@ class blas_opt_info(system_info): +@@ -1823,7 +1842,7 @@ class lapack64__opt_info(lapack_ilp64_op + class blas_opt_info(system_info): + notfounderror = BlasNotFoundError + # List of all known BLAS libraries, in the default order +- blas_order = ['mkl', 'blis', 'openblas', 'atlas', 'accelerate', 'blas'] ++ blas_order = ['generic', 'mkl', 'blis', 'openblas', 'atlas', 'accelerate', 'blas'] + order_env_var_name = 'NPY_BLAS_ORDER' + + def _calc_info_mkl(self): +@@ -1889,6 +1908,18 @@ class blas_opt_info(system_info): + self.set_info(**info) + return True + ++ def _calc_info_generic(self): ++ if 'BLAS_LIBS' in os.environ: ++ info = {} ++ info['language'] = 'f77' ++ info['libraries'] = [] ++ info['include_dirs'] = [] ++ info['define_macros'] = [] ++ info['extra_link_args'] = os.environ['BLAS_LIBS'].split() ++ self.set_info(**info) ++ return True ++ return False ++ + def _calc_info(self, name): return getattr(self, '_calc_info_{}'.format(name))() - def calc_info(self): -- user_order = os.environ.get(self.order_env_var_name, None) -- if user_order is None: -- blas_order = self.blas_order -- else: -- # the user has requested the order of the -- # check they are all in the available list -- user_order = user_order.lower().split(',') -- non_existing = [] -- blas_order = [] -- for order in user_order: -- if order in self.blas_order: -- blas_order.append(order) -- elif len(order) > 0: -- non_existing.append(order) -- if len(non_existing) > 0: -- raise ValueError("blas_opt_info user defined BLAS order has unacceptable values: {}".format(non_existing)) -- -- for blas in blas_order: -- if self._calc_info(blas): -- return -- -- if 'blas' not in blas_order: -- # Since the user may request *not* to use any library, we still need -- # to raise warnings to signal missing packages! -- warnings.warn(BlasNotFoundError.__doc__ or '', stacklevel=2) -- warnings.warn(BlasSrcNotFoundError.__doc__ or '', stacklevel=2) -+ # Fixing usage of libcblas and the BLAS specified in BLAS_LIBS. -+ # Existence of BLAS_LIBS is mandatory. Things shall break early -+ # if it is not set. -+ info = {} -+ # We assume a generic BLAS, which is a Fortran lib. -+ info['language'] = 'f77' -+ # Try to work without cblas, just link BLAS_LIBS. -+ #info['libraries'] = ['cblas'] -+ #info['define_macros'] = [('HAVE_CBLAS', None)] -+ info['libraries'] = [] -+ info['include_dirs'] = [] -+ info['define_macros'] = [] -+ info['extra_link_args'] = os.environ['BLAS_LIBS'].split() -+ -+ self.set_info(**info) -+ return - - - class blas_ilp64_opt_info(blas_opt_info, _ilp64_opt_info_mixin): --_----------=_1617796651156540--