Sat May 30 19:25:38 2020 UTC ()
doc/pkgsrc.*: regen


(rillig)
diff -r1.293 -r1.294 pkgsrc/doc/pkgsrc.html
diff -r1.291 -r1.292 pkgsrc/doc/pkgsrc.txt
cvs diff -r1.293 -r1.294 pkgsrc/doc/pkgsrc.html (expand / switch to unified diff)

--- pkgsrc/doc/pkgsrc.html 2020/05/22 18:51:44 1.293
+++ pkgsrc/doc/pkgsrc.html 2020/05/30 19:25:38 1.294
@@ -116,38 +116,35 @@ @@ -116,38 +116,35 @@
116<dd><dl> 116<dd><dl>
117<dt><span class="sect1"><a href="#building-a-single-binary-package">7.1. Building a single binary package</a></span></dt> 117<dt><span class="sect1"><a href="#building-a-single-binary-package">7.1. Building a single binary package</a></span></dt>
118<dt><span class="sect1"><a href="#settings-for-creationg-of-binary-packages">7.2. Settings for creation of binary packages</a></span></dt> 118<dt><span class="sect1"><a href="#settings-for-creationg-of-binary-packages">7.2. Settings for creation of binary packages</a></span></dt>
119</dl></dd> 119</dl></dd>
120<dt><span class="chapter"><a href="#bulk">8. Creating binary packages for everything in pkgsrc (bulk 120<dt><span class="chapter"><a href="#bulk">8. Creating binary packages for everything in pkgsrc (bulk
121builds)</a></span></dt> 121builds)</a></span></dt>
122<dd><dl> 122<dd><dl>
123<dt><span class="sect1"><a href="#bulk.pre">8.1. Preparations</a></span></dt> 123<dt><span class="sect1"><a href="#bulk.pre">8.1. Preparations</a></span></dt>
124<dt><span class="sect1"><a href="#bulk.pbulk">8.2. Running a pbulk-style bulk build</a></span></dt> 124<dt><span class="sect1"><a href="#bulk.pbulk">8.2. Running a pbulk-style bulk build</a></span></dt>
125<dd><dl><dt><span class="sect2"><a href="#bulk.pbulk.conf">8.2.1. Configuration</a></span></dt></dl></dd> 125<dd><dl><dt><span class="sect2"><a href="#bulk.pbulk.conf">8.2.1. Configuration</a></span></dt></dl></dd>
126<dt><span class="sect1"><a href="#bulk.req">8.3. Requirements of a full bulk build</a></span></dt> 126<dt><span class="sect1"><a href="#bulk.req">8.3. Requirements of a full bulk build</a></span></dt>
127<dt><span class="sect1"><a href="#bulk.var">8.4. Bulk build variants</a></span></dt> 127<dt><span class="sect1"><a href="#bulk.var">8.4. Bulk build variants</a></span></dt>
128<dd><dl> 128<dd><dl>
129<dt><span class="sect2"><a href="#bulk.var.subst_noop">8.4.1. Strict SUBST blocks</a></span></dt> 129<dt><span class="sect2"><a href="#bulk.var.confopt">8.4.1. Detect unknown configure options</a></span></dt>
130<dt><span class="sect2"><a href="#bulk.var.confopt">8.4.2. Detect unknown configure options</a></span></dt> 130<dt><span class="sect2"><a href="#bulk.var.comperr">8.4.2. Detect classes of bugs by forcing compiler warnings</a></span></dt>
131<dt><span class="sect2"><a href="#bulk.var.comperr">8.4.3. Detect classes of bugs by forcing compiler warnings</a></span></dt> 131<dt><span class="sect2"><a href="#bulk.var.dirs">8.4.3. Use custom directories</a></span></dt>
132<dt><span class="sect2"><a href="#bulk.var.dirs">8.4.4. Use custom directories</a></span></dt> 132<dt><span class="sect2"><a href="#bulk.var.warn">8.4.4. Turn warnings into errors</a></span></dt>
133<dt><span class="sect2"><a href="#bulk.var.warn">8.4.5. Turn warnings into errors</a></span></dt> 133<dt><span class="sect2"><a href="#bulk.var.pkglint">8.4.5. Reject packages for which pkglint reports errors</a></span></dt>
134<dt><span class="sect2"><a href="#bulk.var.pkglint">8.4.6. Reject packages for which pkglint reports errors</a></span></dt> 134<dt><span class="sect2"><a href="#bulk.var.strings">8.4.6. Reject packages that contain forbidden strings</a></span></dt>
135<dt><span class="sect2"><a href="#bulk.var.strings">8.4.7. Reject packages that contain forbidden strings</a></span></dt> 135<dt><span class="sect2"><a href="#bulk.var.test">8.4.7. Reject packages whose self-test fails</a></span></dt>
136<dt><span class="sect2"><a href="#bulk.var.test">8.4.8. Reject packages whose self-test fails</a></span></dt> 136<dt><span class="sect2"><a href="#bulk.var.shvar">8.4.8. Reject packages that use undefined shell variables</a></span></dt>
137<dt><span class="sect2"><a href="#bulk.var.shvar">8.4.9. Reject packages that use undefined shell variables</a></span></dt> 137<dt><span class="sect2"><a href="#bulk.var.quiet">8.4.9. Turn off verbose logging</a></span></dt>
138<dt><span class="sect2"><a href="#bulk.var.quiet">8.4.10. Turn off verbose logging</a></span></dt> 
139<dt><span class="sect2"><a href="#bulk.var.options">8.4.11. Select random sets of options</a></span></dt> 
140<dt><span class="sect2"><a href="#bulk.var.build_defs">8.4.12. Select random configurations of BUILD_DEFS</a></span></dt> 
141</dl></dd> 138</dl></dd>
142<dt><span class="sect1"><a href="#creating-cdroms">8.5. Creating a multiple CD-ROM packages collection</a></span></dt> 139<dt><span class="sect1"><a href="#creating-cdroms">8.5. Creating a multiple CD-ROM packages collection</a></span></dt>
143<dd><dl><dt><span class="sect2"><a href="#cdpack-example">8.5.1. Example of cdpack</a></span></dt></dl></dd> 140<dd><dl><dt><span class="sect2"><a href="#cdpack-example">8.5.1. Example of cdpack</a></span></dt></dl></dd>
144</dl></dd> 141</dl></dd>
145<dt><span class="chapter"><a href="#files">9. Directory layout of the installed files</a></span></dt> 142<dt><span class="chapter"><a href="#files">9. Directory layout of the installed files</a></span></dt>
146<dd><dl> 143<dd><dl>
147<dt><span class="sect1"><a href="#files.localbase">9.1. File system layout in <code class="literal">${LOCALBASE}</code></a></span></dt> 144<dt><span class="sect1"><a href="#files.localbase">9.1. File system layout in <code class="literal">${LOCALBASE}</code></a></span></dt>
148<dt><span class="sect1"><a href="#files.varbase">9.2. File system layout in <code class="literal">${VARBASE}</code></a></span></dt> 145<dt><span class="sect1"><a href="#files.varbase">9.2. File system layout in <code class="literal">${VARBASE}</code></a></span></dt>
149</dl></dd> 146</dl></dd>
150<dt><span class="chapter"><a href="#faq">10. Frequently Asked Questions</a></span></dt> 147<dt><span class="chapter"><a href="#faq">10. Frequently Asked Questions</a></span></dt>
151<dd><dl> 148<dd><dl>
152<dt><span class="sect1"><a href="#mailing-list-pointers">10.1. Are there any mailing lists for pkg-related discussion?</a></span></dt> 149<dt><span class="sect1"><a href="#mailing-list-pointers">10.1. Are there any mailing lists for pkg-related discussion?</a></span></dt>
153<dt><span class="sect1"><a href="#faq-pkgtools">10.2. Utilities for package management (pkgtools)</a></span></dt> 150<dt><span class="sect1"><a href="#faq-pkgtools">10.2. Utilities for package management (pkgtools)</a></span></dt>
@@ -876,38 +873,35 @@ minutes!

 @@ -876,38 +873,35 @@ minutes!

876<dd><dl> 873<dd><dl>
877<dt><span class="sect1"><a href="#building-a-single-binary-package">7.1. Building a single binary package</a></span></dt> 874<dt><span class="sect1"><a href="#building-a-single-binary-package">7.1. Building a single binary package</a></span></dt>
878<dt><span class="sect1"><a href="#settings-for-creationg-of-binary-packages">7.2. Settings for creation of binary packages</a></span></dt> 875<dt><span class="sect1"><a href="#settings-for-creationg-of-binary-packages">7.2. Settings for creation of binary packages</a></span></dt>
879</dl></dd> 876</dl></dd>
880<dt><span class="chapter"><a href="#bulk">8. Creating binary packages for everything in pkgsrc (bulk 877<dt><span class="chapter"><a href="#bulk">8. Creating binary packages for everything in pkgsrc (bulk
881builds)</a></span></dt> 878builds)</a></span></dt>
882<dd><dl> 879<dd><dl>
883<dt><span class="sect1"><a href="#bulk.pre">8.1. Preparations</a></span></dt> 880<dt><span class="sect1"><a href="#bulk.pre">8.1. Preparations</a></span></dt>
884<dt><span class="sect1"><a href="#bulk.pbulk">8.2. Running a pbulk-style bulk build</a></span></dt> 881<dt><span class="sect1"><a href="#bulk.pbulk">8.2. Running a pbulk-style bulk build</a></span></dt>
885<dd><dl><dt><span class="sect2"><a href="#bulk.pbulk.conf">8.2.1. Configuration</a></span></dt></dl></dd> 882<dd><dl><dt><span class="sect2"><a href="#bulk.pbulk.conf">8.2.1. Configuration</a></span></dt></dl></dd>
886<dt><span class="sect1"><a href="#bulk.req">8.3. Requirements of a full bulk build</a></span></dt> 883<dt><span class="sect1"><a href="#bulk.req">8.3. Requirements of a full bulk build</a></span></dt>
887<dt><span class="sect1"><a href="#bulk.var">8.4. Bulk build variants</a></span></dt> 884<dt><span class="sect1"><a href="#bulk.var">8.4. Bulk build variants</a></span></dt>
888<dd><dl> 885<dd><dl>
889<dt><span class="sect2"><a href="#bulk.var.subst_noop">8.4.1. Strict SUBST blocks</a></span></dt> 886<dt><span class="sect2"><a href="#bulk.var.confopt">8.4.1. Detect unknown configure options</a></span></dt>
890<dt><span class="sect2"><a href="#bulk.var.confopt">8.4.2. Detect unknown configure options</a></span></dt> 887<dt><span class="sect2"><a href="#bulk.var.comperr">8.4.2. Detect classes of bugs by forcing compiler warnings</a></span></dt>
891<dt><span class="sect2"><a href="#bulk.var.comperr">8.4.3. Detect classes of bugs by forcing compiler warnings</a></span></dt> 888<dt><span class="sect2"><a href="#bulk.var.dirs">8.4.3. Use custom directories</a></span></dt>
892<dt><span class="sect2"><a href="#bulk.var.dirs">8.4.4. Use custom directories</a></span></dt> 889<dt><span class="sect2"><a href="#bulk.var.warn">8.4.4. Turn warnings into errors</a></span></dt>
893<dt><span class="sect2"><a href="#bulk.var.warn">8.4.5. Turn warnings into errors</a></span></dt> 890<dt><span class="sect2"><a href="#bulk.var.pkglint">8.4.5. Reject packages for which pkglint reports errors</a></span></dt>
894<dt><span class="sect2"><a href="#bulk.var.pkglint">8.4.6. Reject packages for which pkglint reports errors</a></span></dt> 891<dt><span class="sect2"><a href="#bulk.var.strings">8.4.6. Reject packages that contain forbidden strings</a></span></dt>
895<dt><span class="sect2"><a href="#bulk.var.strings">8.4.7. Reject packages that contain forbidden strings</a></span></dt> 892<dt><span class="sect2"><a href="#bulk.var.test">8.4.7. Reject packages whose self-test fails</a></span></dt>
896<dt><span class="sect2"><a href="#bulk.var.test">8.4.8. Reject packages whose self-test fails</a></span></dt> 893<dt><span class="sect2"><a href="#bulk.var.shvar">8.4.8. Reject packages that use undefined shell variables</a></span></dt>
897<dt><span class="sect2"><a href="#bulk.var.shvar">8.4.9. Reject packages that use undefined shell variables</a></span></dt> 894<dt><span class="sect2"><a href="#bulk.var.quiet">8.4.9. Turn off verbose logging</a></span></dt>
898<dt><span class="sect2"><a href="#bulk.var.quiet">8.4.10. Turn off verbose logging</a></span></dt> 
899<dt><span class="sect2"><a href="#bulk.var.options">8.4.11. Select random sets of options</a></span></dt> 
900<dt><span class="sect2"><a href="#bulk.var.build_defs">8.4.12. Select random configurations of BUILD_DEFS</a></span></dt> 
901</dl></dd> 895</dl></dd>
902<dt><span class="sect1"><a href="#creating-cdroms">8.5. Creating a multiple CD-ROM packages collection</a></span></dt> 896<dt><span class="sect1"><a href="#creating-cdroms">8.5. Creating a multiple CD-ROM packages collection</a></span></dt>
903<dd><dl><dt><span class="sect2"><a href="#cdpack-example">8.5.1. Example of cdpack</a></span></dt></dl></dd> 897<dd><dl><dt><span class="sect2"><a href="#cdpack-example">8.5.1. Example of cdpack</a></span></dt></dl></dd>
904</dl></dd> 898</dl></dd>
905<dt><span class="chapter"><a href="#files">9. Directory layout of the installed files</a></span></dt> 899<dt><span class="chapter"><a href="#files">9. Directory layout of the installed files</a></span></dt>
906<dd><dl> 900<dd><dl>
907<dt><span class="sect1"><a href="#files.localbase">9.1. File system layout in <code class="literal">${LOCALBASE}</code></a></span></dt> 901<dt><span class="sect1"><a href="#files.localbase">9.1. File system layout in <code class="literal">${LOCALBASE}</code></a></span></dt>
908<dt><span class="sect1"><a href="#files.varbase">9.2. File system layout in <code class="literal">${VARBASE}</code></a></span></dt> 902<dt><span class="sect1"><a href="#files.varbase">9.2. File system layout in <code class="literal">${VARBASE}</code></a></span></dt>
909</dl></dd> 903</dl></dd>
910<dt><span class="chapter"><a href="#faq">10. Frequently Asked Questions</a></span></dt> 904<dt><span class="chapter"><a href="#faq">10. Frequently Asked Questions</a></span></dt>
911<dd><dl> 905<dd><dl>
912<dt><span class="sect1"><a href="#mailing-list-pointers">10.1. Are there any mailing lists for pkg-related discussion?</a></span></dt> 906<dt><span class="sect1"><a href="#mailing-list-pointers">10.1. Are there any mailing lists for pkg-related discussion?</a></span></dt>
913<dt><span class="sect1"><a href="#faq-pkgtools">10.2. Utilities for package management (pkgtools)</a></span></dt> 907<dt><span class="sect1"><a href="#faq-pkgtools">10.2. Utilities for package management (pkgtools)</a></span></dt>
@@ -1240,71 +1234,73 @@ release -d @@ -1240,71 +1234,73 @@ release -d
1240<dt><span class="sect2"><a href="#how-to-build-and-install">5.2.3. How to build and install</a></span></dt> 1234<dt><span class="sect2"><a href="#how-to-build-and-install">5.2.3. How to build and install</a></span></dt>
1241</dl></dd> 1235</dl></dd>
1242</dl> 1236</dl>
1243</div> 1237</div>
1244<p>Basically, there are two ways of using pkgsrc. The first 1238<p>Basically, there are two ways of using pkgsrc. The first
1245is to only install the package tools and to use binary packages 1239is to only install the package tools and to use binary packages
1246that someone else has prepared. This is the <span class="quote">&#8220;<span class="quote">pkg</span>&#8221;</span> 1240that someone else has prepared. This is the <span class="quote">&#8220;<span class="quote">pkg</span>&#8221;</span>
1247in pkgsrc. The second way is to install the <span class="quote">&#8220;<span class="quote">src</span>&#8221;</span> 1241in pkgsrc. The second way is to install the <span class="quote">&#8220;<span class="quote">src</span>&#8221;</span>
1248of pkgsrc, too. Then you are able to build your own packages, 1242of pkgsrc, too. Then you are able to build your own packages,
1249and you can still use binary packages from someone else.</p> 1243and you can still use binary packages from someone else.</p>
1250<div class="sect1"> 1244<div class="sect1">
1251<div class="titlepage"><div><div><h2 class="title" style="clear: both"> 1245<div class="titlepage"><div><div><h2 class="title" style="clear: both">
1252<a name="using-pkg"></a>5.1. Using binary packages</h2></div></div></div> 1246<a name="using-pkg"></a>5.1. Using binary packages</h2></div></div></div>
1253<p>On the <a class="ulink" href="https://cdn.NetBSD.org/" target="_top">cdn.NetBSD.org</a> 1247<p>On the <a class="ulink" href="http://cdn.NetBSD.org/" target="_top">cdn.NetBSD.org</a>
1254 site and mirrors, there are collections of binary packages, 1248 site and mirrors, there are collections of binary packages,
1255 ready to be installed. These binary packages have been built using the 1249 ready to be installed. These binary packages have been built using the
1256 default settings for the directories, that is:</p> 1250 default settings for the directories, that is:</p>
1257<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 1251<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
1258<li class="listitem"><p><code class="filename">/usr/pkg</code> for <code class="varname">LOCALBASE</code>, where most of the files are installed,</p></li> 1252<li class="listitem"><p><code class="filename">/usr/pkg</code> for <code class="varname">LOCALBASE</code>, where most of the files are installed,</p></li>
1259<li class="listitem"><p><code class="filename">/usr/pkg/etc</code> for configuration files,</p></li> 1253<li class="listitem"><p><code class="filename">/usr/pkg/etc</code> for configuration files,</p></li>
1260<li class="listitem"><p><code class="filename">/var</code> for <code class="varname">VARBASE</code>, where those files are installed that may change after installation.</p></li> 1254<li class="listitem"><p><code class="filename">/var</code> for <code class="varname">VARBASE</code>, where those files are installed that may change after installation.</p></li>
1261</ul></div> 1255</ul></div>
1262<p>If you cannot use these directories for whatever reasons (maybe 1256<p>If you cannot use these directories for whatever reasons (maybe
1263 because you're not root), you cannot use these binary packages, but 1257 because you're not root), you cannot use these binary packages, but
1264 have to build the packages yourself, which is explained in <a class="xref" href="#bootstrapping-pkgsrc" title="4.2. Bootstrapping pkgsrc">Section 4.2, &#8220;Bootstrapping pkgsrc&#8221;</a>.</p> 1258 have to build the packages yourself, which is explained in <a class="xref" href="#bootstrapping-pkgsrc" title="4.2. Bootstrapping pkgsrc">Section 4.2, &#8220;Bootstrapping pkgsrc&#8221;</a>.</p>
1265<div class="sect2"> 1259<div class="sect2">
1266<div class="titlepage"><div><div><h3 class="title"> 1260<div class="titlepage"><div><div><h3 class="title">
1267<a name="finding-binary-packages"></a>5.1.1. Finding binary packages</h3></div></div></div> 1261<a name="finding-binary-packages"></a>5.1.1. Finding binary packages</h3></div></div></div>
1268<p>To install binary packages, you first need to know from where 1262<p>To install binary packages, you first need to know from where
1269 to get them. The first place where you should look is on the main 1263 to get them. The first place where you should look is on the main
1270 pkgsrc FTP server in the directory <a class="ulink" href="https://cdn.NetBSD.org/pub/pkgsrc/packages/" target="_top"><code class="filename">/pub/pkgsrc/packages</code></a>.</p> 1264
 1265 pkgsrc FTP server in the directory <a class="ulink" href="http://cdn.NetBSD.org/pub/pkgsrc/packages/" target="_top"><code class="filename">/pub/pkgsrc/packages</code></a>.</p>
1271<p>This directory contains binary packages for multiple 1266<p>This directory contains binary packages for multiple
1272 platforms. First, select your operating system. (Ignore the 1267 platforms. First, select your operating system. (Ignore the
1273 directories with version numbers attached to it, they just exist for 1268 directories with version numbers attached to it, they just exist for
1274 legacy reasons.) Then, select your hardware architecture, and in the 1269 legacy reasons.) Then, select your hardware architecture, and in the
1275 third step, the OS version and the <span class="quote">&#8220;<span class="quote">version</span>&#8221;</span> of pkgsrc.</p> 1270 third step, the OS version and the <span class="quote">&#8220;<span class="quote">version</span>&#8221;</span> of pkgsrc.</p>
1276<p>In this directory, you often find a file called 1271<p>In this directory, you often find a file called
1277 <code class="filename">bootstrap.tar.gz</code> which contains the package 1272 <code class="filename">bootstrap.tar.gz</code> which contains the package
1278 management tools. If the file is missing, it is likely that your 1273 management tools. If the file is missing, it is likely that your
1279 operating system already provides those tools. Download the file and 1274 operating system already provides those tools. Download the file and
1280 extract it in the <code class="filename">/</code> directory. It will create 1275 extract it in the <code class="filename">/</code> directory. It will create
1281 the directories <code class="filename">/usr/pkg</code> (containing the tools 1276 the directories <code class="filename">/usr/pkg</code> (containing the tools
1282 for managing binary packages and the database of installed packages).</p> 1277 for managing binary packages and the database of installed packages).</p>
1283</div> 1278</div>
1284<div class="sect2"> 1279<div class="sect2">
1285<div class="titlepage"><div><div><h3 class="title"> 1280<div class="titlepage"><div><div><h3 class="title">
1286<a name="installing-binary-packages"></a>5.1.2. Installing binary packages</h3></div></div></div> 1281<a name="installing-binary-packages"></a>5.1.2. Installing binary packages</h3></div></div></div>
1287<p>In the directory from the last section, there is a 1282<p>In the directory from the last section, there is a
1288 subdirectory called <code class="filename">All/</code>, which contains all the 1283 subdirectory called <code class="filename">All/</code>, which contains all the
1289 binary packages that are available for the platform, excluding those 1284 binary packages that are available for the platform, excluding those
1290 that may not be distributed via FTP or CDROM (depending on which 1285 that may not be distributed via FTP or CDROM (depending on which
1291 medium you are using).</p> 1286 medium you are using).</p>
1292<p>To install packages directly from an FTP or HTTP server, run 1287<p>To install packages directly from an FTP or HTTP server, run
1293 the following commands in a Bourne-compatible shell (be sure to 1288 the following commands in a Bourne-compatible shell (be sure to
1294 <span class="command"><strong>su</strong></span> to root first):</p> 1289 <span class="command"><strong>su</strong></span> to root first):</p>
1295<pre class="screen"> 1290<pre class="screen">
1296<code class="prompt">#</code> <strong class="userinput"><code>PATH="/usr/pkg/sbin:$PATH"</code></strong> 1291<code class="prompt">#</code> <strong class="userinput"><code>PATH="/usr/pkg/sbin:$PATH"</code></strong>
1297<code class="prompt">#</code> <strong class="userinput"><code>PKG_PATH="https://cdn.NetBSD.org/pub/pkgsrc/packages"</code></strong> 1292
 1293<code class="prompt">#</code> <strong class="userinput"><code>PKG_PATH="http://cdn.NetBSD.org/pub/pkgsrc/packages"</code></strong>
1298<code class="prompt">#</code> <strong class="userinput"><code>PKG_PATH="$PKG_PATH/<em class="replaceable"><code>OPSYS</code></em>/<em class="replaceable"><code>ARCH</code></em>/<em class="replaceable"><code>VERSIONS</code></em>/All/"</code></strong> 1294<code class="prompt">#</code> <strong class="userinput"><code>PKG_PATH="$PKG_PATH/<em class="replaceable"><code>OPSYS</code></em>/<em class="replaceable"><code>ARCH</code></em>/<em class="replaceable"><code>VERSIONS</code></em>/All/"</code></strong>
1299<code class="prompt">#</code> <strong class="userinput"><code>export PATH PKG_PATH</code></strong> 1295<code class="prompt">#</code> <strong class="userinput"><code>export PATH PKG_PATH</code></strong>
1300</pre> 1296</pre>
1301<p>Instead of URLs, you can also use local paths, for example if 1297<p>Instead of URLs, you can also use local paths, for example if
1302 you are installing from a set of CDROMs, DVDs or an NFS-mounted 1298 you are installing from a set of CDROMs, DVDs or an NFS-mounted
1303 repository. If you want to install packages from multiple sources, 1299 repository. If you want to install packages from multiple sources,
1304 you can separate them by a semicolon in 1300 you can separate them by a semicolon in
1305 <code class="varname">PKG_PATH</code>.</p> 1301 <code class="varname">PKG_PATH</code>.</p>
1306<p>After these preparations, installing a package is very 1302<p>After these preparations, installing a package is very
1307 easy:</p> 1303 easy:</p>
1308<pre class="screen"> 1304<pre class="screen">
1309<code class="prompt">#</code> <strong class="userinput"><code>pkg_add libreoffice</code></strong> 1305<code class="prompt">#</code> <strong class="userinput"><code>pkg_add libreoffice</code></strong>
1310<code class="prompt">#</code> <strong class="userinput"><code>pkg_add ap24-php71-*</code></strong> 1306<code class="prompt">#</code> <strong class="userinput"><code>pkg_add ap24-php71-*</code></strong>
@@ -1350,48 +1346,50 @@ and you can still use binary packages fr @@ -1350,48 +1346,50 @@ and you can still use binary packages fr
1350<div class="sect2"> 1346<div class="sect2">
1351<div class="titlepage"><div><div><h3 class="title"> 1347<div class="titlepage"><div><div><h3 class="title">
1352<a name="using.pkg_info"></a>5.1.4. Getting information about installed packages</h3></div></div></div> 1348<a name="using.pkg_info"></a>5.1.4. Getting information about installed packages</h3></div></div></div>
1353<p>The <span class="command"><strong>pkg_info</strong></span> shows information about 1349<p>The <span class="command"><strong>pkg_info</strong></span> shows information about
1354 installed packages or binary package files.</p> 1350 installed packages or binary package files.</p>
1355</div> 1351</div>
1356<div class="sect2"> 1352<div class="sect2">
1357<div class="titlepage"><div><div><h3 class="title"> 1353<div class="titlepage"><div><div><h3 class="title">
1358<a name="vulnerabilities"></a>5.1.5. Checking for security vulnerabilities in installed packages</h3></div></div></div> 1354<a name="vulnerabilities"></a>5.1.5. Checking for security vulnerabilities in installed packages</h3></div></div></div>
1359<p> 1355<p>
1360 The pkgsrc Security Team and Packages Groups maintain a list of 1356 The pkgsrc Security Team and Packages Groups maintain a list of
1361 known security vulnerabilities to packages which are (or have been) 1357 known security vulnerabilities to packages which are (or have been)
1362 included in pkgsrc. The list is available from the NetBSD 1358 included in pkgsrc. The list is available from the NetBSD
1363 FTP site at <a class="ulink" href="https://ftp.NetBSD.org/pub/NetBSD/packages/vulns/pkg-vulnerabilities" target="_top">https://ftp.NetBSD.org/pub/NetBSD/packages/vulns/pkg-vulnerabilities</a>. 1359
 1360 FTP site at <a class="ulink" href="http://ftp.NetBSD.org/pub/NetBSD/packages/vulns/pkg-vulnerabilities" target="_top">http://ftp.NetBSD.org/pub/NetBSD/packages/vulns/pkg-vulnerabilities</a>.
1364 </p> 1361 </p>
1365<p> 1362<p>
1366 Through <span class="command"><strong>pkg_admin fetch-pkg-vulnerabilities</strong></span>, 1363 Through <span class="command"><strong>pkg_admin fetch-pkg-vulnerabilities</strong></span>,
1367 this list can be downloaded 1364 this list can be downloaded
1368 automatically, and a security audit of all packages installed on a system 1365 automatically, and a security audit of all packages installed on a system
1369 can take place. 1366 can take place.
1370 </p> 1367 </p>
1371<p> 1368<p>
1372 There are two components to auditing. The first 1369 There are two components to auditing. The first
1373 step, <span class="command"><strong>pkg_admin fetch-pkg-vulnerabilities</strong></span>, 1370 step, <span class="command"><strong>pkg_admin fetch-pkg-vulnerabilities</strong></span>,
1374 is for downloading 1371 is for downloading
1375 the list of vulnerabilities from the NetBSD FTP site. The second 1372 the list of vulnerabilities from the NetBSD FTP site. The second
1376 step, <span class="command"><strong>pkg_admin audit</strong></span>, checks to see if any of your 1373 step, <span class="command"><strong>pkg_admin audit</strong></span>, checks to see if any of your
1377 installed packages are vulnerable. If a package is vulnerable, you 1374 installed packages are vulnerable. If a package is vulnerable, you
1378 will see output similar to the following: 1375 will see output similar to the following:
1379 </p> 1376 </p>
1380<pre class="screen">Package samba-2.0.9 has a local-root-shell vulnerability, see 1377<pre class="screen">Package samba-2.0.9 has a local-root-shell vulnerability, see
1381 https://www.samba.org/samba/whatsnew/macroexploit.html</pre> 1378 https://www.samba.org/samba/whatsnew/macroexploit.html</pre>
1382<p> 1379<p>
1383 You may wish to have the 1380 You may wish to have the
1384 <a class="ulink" href="https://ftp.NetBSD.org/pub/pkgsrc/distfiles/vulnerabilities" target="_top">vulnerabilities</a> 1381
 1382 <a class="ulink" href="http://ftp.NetBSD.org/pub/pkgsrc/distfiles/vulnerabilities" target="_top">vulnerabilities</a>
1385 file downloaded daily so that 1383 file downloaded daily so that
1386 it remains current. This may be done by adding an appropriate entry 1384 it remains current. This may be done by adding an appropriate entry
1387 to the root users <a class="citerefentry" href="https://netbsd.gw.com/cgi-bin/man-cgi?crontab+5.i386+NetBSD-9.0"><span class="citerefentry"><span class="refentrytitle">crontab</span>(5)</span></a> entry. For example the entry 1385 to the root users <a class="citerefentry" href="https://netbsd.gw.com/cgi-bin/man-cgi?crontab+5.i386+NetBSD-9.0"><span class="citerefentry"><span class="refentrytitle">crontab</span>(5)</span></a> entry. For example the entry
1388 </p> 1386 </p>
1389<pre class="screen"> 1387<pre class="screen">
1390# Download vulnerabilities file 1388# Download vulnerabilities file
13910 3 * * * /usr/pkg/sbin/pkg_admin fetch-pkg-vulnerabilities &gt;/dev/null 2&gt;&amp;1 13890 3 * * * /usr/pkg/sbin/pkg_admin fetch-pkg-vulnerabilities &gt;/dev/null 2&gt;&amp;1
1392# Audit the installed packages and email results to root 1390# Audit the installed packages and email results to root
13939 3 * * * /usr/pkg/sbin/pkg_admin audit |mail -s "Installed package audit result" \ 13919 3 * * * /usr/pkg/sbin/pkg_admin audit |mail -s "Installed package audit result" \
1394 root &gt;/dev/null 2&gt;&amp;1 1392 root &gt;/dev/null 2&gt;&amp;1
1395 </pre> 1393 </pre>
1396<p> 1394<p>
1397 will update the vulnerability list every day at 3AM, followed by an audit 1395 will update the vulnerability list every day at 3AM, followed by an audit
@@ -2083,38 +2081,35 @@ PKG_OPTIONS.apache= suexec  @@ -2083,38 +2081,35 @@ PKG_OPTIONS.apache= suexec
2083<div class="chapter"> 2081<div class="chapter">
2084<div class="titlepage"><div><div><h2 class="title"> 2082<div class="titlepage"><div><div><h2 class="title">
2085<a name="bulk"></a>Chapter 8. Creating binary packages for everything in pkgsrc (bulk 2083<a name="bulk"></a>Chapter 8. Creating binary packages for everything in pkgsrc (bulk
2086builds)</h2></div></div></div> 2084builds)</h2></div></div></div>
2087<div class="toc"> 2085<div class="toc">
2088<p><b>Table of Contents</b></p> 2086<p><b>Table of Contents</b></p>
2089<dl class="toc"> 2087<dl class="toc">
2090<dt><span class="sect1"><a href="#bulk.pre">8.1. Preparations</a></span></dt> 2088<dt><span class="sect1"><a href="#bulk.pre">8.1. Preparations</a></span></dt>
2091<dt><span class="sect1"><a href="#bulk.pbulk">8.2. Running a pbulk-style bulk build</a></span></dt> 2089<dt><span class="sect1"><a href="#bulk.pbulk">8.2. Running a pbulk-style bulk build</a></span></dt>
2092<dd><dl><dt><span class="sect2"><a href="#bulk.pbulk.conf">8.2.1. Configuration</a></span></dt></dl></dd> 2090<dd><dl><dt><span class="sect2"><a href="#bulk.pbulk.conf">8.2.1. Configuration</a></span></dt></dl></dd>
2093<dt><span class="sect1"><a href="#bulk.req">8.3. Requirements of a full bulk build</a></span></dt> 2091<dt><span class="sect1"><a href="#bulk.req">8.3. Requirements of a full bulk build</a></span></dt>
2094<dt><span class="sect1"><a href="#bulk.var">8.4. Bulk build variants</a></span></dt> 2092<dt><span class="sect1"><a href="#bulk.var">8.4. Bulk build variants</a></span></dt>
2095<dd><dl> 2093<dd><dl>
2096<dt><span class="sect2"><a href="#bulk.var.subst_noop">8.4.1. Strict SUBST blocks</a></span></dt> 2094<dt><span class="sect2"><a href="#bulk.var.confopt">8.4.1. Detect unknown configure options</a></span></dt>
2097<dt><span class="sect2"><a href="#bulk.var.confopt">8.4.2. Detect unknown configure options</a></span></dt> 2095<dt><span class="sect2"><a href="#bulk.var.comperr">8.4.2. Detect classes of bugs by forcing compiler warnings</a></span></dt>
2098<dt><span class="sect2"><a href="#bulk.var.comperr">8.4.3. Detect classes of bugs by forcing compiler warnings</a></span></dt> 2096<dt><span class="sect2"><a href="#bulk.var.dirs">8.4.3. Use custom directories</a></span></dt>
2099<dt><span class="sect2"><a href="#bulk.var.dirs">8.4.4. Use custom directories</a></span></dt> 2097<dt><span class="sect2"><a href="#bulk.var.warn">8.4.4. Turn warnings into errors</a></span></dt>
2100<dt><span class="sect2"><a href="#bulk.var.warn">8.4.5. Turn warnings into errors</a></span></dt> 2098<dt><span class="sect2"><a href="#bulk.var.pkglint">8.4.5. Reject packages for which pkglint reports errors</a></span></dt>
2101<dt><span class="sect2"><a href="#bulk.var.pkglint">8.4.6. Reject packages for which pkglint reports errors</a></span></dt> 2099<dt><span class="sect2"><a href="#bulk.var.strings">8.4.6. Reject packages that contain forbidden strings</a></span></dt>
2102<dt><span class="sect2"><a href="#bulk.var.strings">8.4.7. Reject packages that contain forbidden strings</a></span></dt> 2100<dt><span class="sect2"><a href="#bulk.var.test">8.4.7. Reject packages whose self-test fails</a></span></dt>
2103<dt><span class="sect2"><a href="#bulk.var.test">8.4.8. Reject packages whose self-test fails</a></span></dt> 2101<dt><span class="sect2"><a href="#bulk.var.shvar">8.4.8. Reject packages that use undefined shell variables</a></span></dt>
2104<dt><span class="sect2"><a href="#bulk.var.shvar">8.4.9. Reject packages that use undefined shell variables</a></span></dt> 2102<dt><span class="sect2"><a href="#bulk.var.quiet">8.4.9. Turn off verbose logging</a></span></dt>
2105<dt><span class="sect2"><a href="#bulk.var.quiet">8.4.10. Turn off verbose logging</a></span></dt> 
2106<dt><span class="sect2"><a href="#bulk.var.options">8.4.11. Select random sets of options</a></span></dt> 
2107<dt><span class="sect2"><a href="#bulk.var.build_defs">8.4.12. Select random configurations of BUILD_DEFS</a></span></dt> 
2108</dl></dd> 2103</dl></dd>
2109<dt><span class="sect1"><a href="#creating-cdroms">8.5. Creating a multiple CD-ROM packages collection</a></span></dt> 2104<dt><span class="sect1"><a href="#creating-cdroms">8.5. Creating a multiple CD-ROM packages collection</a></span></dt>
2110<dd><dl><dt><span class="sect2"><a href="#cdpack-example">8.5.1. Example of cdpack</a></span></dt></dl></dd> 2105<dd><dl><dt><span class="sect2"><a href="#cdpack-example">8.5.1. Example of cdpack</a></span></dt></dl></dd>
2111</dl> 2106</dl>
2112</div> 2107</div>
2113<p>For a number of reasons, you may want to build binary packages 2108<p>For a number of reasons, you may want to build binary packages
2114for a large selected set of packages in pkgsrc, or even for all pkgsrc packages. 2109for a large selected set of packages in pkgsrc, or even for all pkgsrc packages.
2115For instance, when you have multiple machines that should run the same software, 2110For instance, when you have multiple machines that should run the same software,
2116it is wasted time if they all build their packages themselves from source. 2111it is wasted time if they all build their packages themselves from source.
2117Or you may want to build a list of packages you want and check them before 2112Or you may want to build a list of packages you want and check them before
2118deploying onto production systems. 2113deploying onto production systems.
2119There is a way of getting a set of binary packages: 2114There is a way of getting a set of binary packages:
2120the bulk build system, or pbulk ("p" stands for "parallel"). 2115the bulk build system, or pbulk ("p" stands for "parallel").
@@ -2228,236 +2223,180 @@ temporary filesystems, others must survi @@ -2228,236 +2223,180 @@ temporary filesystems, others must survi
2228<li class="listitem"><p>10 GB for the log files (read-write, remote, permanent)</p></li> 2223<li class="listitem"><p>10 GB for the log files (read-write, remote, permanent)</p></li>
2229<li class="listitem"><p>5 GB for temporary files (read-write, local, temporary)</p></li> 2224<li class="listitem"><p>5 GB for temporary files (read-write, local, temporary)</p></li>
2230</ul></div> 2225</ul></div>
2231</div> 2226</div>
2232<div class="sect1"> 2227<div class="sect1">
2233<div class="titlepage"><div><div><h2 class="title" style="clear: both"> 2228<div class="titlepage"><div><div><h2 class="title" style="clear: both">
2234<a name="bulk.var"></a>8.4. Bulk build variants</h2></div></div></div> 2229<a name="bulk.var"></a>8.4. Bulk build variants</h2></div></div></div>
2235<p>To ensure that pkgsrc packages work in different configurations, it 2230<p>To ensure that pkgsrc packages work in different configurations, it
2236makes sense to run non-default bulk builds from time to time. This 2231makes sense to run non-default bulk builds from time to time. This
2237section lists some ideas for bulk builds that intentionally let packages 2232section lists some ideas for bulk builds that intentionally let packages
2238fail if they don't follow the pkgsrc style.</p> 2233fail if they don't follow the pkgsrc style.</p>
2239<div class="sect2"> 2234<div class="sect2">
2240<div class="titlepage"><div><div><h3 class="title"> 2235<div class="titlepage"><div><div><h3 class="title">
2241<a name="bulk.var.subst_noop"></a>8.4.1. Strict SUBST blocks</h3></div></div></div> 2236<a name="bulk.var.confopt"></a>8.4.1. Detect unknown configure options</h3></div></div></div>
2242<p>Up to May 2020, the SUBST blocks ignored files that didn't exist, 
2243as well as substitutions that didn't have any effect. There were quite a 
2244few SUBST blocks that were redundant, and these have been removed 
2245already.</p> 
2246<p>The next step would be to not only check that each filename pattern 
2247has an effect but also that each substitution in SUBST_SED or SUBST_VARS 
2248applies to at least one file.</p> 
2249<p>To do this, <code class="filename">mk/subst.mk</code> would have to be 
2250adjusted, in a similar way as the check for no-op SUBST_FILES. There are 
2251several regression tests in <code class="filename">regress/infra-unittests</code> 
2252that help to get all edge cases correct.</p> 
2253<p>When a package fails this additional check, there are various 
2254possible causes why the <code class="varname">SUBST_SED</code> became a 
2255no-op.</p> 
2256<div class="orderedlist"><ol class="orderedlist" type="1"> 
2257<li class="listitem"><p>The pattern used to be found in a former version of the 
2258package, but is not needed anymore. In that case, just remove 
2259it.</p></li> 
2260<li class="listitem"><p>The pattern contains a typo. In that case, fix the typo 
2261and bump <code class="varname">PKGREVISION</code>, since the fixed typo will 
2262probably modify the resulting binary package.</p></li> 
2263<li class="listitem"><p>There is a patch that is applied before the SUBST block, 
2264and the patch accidentally contains the change that was intended for the 
2265SUBST block. In that case, remove the respective hunk from the 
2266patch.</p></li> 
2267</ol></div> 
2268</div> 
2269<div class="sect2"> 
2270<div class="titlepage"><div><div><h3 class="title"> 
2271<a name="bulk.var.confopt"></a>8.4.2. Detect unknown configure options</h3></div></div></div> 
2272<p>Add the following line to <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p> 2237<p>Add the following line to <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
2273<pre class="programlisting"> 2238<pre class="programlisting">
2274GNU_CONFIGURE_STRICT= yes 2239GNU_CONFIGURE_STRICT= yes
2275</pre> 2240</pre>
2276<p>When a package fails this additional check, the most common cause 2241<p>When a package fails this additional check, the most common cause
2277is that the configure option was valid for an older version of the 2242is that the configure option was valid for an older version of the
2278package but does not apply anymore. In that case, just remove it.</p> 2243package but does not apply anymore. In that case, just remove it.</p>
2279</div> 2244</div>
2280<div class="sect2"> 2245<div class="sect2">
2281<div class="titlepage"><div><div><h3 class="title"> 2246<div class="titlepage"><div><div><h3 class="title">
2282<a name="bulk.var.comperr"></a>8.4.3. Detect classes of bugs by forcing compiler warnings</h3></div></div></div> 2247<a name="bulk.var.comperr"></a>8.4.2. Detect classes of bugs by forcing compiler warnings</h3></div></div></div>
2283<p>The job of a compiler is not restricted to producing executable 2248<p>The job of a compiler is not restricted to producing executable
2284code, most compilers also detects typical mistakes.</p> 2249code, most compilers also detect typical programming mistakes. The pkgsrc
2285<p>Add the following line to <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p> 2250compiler wrappers make it easy to force compiler options when the package
 2251is built. This can be used to find typical bugs across all packages that
 2252are in pkgsrc. By reporting these bugs upstream, the packages will be
 2253more reliable with the next updates.</p>
 2254<p>Add some of the following lines to <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>:</p>
2286<pre class="programlisting"> 2255<pre class="programlisting">
2287CFLAGS+= -Werror=char-subscripts 2256CFLAGS+= -Werror=char-subscripts
 2257CFLAGS+= -Werror=implicit-function-declaration
2288</pre> 2258</pre>
2289<p>When a package fails this additional check, first document the 2259<p>When a package fails to build using these stricter compiler
2290circumstances in which the compiler produced the error message. This 2260options, document the circumstances in which the compiler produced the
2291includes:</p> 2261error message. This includes:</p>
2292<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 2262<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
2293<li class="listitem"><p>The platform 2263<li class="listitem"><p>The platform
2294(<code class="varname">MACHINE_PLATFORM</code>)</p></li> 2264(<code class="varname">MACHINE_PLATFORM</code>)</p></li>
2295<li class="listitem"><p>The source file</p></li> 2265<li class="listitem"><p>The source file</p></li>
2296<li class="listitem"><p>An excerpt of the code. GCC and Clang already do this as 2266<li class="listitem"><p>An excerpt of the code. GCC and Clang already do this as
2297part of the diagnostic.</p></li> 2267part of the diagnostic.</p></li>
2298<li class="listitem"><p>The error message from the compiler.</p></li> 2268<li class="listitem"><p>The exact error message from the compiler.</p></li>
2299</ul></div> 2269</ul></div>
2300<p>If a package produces these error messages, but the package is 2270<p>If a package produces these error messages, but the package is
2301fine, document this in the package Makefile, like this:</p> 2271fine, record this in your local <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>, like this, to skip this check
 2272in the next builds:</p>
2302<pre class="programlisting"> 2273<pre class="programlisting">
 2274.if ${PKGPATH} == category/package
2303# Version ${VERSION} failed on ${MACHINE_PLATFORM}: 2275# Version ${VERSION} failed on ${MACHINE_PLATFORM}:
2304# error message 2276# error message
2305# code 2277# code
2306# reason why the code does not need to be fixed 2278# Reason why the code does not need to be fixed.
2307BUILDLINK_TRANSFORM+= rm:-Werror=char-subscripts 2279BUILDLINK_TRANSFORM+= rm:-Werror=char-subscripts
 2280.endif
2308</pre> 2281</pre>
2309<p>If the error messages from the compiler are valid and the code 2282<p>If the error messages from the compiler are valid and the code
2310needs to be fixed, prepare a patch for a single source file, or if it's a 2283needs to be fixed, prepare a local patch (see
2311one-liner fix, add a SUBST block to the package Makefile. In any case, 2284<code class="varname">LOCALPATCHES</code>) and report the bug to the upstream
2312report it to the upstream authors of the package.</p> 2285authors of the package, providing them with the information you collected
 2286above.</p>
 2287<p>Patches that are not essential for the package to work should only
 2288be reported upstream but not committed to pkgsrc, to make future updates
 2289easier.</p>
2313</div> 2290</div>
2314<div class="sect2"> 2291<div class="sect2">
2315<div class="titlepage"><div><div><h3 class="title"> 2292<div class="titlepage"><div><div><h3 class="title">
2316<a name="bulk.var.dirs"></a>8.4.4. Use custom directories</h3></div></div></div> 2293<a name="bulk.var.dirs"></a>8.4.3. Use custom directories</h3></div></div></div>
2317<p>Some directories like <code class="varname">PREFIX</code>, 2294<p>Some directories like <code class="varname">PREFIX</code>,
2318<code class="varname">VARBASE</code>, <code class="varname">PKG_SYSCONFDIR</code>, 2295<code class="varname">VARBASE</code>, <code class="varname">PKG_SYSCONFDIR</code>,
2319<code class="varname">PKGMANDIR</code>, <code class="varname">PKG_INFODIR</code> can be 2296<code class="varname">PKGMANDIR</code>, <code class="varname">PKG_INFODIR</code> can be
2320configured in pkgsrc. Set these to arbitrary paths during bootstrap or 2297configured in pkgsrc. Set these to arbitrary paths during bootstrap or
2321afterwards in <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p> 2298afterwards in <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
2322<pre class="programlisting"> 2299<pre class="programlisting">
2323PREFIX= /a-random-uuid 2300PREFIX= /a-random-uuid
2324PKG_SYSCONFDIR= /a-random-uuid 2301PKG_SYSCONFDIR= /a-random-uuid
2325VARBASE= /a-random-uuid 2302VARBASE= /a-random-uuid
2326PKGMANDIR= a-random-uuid 2303PKGMANDIR= a-random-uuid
2327PKG_INFODIR= a-random-uuid 2304PKG_INFODIR= a-random-uuid
2328</pre> 2305</pre>
2329</div> 2306</div>
2330<div class="sect2"> 2307<div class="sect2">
2331<div class="titlepage"><div><div><h3 class="title"> 2308<div class="titlepage"><div><div><h3 class="title">
2332<a name="bulk.var.warn"></a>8.4.5. Turn warnings into errors</h3></div></div></div> 2309<a name="bulk.var.warn"></a>8.4.4. Turn warnings into errors</h3></div></div></div>
2333<p>When building a package, warnings are typically ignored since they 2310<p>When building a package, warnings are typically ignored since they
2334just flow by and do not cause the build to fail immediately. To find 2311just flow by and do not cause the build to fail immediately. To find
2335these warnings, redefine them to errors in <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p> 2312these warnings, redefine them to errors in <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
2336<pre class="programlisting"> 2313<pre class="programlisting">
2337DELAYED_WARNING_MSG= ${DELAYED_ERROR_MSG} "(was warning)" 2314DELAYED_WARNING_MSG= ${DELAYED_ERROR_MSG} "(was warning)"
2338WARNING_MSG= ${FAIL_MSG} "(was warning)" 2315WARNING_MSG= ${FAIL_MSG} "(was warning)"
2339</pre> 2316</pre>
2340<p>(There are many more classes of warnings in pkgsrc, and most of 2317<p>(There are many more classes of warnings in pkgsrc, and most of
2341them can be redefined with a simple definition like above.</p> 2318them can be redefined with a simple definition like above.</p>
2342<p>If a package suggest to add <code class="varname">USE_TOOLS+=perl</code> to 2319<p>If a package suggests to add <code class="varname">USE_TOOLS+=perl</code> to
2343the package Makefile, research whether the package actually needs Perl. 2320the package Makefile, research whether the package actually needs Perl.
2344If it does, add <code class="varname">USE_TOOLS+=perl</code> to the package 2321If it does, add <code class="varname">USE_TOOLS+=perl</code> to the package
2345Makefile, and if it doesn't, add 2322Makefile, and if it doesn't, add
2346<code class="varname">TOOLS_BROKEN+=perl</code>.</p> 2323<code class="varname">TOOLS_BROKEN+=perl</code>.</p>
2347</div> 2324</div>
2348<div class="sect2"> 2325<div class="sect2">
2349<div class="titlepage"><div><div><h3 class="title"> 2326<div class="titlepage"><div><div><h3 class="title">
2350<a name="bulk.var.pkglint"></a>8.4.6. Reject packages for which pkglint reports errors</h3></div></div></div> 2327<a name="bulk.var.pkglint"></a>8.4.5. Reject packages for which pkglint reports errors</h3></div></div></div>
2351<p>Using pkglint as part of the regular build process is mostly a 2328<p>Using pkglint as part of the regular build process is mostly a
2352waste of time. If you want to fix some of the warnings, just run pkglint 2329waste of time. If you want to fix some of the warnings, just run pkglint
2353recursively on the whole pkgsrc tree. This will take a few minutes (up to 2330recursively on the whole pkgsrc tree. This will take a few minutes (up to
235410), which is much faster than a complete bulk build.</p> 233110), which is much faster than a complete bulk build.</p>
2355</div> 2332</div>
2356<div class="sect2"> 2333<div class="sect2">
2357<div class="titlepage"><div><div><h3 class="title"> 2334<div class="titlepage"><div><div><h3 class="title">
2358<a name="bulk.var.strings"></a>8.4.7. Reject packages that contain forbidden strings</h3></div></div></div> 2335<a name="bulk.var.strings"></a>8.4.6. Reject packages that contain forbidden strings</h3></div></div></div>
2359<p>To ensure that the binary packages don't contain references to the 2336<p>To ensure that the binary packages don't contain references to the
2360build directory, there is already <code class="varname">CHECK_WRKREF</code>. If 2337build directory, there is already <code class="varname">CHECK_WRKREF</code>. If
2361that variable includes the item <code class="literal">extra</code>, it is 2338that variable includes the item <code class="literal">extra</code>, it is
2362possible to define additional patterns that must not appear in any 2339possible to define additional patterns that must not appear in any
2363installed file. This is specified in <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p> 2340installed file. This is specified in <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
2364<pre class="programlisting"> 2341<pre class="programlisting">
2365CHECK_WRKREF= extra 2342CHECK_WRKREF= extra
2366CHECK_WRKREF_EXTRA_DIRS+= /usr/local 2343CHECK_WRKREF_EXTRA_DIRS+= /usr/local
2367CHECK_WRKREF_EXTRA_DIRS+= /usr/pkg 2344CHECK_WRKREF_EXTRA_DIRS+= /usr/pkg
2368CHECK_WRKREF_EXTRA_DIRS+= @[A-Z][A-Z]*@ 2345CHECK_WRKREF_EXTRA_DIRS+= @[A-Z][A-Z]*@
2369</pre> 2346</pre>
2370<p>The above patterns will probably generate many false positives, 2347<p>The above patterns will probably generate many false positives,
2371therefore the results need to be taken with a grain of salt.</p> 2348therefore the results need to be taken with a grain of salt.</p>
2372</div> 2349</div>
2373<div class="sect2"> 2350<div class="sect2">
2374<div class="titlepage"><div><div><h3 class="title"> 2351<div class="titlepage"><div><div><h3 class="title">
2375<a name="bulk.var.test"></a>8.4.8. Reject packages whose self-test fails</h3></div></div></div> 2352<a name="bulk.var.test"></a>8.4.7. Reject packages whose self-test fails</h3></div></div></div>
2376<p>To run the test suites that come with each package, add this line 2353<p>To run the test suites that come with each package, add this line
2377to <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p> 2354to <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
2378<pre class="programlisting"> 2355<pre class="programlisting">
2379PKGSRC_RUN_TEST= yes 2356PKGSRC_RUN_TEST= yes
2380</pre> 2357</pre>
2381<p>Be prepared that even the most basic packages fail this test. When 2358<p>Be prepared that even the most basic packages fail this test. When
2382doing a bulk build with this, it will often abort in the early phase 2359doing a bulk build with this, it will often abort in the early phase
2383where the packages are scanned for their dependencies since there are 2360where the packages are scanned for their dependencies since there are
2384cyclic dependencies. There is still a lot to do in this area.</p> 2361cyclic dependencies. There is still a lot to do in this area.</p>
2385</div> 2362</div>
2386<div class="sect2"> 2363<div class="sect2">
2387<div class="titlepage"><div><div><h3 class="title"> 2364<div class="titlepage"><div><div><h3 class="title">
2388<a name="bulk.var.shvar"></a>8.4.9. Reject packages that use undefined shell variables</h3></div></div></div> 2365<a name="bulk.var.shvar"></a>8.4.8. Reject packages that use undefined shell variables</h3></div></div></div>
2389<p>To catch typos in the shell snippets from the Makefile fragments, 2366<p>To catch typos in the shell snippets from the Makefile fragments,
2390add the <code class="literal">-u</code> flag to most of the commands by adding this 2367add the <code class="literal">-u</code> flag to most of the commands by adding this
2391line to <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p> 2368line to <a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
2392<pre class="programlisting"> 2369<pre class="programlisting">
2393RUN= @set -eu; 2370RUN= @set -eu;
2394</pre> 2371</pre>
2395<p>See <code class="filename">mk/misc/common.mk</code> for the existing 2372<p>See <code class="filename">mk/misc/common.mk</code> for the existing
2396definition.</p> 2373definition.</p>
2397</div> 2374</div>
2398<div class="sect2"> 2375<div class="sect2">
2399<div class="titlepage"><div><div><h3 class="title"> 2376<div class="titlepage"><div><div><h3 class="title">
2400<a name="bulk.var.quiet"></a>8.4.10. Turn off verbose logging</h3></div></div></div> 2377<a name="bulk.var.quiet"></a>8.4.9. Turn off verbose logging</h3></div></div></div>
2401<p>The build logs of a package are often quite long. This allows error 2378<p>The build logs of a package are often quite long. This allows error
2402messages or other interesting details to hide between the noise. To make 2379messages or other interesting details to hide between the noise. To make
2403the actual error message stand out more, add these lines to 2380the actual error message stand out more, add these lines to
2404<a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p> 2381<a class="link" href="#mk.conf"><code class="filename">mk.conf</code></a>.</p>
2405<pre class="programlisting"> 2382<pre class="programlisting">
2406GNU_CONFIGURE_QUIET= yes 2383GNU_CONFIGURE_QUIET= yes
2407MAKE_FLAGS+= -s 2384MAKE_FLAGS+= -s
2408</pre> 2385</pre>
2409<p>The <code class="literal">-s</code> option works for both GNU Make and BSD 2386<p>The <code class="literal">-s</code> option works for both GNU Make and BSD
2410Make. On exotic platforms with their own make, it may be a little 2387Make. On exotic platforms with their own make, it may be a little
2411different.</p> 2388different.</p>
2412</div> 2389</div>
2413<div class="sect2"> 
2414<div class="titlepage"><div><div><h3 class="title"> 
2415<a name="bulk.var.options"></a>8.4.11. Select random sets of options</h3></div></div></div> 
2416<p>Most bulk builds run with the default package options. This means 
2417that other combinations of options are not regularly tested. To do this, 
2418run a bulk build with these configurations.</p> 
2419<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 
2420<li class="listitem"><p>no options enabled</p></li> 
2421<li class="listitem"><p>all options enabled</p></li> 
2422<li class="listitem"><p>2n + 0</p></li> 
2423<li class="listitem"><p>2n + 1</p></li> 
2424<li class="listitem"><p>4n + 0..1</p></li> 
2425<li class="listitem"><p>4n + 2..3</p></li> 
2426<li class="listitem"><p>8n + 0..3</p></li> 
2427<li class="listitem"><p>8n + 4..7</p></li> 
2428<li class="listitem"><p>2048n + 0..1023</p></li> 
2429<li class="listitem"><p>2048n + 1024..2047</p></li> 
2430</ul></div> 
2431<p>Open questions are:</p> 
2432<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "> 
2433<li class="listitem"><p>how to collect all options from the entire 
2434pkgsrc</p></li> 
2435<li class="listitem"><p>how to handle mutually exclusive 
2436options</p></li> 
2437<li class="listitem"><p>the sets of mutually exclusive options are defined 
2438per-package</p></li> 
2439<li class="listitem"><p>the sets of nonempty sets are defined 
2440per-package</p></li> 
2441</ul></div> 
2442</div> 
2443<div class="sect2"> 
2444<div class="titlepage"><div><div><h3 class="title"> 
2445<a name="bulk.var.build_defs"></a>8.4.12. Select random configurations of BUILD_DEFS</h3></div></div></div> 
2446<p>Just like the <code class="varname">PKG_OPTIONS</code>, the 
2447<code class="varname">BUILD_DEFS</code> also allow different variants of pkgsrc to 
2448be created. The same ideas as in <a class="xref" href="#bulk.var.options" title="8.4.11. Select random sets of options">Section 8.4.11, &#8220;Select random sets of options&#8221;</a> 
2449apply.</p> 
2450</div> 
2451</div> 2390</div>
2452<div class="sect1"> 2391<div class="sect1">
2453<div class="titlepage"><div><div><h2 class="title" style="clear: both"> 2392<div class="titlepage"><div><div><h2 class="title" style="clear: both">
2454<a name="creating-cdroms"></a>8.5. Creating a multiple CD-ROM packages collection</h2></div></div></div> 2393<a name="creating-cdroms"></a>8.5. Creating a multiple CD-ROM packages collection</h2></div></div></div>
2455<p>After your pkgsrc bulk-build has completed, you may wish to 2394<p>After your pkgsrc bulk-build has completed, you may wish to
2456 create a CD-ROM set of the resulting binary packages to assist 2395 create a CD-ROM set of the resulting binary packages to assist
2457 in installing packages on other machines. The 2396 in installing packages on other machines. The
2458 <a href="https://cdn.NetBSD.org/pub/pkgsrc/current/pkgsrc/pkgtools/cdpack/README.html" target="_top"><code class="filename">pkgtools/cdpack</code></a> package provides 2397 <a href="https://cdn.NetBSD.org/pub/pkgsrc/current/pkgsrc/pkgtools/cdpack/README.html" target="_top"><code class="filename">pkgtools/cdpack</code></a> package provides
2459 a simple tool for creating the ISO 9660 images. 2398 a simple tool for creating the ISO 9660 images.
2460 <span class="command"><strong>cdpack</strong></span> arranges the packages on the CD-ROMs in a 2399 <span class="command"><strong>cdpack</strong></span> arranges the packages on the CD-ROMs in a
2461 way that keeps all the dependencies for a given package on the same 2400 way that keeps all the dependencies for a given package on the same
2462 CD as that package.</p> 2401 CD as that package.</p>
2463<div class="sect2"> 2402<div class="sect2">
cvs diff -r1.291 -r1.292 pkgsrc/doc/pkgsrc.txt (expand / switch to unified diff)

--- pkgsrc/doc/pkgsrc.txt 2020/05/22 18:51:44 1.291
+++ pkgsrc/doc/pkgsrc.txt 2020/05/30 19:25:38 1.292
@@ -99,38 +99,35 @@ I. The pkgsrc user's guide @@ -99,38 +99,35 @@ I. The pkgsrc user's guide
99 7.1. Building a single binary package 99 7.1. Building a single binary package
100 7.2. Settings for creation of binary packages 100 7.2. Settings for creation of binary packages
101 101
102 8. Creating binary packages for everything in pkgsrc (bulk builds) 102 8. Creating binary packages for everything in pkgsrc (bulk builds)
103 103
104 8.1. Preparations 104 8.1. Preparations
105 8.2. Running a pbulk-style bulk build 105 8.2. Running a pbulk-style bulk build
106 106
107 8.2.1. Configuration 107 8.2.1. Configuration
108 108
109 8.3. Requirements of a full bulk build 109 8.3. Requirements of a full bulk build
110 8.4. Bulk build variants 110 8.4. Bulk build variants
111 111
112 8.4.1. Strict SUBST blocks 112 8.4.1. Detect unknown configure options
113 8.4.2. Detect unknown configure options 113 8.4.2. Detect classes of bugs by forcing compiler warnings
114 8.4.3. Detect classes of bugs by forcing compiler warnings 114 8.4.3. Use custom directories
115 8.4.4. Use custom directories 115 8.4.4. Turn warnings into errors
116 8.4.5. Turn warnings into errors 116 8.4.5. Reject packages for which pkglint reports errors
117 8.4.6. Reject packages for which pkglint reports errors 117 8.4.6. Reject packages that contain forbidden strings
118 8.4.7. Reject packages that contain forbidden strings 118 8.4.7. Reject packages whose self-test fails
119 8.4.8. Reject packages whose self-test fails 119 8.4.8. Reject packages that use undefined shell variables
120 8.4.9. Reject packages that use undefined shell variables 120 8.4.9. Turn off verbose logging
121 8.4.10. Turn off verbose logging 
122 8.4.11. Select random sets of options 
123 8.4.12. Select random configurations of BUILD_DEFS 
124 121
125 8.5. Creating a multiple CD-ROM packages collection 122 8.5. Creating a multiple CD-ROM packages collection
126 123
127 8.5.1. Example of cdpack 124 8.5.1. Example of cdpack
128 125
129 9. Directory layout of the installed files 126 9. Directory layout of the installed files
130 127
131 9.1. File system layout in ${LOCALBASE} 128 9.1. File system layout in ${LOCALBASE}
132 9.2. File system layout in ${VARBASE} 129 9.2. File system layout in ${VARBASE}
133 130
134 10. Frequently Asked Questions 131 10. Frequently Asked Questions
135 132
136 10.1. Are there any mailing lists for pkg-related discussion? 133 10.1. Are there any mailing lists for pkg-related discussion?
@@ -755,38 +752,35 @@ Table of Contents @@ -755,38 +752,35 @@ Table of Contents
755 7.1. Building a single binary package 752 7.1. Building a single binary package
756 7.2. Settings for creation of binary packages 753 7.2. Settings for creation of binary packages
757 754
7588. Creating binary packages for everything in pkgsrc (bulk builds) 7558. Creating binary packages for everything in pkgsrc (bulk builds)
759 756
760 8.1. Preparations 757 8.1. Preparations
761 8.2. Running a pbulk-style bulk build 758 8.2. Running a pbulk-style bulk build
762 759
763 8.2.1. Configuration 760 8.2.1. Configuration
764 761
765 8.3. Requirements of a full bulk build 762 8.3. Requirements of a full bulk build
766 8.4. Bulk build variants 763 8.4. Bulk build variants
767 764
768 8.4.1. Strict SUBST blocks 765 8.4.1. Detect unknown configure options
769 8.4.2. Detect unknown configure options 766 8.4.2. Detect classes of bugs by forcing compiler warnings
770 8.4.3. Detect classes of bugs by forcing compiler warnings 767 8.4.3. Use custom directories
771 8.4.4. Use custom directories 768 8.4.4. Turn warnings into errors
772 8.4.5. Turn warnings into errors 769 8.4.5. Reject packages for which pkglint reports errors
773 8.4.6. Reject packages for which pkglint reports errors 770 8.4.6. Reject packages that contain forbidden strings
774 8.4.7. Reject packages that contain forbidden strings 771 8.4.7. Reject packages whose self-test fails
775 8.4.8. Reject packages whose self-test fails 772 8.4.8. Reject packages that use undefined shell variables
776 8.4.9. Reject packages that use undefined shell variables 773 8.4.9. Turn off verbose logging
777 8.4.10. Turn off verbose logging 
778 8.4.11. Select random sets of options 
779 8.4.12. Select random configurations of BUILD_DEFS 
780 774
781 8.5. Creating a multiple CD-ROM packages collection 775 8.5. Creating a multiple CD-ROM packages collection
782 776
783 8.5.1. Example of cdpack 777 8.5.1. Example of cdpack
784 778
7859. Directory layout of the installed files 7799. Directory layout of the installed files
786 780
787 9.1. File system layout in ${LOCALBASE} 781 9.1. File system layout in ${LOCALBASE}
788 9.2. File system layout in ${VARBASE} 782 9.2. File system layout in ${VARBASE}
789 783
79010. Frequently Asked Questions 78410. Frequently Asked Questions
791 785
792 10.1. Are there any mailing lists for pkg-related discussion? 786 10.1. Are there any mailing lists for pkg-related discussion?
@@ -1111,27 +1105,28 @@ in the / directory. It will create the d @@ -1111,27 +1105,28 @@ in the / directory. It will create the d
1111tools for managing binary packages and the database of installed packages). 1105tools for managing binary packages and the database of installed packages).
1112 1106
11135.1.2. Installing binary packages 11075.1.2. Installing binary packages
1114 1108
1115In the directory from the last section, there is a subdirectory called All/, 1109In the directory from the last section, there is a subdirectory called All/,
1116which contains all the binary packages that are available for the platform, 1110which contains all the binary packages that are available for the platform,
1117excluding those that may not be distributed via FTP or CDROM (depending on 1111excluding those that may not be distributed via FTP or CDROM (depending on
1118which medium you are using). 1112which medium you are using).
1119 1113
1120To install packages directly from an FTP or HTTP server, run the following 1114To install packages directly from an FTP or HTTP server, run the following
1121commands in a Bourne-compatible shell (be sure to su to root first): 1115commands in a Bourne-compatible shell (be sure to su to root first):
1122 1116
1123# PATH="/usr/pkg/sbin:$PATH" 1117# PATH="/usr/pkg/sbin:$PATH"
1124# PKG_PATH="https://cdn.NetBSD.org/pub/pkgsrc/packages" 1118
 1119# PKG_PATH="http://cdn.NetBSD.org/pub/pkgsrc/packages"
1125# PKG_PATH="$PKG_PATH/OPSYS/ARCH/VERSIONS/All/" 1120# PKG_PATH="$PKG_PATH/OPSYS/ARCH/VERSIONS/All/"
1126# export PATH PKG_PATH 1121# export PATH PKG_PATH
1127 1122
1128Instead of URLs, you can also use local paths, for example if you are 1123Instead of URLs, you can also use local paths, for example if you are
1129installing from a set of CDROMs, DVDs or an NFS-mounted repository. If you want 1124installing from a set of CDROMs, DVDs or an NFS-mounted repository. If you want
1130to install packages from multiple sources, you can separate them by a semicolon 1125to install packages from multiple sources, you can separate them by a semicolon
1131in PKG_PATH. 1126in PKG_PATH.
1132 1127
1133After these preparations, installing a package is very easy: 1128After these preparations, installing a package is very easy:
1134 1129
1135# pkg_add libreoffice 1130# pkg_add libreoffice
1136# pkg_add ap24-php71-* 1131# pkg_add ap24-php71-*
1137 1132
@@ -1162,28 +1157,28 @@ package in question and then removes the @@ -1162,28 +1157,28 @@ package in question and then removes the
1162 1157
1163will remove jpeg and all the packages that used it; this allows upgrading the 1158will remove jpeg and all the packages that used it; this allows upgrading the
1164jpeg package. 1159jpeg package.
1165 1160
11665.1.4. Getting information about installed packages 11615.1.4. Getting information about installed packages
1167 1162
1168The pkg_info shows information about installed packages or binary package 1163The pkg_info shows information about installed packages or binary package
1169files. 1164files.
1170 1165
11715.1.5. Checking for security vulnerabilities in installed packages 11665.1.5. Checking for security vulnerabilities in installed packages
1172 1167
1173The pkgsrc Security Team and Packages Groups maintain a list of known security 1168The pkgsrc Security Team and Packages Groups maintain a list of known security
1174vulnerabilities to packages which are (or have been) included in pkgsrc. The 1169vulnerabilities to packages which are (or have been) included in pkgsrc. The
1175list is available from the NetBSD FTP site at https://ftp.NetBSD.org/pub/NetBSD 1170list is available from the NetBSD FTP site at http://ftp.NetBSD.org/pub/NetBSD/
1176/packages/vulns/pkg-vulnerabilities. 1171packages/vulns/pkg-vulnerabilities.
1177 1172
1178Through pkg_admin fetch-pkg-vulnerabilities, this list can be downloaded 1173Through pkg_admin fetch-pkg-vulnerabilities, this list can be downloaded
1179automatically, and a security audit of all packages installed on a system can 1174automatically, and a security audit of all packages installed on a system can
1180take place. 1175take place.
1181 1176
1182There are two components to auditing. The first step, pkg_admin 1177There are two components to auditing. The first step, pkg_admin
1183fetch-pkg-vulnerabilities, is for downloading the list of vulnerabilities from 1178fetch-pkg-vulnerabilities, is for downloading the list of vulnerabilities from
1184the NetBSD FTP site. The second step, pkg_admin audit, checks to see if any of 1179the NetBSD FTP site. The second step, pkg_admin audit, checks to see if any of
1185your installed packages are vulnerable. If a package is vulnerable, you will 1180your installed packages are vulnerable. If a package is vulnerable, you will
1186see output similar to the following: 1181see output similar to the following:
1187 1182
1188Package samba-2.0.9 has a local-root-shell vulnerability, see 1183Package samba-2.0.9 has a local-root-shell vulnerability, see
1189 https://www.samba.org/samba/whatsnew/macroexploit.html 1184 https://www.samba.org/samba/whatsnew/macroexploit.html
@@ -1727,38 +1722,35 @@ See Section 19.17, "Other helpful target @@ -1727,38 +1722,35 @@ See Section 19.17, "Other helpful target
1727 1722
1728Chapter 8. Creating binary packages for everything in pkgsrc (bulk builds) 1723Chapter 8. Creating binary packages for everything in pkgsrc (bulk builds)
1729 1724
1730Table of Contents 1725Table of Contents
1731 1726
17328.1. Preparations 17278.1. Preparations
17338.2. Running a pbulk-style bulk build 17288.2. Running a pbulk-style bulk build
1734 1729
1735 8.2.1. Configuration 1730 8.2.1. Configuration
1736 1731
17378.3. Requirements of a full bulk build 17328.3. Requirements of a full bulk build
17388.4. Bulk build variants 17338.4. Bulk build variants
1739 1734
1740 8.4.1. Strict SUBST blocks 1735 8.4.1. Detect unknown configure options
1741 8.4.2. Detect unknown configure options 1736 8.4.2. Detect classes of bugs by forcing compiler warnings
1742 8.4.3. Detect classes of bugs by forcing compiler warnings 1737 8.4.3. Use custom directories
1743 8.4.4. Use custom directories 1738 8.4.4. Turn warnings into errors
1744 8.4.5. Turn warnings into errors 1739 8.4.5. Reject packages for which pkglint reports errors
1745 8.4.6. Reject packages for which pkglint reports errors 1740 8.4.6. Reject packages that contain forbidden strings
1746 8.4.7. Reject packages that contain forbidden strings 1741 8.4.7. Reject packages whose self-test fails
1747 8.4.8. Reject packages whose self-test fails 1742 8.4.8. Reject packages that use undefined shell variables
1748 8.4.9. Reject packages that use undefined shell variables 1743 8.4.9. Turn off verbose logging
1749 8.4.10. Turn off verbose logging 
1750 8.4.11. Select random sets of options 
1751 8.4.12. Select random configurations of BUILD_DEFS 
1752 1744
17538.5. Creating a multiple CD-ROM packages collection 17458.5. Creating a multiple CD-ROM packages collection
1754 1746
1755 8.5.1. Example of cdpack 1747 8.5.1. Example of cdpack
1756 1748
1757For a number of reasons, you may want to build binary packages for a large 1749For a number of reasons, you may want to build binary packages for a large
1758selected set of packages in pkgsrc, or even for all pkgsrc packages. For 1750selected set of packages in pkgsrc, or even for all pkgsrc packages. For
1759instance, when you have multiple machines that should run the same software, it 1751instance, when you have multiple machines that should run the same software, it
1760is wasted time if they all build their packages themselves from source. Or you 1752is wasted time if they all build their packages themselves from source. Or you
1761may want to build a list of packages you want and check them before deploying 1753may want to build a list of packages you want and check them before deploying
1762onto production systems. There is a way of getting a set of binary packages: 1754onto production systems. There is a way of getting a set of binary packages:
1763the bulk build system, or pbulk ("p" stands for "parallel"). This chapter 1755the bulk build system, or pbulk ("p" stands for "parallel"). This chapter
1764describes how to set it up. 1756describes how to set it up.
@@ -1868,223 +1860,164 @@ others must survive a sudden reboot. @@ -1868,223 +1860,164 @@ others must survive a sudden reboot.
1868 * 5 GB for LOCALBASE (read-write, local, temporary) 1860 * 5 GB for LOCALBASE (read-write, local, temporary)
1869 1861
1870 * 10 GB for the log files (read-write, remote, permanent) 1862 * 10 GB for the log files (read-write, remote, permanent)
1871 1863
1872 * 5 GB for temporary files (read-write, local, temporary) 1864 * 5 GB for temporary files (read-write, local, temporary)
1873 1865
18748.4. Bulk build variants 18668.4. Bulk build variants
1875 1867
1876To ensure that pkgsrc packages work in different configurations, it makes sense 1868To ensure that pkgsrc packages work in different configurations, it makes sense
1877to run non-default bulk builds from time to time. This section lists some ideas 1869to run non-default bulk builds from time to time. This section lists some ideas
1878for bulk builds that intentionally let packages fail if they don't follow the 1870for bulk builds that intentionally let packages fail if they don't follow the
1879pkgsrc style. 1871pkgsrc style.
1880 1872
18818.4.1. Strict SUBST blocks 18738.4.1. Detect unknown configure options
1882 
1883Up to May 2020, the SUBST blocks ignored files that didn't exist, as well as 
1884substitutions that didn't have any effect. There were quite a few SUBST blocks 
1885that were redundant, and these have been removed already. 
1886 
1887The next step would be to not only check that each filename pattern has an 
1888effect but also that each substitution in SUBST_SED or SUBST_VARS applies to at 
1889least one file. 
1890 
1891To do this, mk/subst.mk would have to be adjusted, in a similar way as the 
1892check for no-op SUBST_FILES. There are several regression tests in regress/ 
1893infra-unittests that help to get all edge cases correct. 
1894 
1895When a package fails this additional check, there are various possible causes 
1896why the SUBST_SED became a no-op. 
1897 
1898 1. The pattern used to be found in a former version of the package, but is not 
1899 needed anymore. In that case, just remove it. 
1900 
1901 2. The pattern contains a typo. In that case, fix the typo and bump 
1902 PKGREVISION, since the fixed typo will probably modify the resulting binary 
1903 package. 
1904 
1905 3. There is a patch that is applied before the SUBST block, and the patch 
1906 accidentally contains the change that was intended for the SUBST block. In 
1907 that case, remove the respective hunk from the patch. 
1908 
19098.4.2. Detect unknown configure options 
1910 1874
1911Add the following line to mk.conf. 1875Add the following line to mk.conf.
1912 1876
1913GNU_CONFIGURE_STRICT= yes 1877GNU_CONFIGURE_STRICT= yes
1914 1878
1915When a package fails this additional check, the most common cause is that the 1879When a package fails this additional check, the most common cause is that the
1916configure option was valid for an older version of the package but does not 1880configure option was valid for an older version of the package but does not
1917apply anymore. In that case, just remove it. 1881apply anymore. In that case, just remove it.
1918 1882
19198.4.3. Detect classes of bugs by forcing compiler warnings 18838.4.2. Detect classes of bugs by forcing compiler warnings
1920 1884
1921The job of a compiler is not restricted to producing executable code, most 1885The job of a compiler is not restricted to producing executable code, most
1922compilers also detects typical mistakes. 1886compilers also detect typical programming mistakes. The pkgsrc compiler
 1887wrappers make it easy to force compiler options when the package is built. This
 1888can be used to find typical bugs across all packages that are in pkgsrc. By
 1889reporting these bugs upstream, the packages will be more reliable with the next
 1890updates.
1923 1891
1924Add the following line to mk.conf. 1892Add some of the following lines to mk.conf:
1925 1893
1926CFLAGS+= -Werror=char-subscripts 1894CFLAGS+= -Werror=char-subscripts
 1895CFLAGS+= -Werror=implicit-function-declaration
1927 1896
1928When a package fails this additional check, first document the circumstances in 1897When a package fails to build using these stricter compiler options, document
1929which the compiler produced the error message. This includes: 1898the circumstances in which the compiler produced the error message. This
 1899includes:
1930 1900
1931 * The platform (MACHINE_PLATFORM) 1901 * The platform (MACHINE_PLATFORM)
1932 1902
1933 * The source file 1903 * The source file
1934 1904
1935 * An excerpt of the code. GCC and Clang already do this as part of the 1905 * An excerpt of the code. GCC and Clang already do this as part of the
1936 diagnostic. 1906 diagnostic.
1937 1907
1938 * The error message from the compiler. 1908 * The exact error message from the compiler.
1939 1909
1940If a package produces these error messages, but the package is fine, document 1910If a package produces these error messages, but the package is fine, record
1941this in the package Makefile, like this: 1911this in your local mk.conf, like this, to skip this check in the next builds:
1942 1912
 1913.if ${PKGPATH} == category/package
1943# Version ${VERSION} failed on ${MACHINE_PLATFORM}: 1914# Version ${VERSION} failed on ${MACHINE_PLATFORM}:
1944# error message 1915# error message
1945# code 1916# code
1946# reason why the code does not need to be fixed 1917# Reason why the code does not need to be fixed.
1947BUILDLINK_TRANSFORM+= rm:-Werror=char-subscripts 1918BUILDLINK_TRANSFORM+= rm:-Werror=char-subscripts
 1919.endif
1948 1920
1949If the error messages from the compiler are valid and the code needs to be 1921If the error messages from the compiler are valid and the code needs to be
1950fixed, prepare a patch for a single source file, or if it's a one-liner fix, 1922fixed, prepare a local patch (see LOCALPATCHES) and report the bug to the
1951add a SUBST block to the package Makefile. In any case, report it to the 1923upstream authors of the package, providing them with the information you
1952upstream authors of the package. 1924collected above.
1953 1925
19548.4.4. Use custom directories 1926Patches that are not essential for the package to work should only be reported
 1927upstream but not committed to pkgsrc, to make future updates easier.
 1928
 19298.4.3. Use custom directories
1955 1930
1956Some directories like PREFIX, VARBASE, PKG_SYSCONFDIR, PKGMANDIR, PKG_INFODIR 1931Some directories like PREFIX, VARBASE, PKG_SYSCONFDIR, PKGMANDIR, PKG_INFODIR
1957can be configured in pkgsrc. Set these to arbitrary paths during bootstrap or 1932can be configured in pkgsrc. Set these to arbitrary paths during bootstrap or
1958afterwards in mk.conf. 1933afterwards in mk.conf.
1959 1934
1960PREFIX= /a-random-uuid 1935PREFIX= /a-random-uuid
1961PKG_SYSCONFDIR= /a-random-uuid 1936PKG_SYSCONFDIR= /a-random-uuid
1962VARBASE= /a-random-uuid 1937VARBASE= /a-random-uuid
1963PKGMANDIR= a-random-uuid 1938PKGMANDIR= a-random-uuid
1964PKG_INFODIR= a-random-uuid 1939PKG_INFODIR= a-random-uuid
1965 1940
19668.4.5. Turn warnings into errors 19418.4.4. Turn warnings into errors
1967 1942
1968When building a package, warnings are typically ignored since they just flow by 1943When building a package, warnings are typically ignored since they just flow by
1969and do not cause the build to fail immediately. To find these warnings, 1944and do not cause the build to fail immediately. To find these warnings,
1970redefine them to errors in mk.conf. 1945redefine them to errors in mk.conf.
1971 1946
1972DELAYED_WARNING_MSG= ${DELAYED_ERROR_MSG} "(was warning)" 1947DELAYED_WARNING_MSG= ${DELAYED_ERROR_MSG} "(was warning)"
1973WARNING_MSG= ${FAIL_MSG} "(was warning)" 1948WARNING_MSG= ${FAIL_MSG} "(was warning)"
1974 1949
1975(There are many more classes of warnings in pkgsrc, and most of them can be 1950(There are many more classes of warnings in pkgsrc, and most of them can be
1976redefined with a simple definition like above. 1951redefined with a simple definition like above.
1977 1952
1978If a package suggest to add USE_TOOLS+=perl to the package Makefile, research 1953If a package suggests to add USE_TOOLS+=perl to the package Makefile, research
1979whether the package actually needs Perl. If it does, add USE_TOOLS+=perl to the 1954whether the package actually needs Perl. If it does, add USE_TOOLS+=perl to the
1980package Makefile, and if it doesn't, add TOOLS_BROKEN+=perl. 1955package Makefile, and if it doesn't, add TOOLS_BROKEN+=perl.
1981 1956
19828.4.6. Reject packages for which pkglint reports errors 19578.4.5. Reject packages for which pkglint reports errors
1983 1958
1984Using pkglint as part of the regular build process is mostly a waste of time. 1959Using pkglint as part of the regular build process is mostly a waste of time.
1985If you want to fix some of the warnings, just run pkglint recursively on the 1960If you want to fix some of the warnings, just run pkglint recursively on the
1986whole pkgsrc tree. This will take a few minutes (up to 10), which is much 1961whole pkgsrc tree. This will take a few minutes (up to 10), which is much
1987faster than a complete bulk build. 1962faster than a complete bulk build.
1988 1963
19898.4.7. Reject packages that contain forbidden strings 19648.4.6. Reject packages that contain forbidden strings
1990 1965
1991To ensure that the binary packages don't contain references to the build 1966To ensure that the binary packages don't contain references to the build
1992directory, there is already CHECK_WRKREF. If that variable includes the item 1967directory, there is already CHECK_WRKREF. If that variable includes the item
1993extra, it is possible to define additional patterns that must not appear in any 1968extra, it is possible to define additional patterns that must not appear in any
1994installed file. This is specified in mk.conf. 1969installed file. This is specified in mk.conf.
1995 1970
1996CHECK_WRKREF= extra 1971CHECK_WRKREF= extra
1997CHECK_WRKREF_EXTRA_DIRS+= /usr/local 1972CHECK_WRKREF_EXTRA_DIRS+= /usr/local
1998CHECK_WRKREF_EXTRA_DIRS+= /usr/pkg 1973CHECK_WRKREF_EXTRA_DIRS+= /usr/pkg
1999CHECK_WRKREF_EXTRA_DIRS+= @[A-Z][A-Z]*@ 1974CHECK_WRKREF_EXTRA_DIRS+= @[A-Z][A-Z]*@
2000 1975
2001The above patterns will probably generate many false positives, therefore the 1976The above patterns will probably generate many false positives, therefore the
2002results need to be taken with a grain of salt. 1977results need to be taken with a grain of salt.
2003 1978
20048.4.8. Reject packages whose self-test fails 19798.4.7. Reject packages whose self-test fails
2005 1980
2006To run the test suites that come with each package, add this line to mk.conf. 1981To run the test suites that come with each package, add this line to mk.conf.
2007 1982
2008PKGSRC_RUN_TEST= yes 1983PKGSRC_RUN_TEST= yes
2009 1984
2010Be prepared that even the most basic packages fail this test. When doing a bulk 1985Be prepared that even the most basic packages fail this test. When doing a bulk
2011build with this, it will often abort in the early phase where the packages are 1986build with this, it will often abort in the early phase where the packages are
2012scanned for their dependencies since there are cyclic dependencies. There is 1987scanned for their dependencies since there are cyclic dependencies. There is
2013still a lot to do in this area. 1988still a lot to do in this area.
2014 1989
20158.4.9. Reject packages that use undefined shell variables 19908.4.8. Reject packages that use undefined shell variables
2016 1991
2017To catch typos in the shell snippets from the Makefile fragments, add the -u 1992To catch typos in the shell snippets from the Makefile fragments, add the -u
2018flag to most of the commands by adding this line to mk.conf. 1993flag to most of the commands by adding this line to mk.conf.
2019 1994
2020RUN= @set -eu; 1995RUN= @set -eu;
2021 1996
2022See mk/misc/common.mk for the existing definition. 1997See mk/misc/common.mk for the existing definition.
2023 1998
20248.4.10. Turn off verbose logging 19998.4.9. Turn off verbose logging
2025 2000
2026The build logs of a package are often quite long. This allows error messages or 2001The build logs of a package are often quite long. This allows error messages or
2027other interesting details to hide between the noise. To make the actual error 2002other interesting details to hide between the noise. To make the actual error
2028message stand out more, add these lines to mk.conf. 2003message stand out more, add these lines to mk.conf.
2029 2004
2030GNU_CONFIGURE_QUIET= yes 2005GNU_CONFIGURE_QUIET= yes
2031MAKE_FLAGS+= -s 2006MAKE_FLAGS+= -s
2032 2007
2033The -s option works for both GNU Make and BSD Make. On exotic platforms with 2008The -s option works for both GNU Make and BSD Make. On exotic platforms with
2034their own make, it may be a little different. 2009their own make, it may be a little different.
2035 2010
20368.4.11. Select random sets of options 
2037 
2038Most bulk builds run with the default package options. This means that other 
2039combinations of options are not regularly tested. To do this, run a bulk build 
2040with these configurations. 
2041 
2042 * no options enabled 
2043 
2044 * all options enabled 
2045 
2046 * 2n + 0 
2047 
2048 * 2n + 1 
2049 
2050 * 4n + 0..1 
2051 
2052 * 4n + 2..3 
2053 
2054 * 8n + 0..3 
2055 
2056 * 8n + 4..7 
2057 
2058 * 2048n + 0..1023 
2059 
2060 * 2048n + 1024..2047 
2061 
2062Open questions are: 
2063 
2064 * how to collect all options from the entire pkgsrc 
2065 
2066 * how to handle mutually exclusive options 
2067 
2068 * the sets of mutually exclusive options are defined per-package 
2069 
2070 * the sets of nonempty sets are defined per-package 
2071 
20728.4.12. Select random configurations of BUILD_DEFS 
2073 
2074Just like the PKG_OPTIONS, the BUILD_DEFS also allow different variants of 
2075pkgsrc to be created. The same ideas as in Section 8.4.11, "Select random sets 
2076of options" apply. 
2077 
20788.5. Creating a multiple CD-ROM packages collection 20118.5. Creating a multiple CD-ROM packages collection
2079 2012
2080After your pkgsrc bulk-build has completed, you may wish to create a CD-ROM set 2013After your pkgsrc bulk-build has completed, you may wish to create a CD-ROM set
2081of the resulting binary packages to assist in installing packages on other 2014of the resulting binary packages to assist in installing packages on other
2082machines. The pkgtools/cdpack package provides a simple tool for creating the 2015machines. The pkgtools/cdpack package provides a simple tool for creating the
2083ISO 9660 images. cdpack arranges the packages on the CD-ROMs in a way that 2016ISO 9660 images. cdpack arranges the packages on the CD-ROMs in a way that
2084keeps all the dependencies for a given package on the same CD as that package. 2017keeps all the dependencies for a given package on the same CD as that package.
2085 2018
20868.5.1. Example of cdpack 20198.5.1. Example of cdpack
2087 2020
2088Complete documentation for cdpack is found in the cdpack(1) man page. The 2021Complete documentation for cdpack is found in the cdpack(1) man page. The
2089following short example assumes that the binary packages are left in /usr/ 2022following short example assumes that the binary packages are left in /usr/
2090pkgsrc/packages/All and that sufficient disk space exists in /u2 to hold the 2023pkgsrc/packages/All and that sufficient disk space exists in /u2 to hold the