Sat May 20 15:43:53 2023 UTC ()
doc/guide: move the PREFER section to the user's guide and rewrite it

The variables PREFER_PKGSRC and PREFER_NATIVE are user-settable,
therefore it was confusing to see them documented only in the developer
part.

Rewrite the whole section to sync with reality since neither of these
variables is tested for the word "no".


(rillig)
diff -r1.48 -r1.49 pkgsrc/doc/guide/files/buildlink.xml
diff -r1.59 -r1.60 pkgsrc/doc/guide/files/configuring.xml
cvs diff -r1.48 -r1.49 pkgsrc/doc/guide/files/buildlink.xml (expand / switch to unified diff)

--- pkgsrc/doc/guide/files/buildlink.xml 2023/01/24 14:32:37 1.48
+++ pkgsrc/doc/guide/files/buildlink.xml 2023/05/20 15:43:53 1.49
@@ -1,15 +1,15 @@ @@ -1,15 +1,15 @@
1<?xml version="1.0" encoding="utf-8"?> 1<?xml version="1.0" encoding="utf-8"?>
2<!-- $NetBSD: buildlink.xml,v 1.48 2023/01/24 14:32:37 wiz Exp $ --> 2<!-- $NetBSD: buildlink.xml,v 1.49 2023/05/20 15:43:53 rillig Exp $ -->
3 3
4<chapter id="buildlink"> 4<chapter id="buildlink">
5 <title>Buildlink methodology</title> 5 <title>Buildlink methodology</title>
6 6
7 <para>Buildlink is a framework in pkgsrc that controls what headers and libraries 7 <para>Buildlink is a framework in pkgsrc that controls what headers and libraries
8 are seen by a package's configure and build processes. This is implemented 8 are seen by a package's configure and build processes. This is implemented
9 in a two step process:</para> 9 in a two step process:</para>
10 10
11 <orderedlist> 11 <orderedlist>
12 <listitem> 12 <listitem>
13 <para>Symlink headers and libraries for dependencies into 13 <para>Symlink headers and libraries for dependencies into
14 <varname>BUILDLINK_DIR</varname>, which by default is a subdirectory 14 <varname>BUILDLINK_DIR</varname>, which by default is a subdirectory
15 of <varname>WRKDIR</varname>.</para> 15 of <varname>WRKDIR</varname>.</para>
@@ -560,83 +560,15 @@ CHECK_BUILTIN.foo?= no @@ -560,83 +560,15 @@ CHECK_BUILTIN.foo?= no
560 is <quote>no</quote> because we may make the determination 560 is <quote>no</quote> because we may make the determination
561 that the built-in version of the software is similar enough to 561 that the built-in version of the software is similar enough to
562 be used as a replacement.</para> 562 be used as a replacement.</para>
563 563
564 <para>The last section is guarded by 564 <para>The last section is guarded by
565 <varname>CHECK_BUILTIN.<replaceable>pkg</replaceable></varname>, 565 <varname>CHECK_BUILTIN.<replaceable>pkg</replaceable></varname>,
566 and includes code that uses the value of 566 and includes code that uses the value of
567 <varname>USE_BUILTIN.<replaceable>pkg</replaceable></varname> 567 <varname>USE_BUILTIN.<replaceable>pkg</replaceable></varname>
568 set in the previous section. This typically includes, e.g., 568 set in the previous section. This typically includes, e.g.,
569 adding additional dependency restrictions and listing additional 569 adding additional dependency restrictions and listing additional
570 files to symlink into <filename>${BUILDLINK_DIR}</filename> (via 570 files to symlink into <filename>${BUILDLINK_DIR}</filename> (via
571 <varname>BUILDLINK_FILES.<replaceable>pkg</replaceable></varname>).</para> 571 <varname>BUILDLINK_FILES.<replaceable>pkg</replaceable></varname>).</para>
572 </sect2> 572 </sect2>
573 
574 <sect2 id="native-or-pkgsrc-preference"> 
575 <title>Global preferences for native or pkgsrc software</title> 
576 
577 <para>When building packages, it's possible to choose whether to set 
578 a global preference for using either the built-in (native) 
579 version or the pkgsrc version of software to satisfy a 
580 dependency. This is controlled by setting 
581 <varname>PREFER_PKGSRC</varname> and 
582 <varname>PREFER_NATIVE</varname>. These variables take values 
583 of either <quote>yes</quote>, <quote>no</quote>, or a list of 
584 packages. <varname>PREFER_PKGSRC</varname> tells pkgsrc to 
585 use the pkgsrc versions of software, while 
586 <varname>PREFER_NATIVE</varname> tells pkgsrc to use the 
587 built-in versions. Preferences are determined by the most 
588 specific instance of the package in either 
589 <varname>PREFER_PKGSRC</varname> or 
590 <varname>PREFER_NATIVE</varname>. If a package is specified 
591 in neither or in both variables, then 
592 <varname>PREFER_PKGSRC</varname> has precedence over 
593 <varname>PREFER_NATIVE</varname>. For example, to require 
594 using pkgsrc versions of software for all but the most basic 
595 bits on a NetBSD system, you can set:</para> 
596 
597<programlisting> 
598PREFER_PKGSRC= yes 
599PREFER_NATIVE= getopt skey tcp_wrappers 
600</programlisting> 
601 
602 <para>A package <emphasis>must</emphasis> have a 
603 <filename>builtin.mk</filename> 
604 file to be listed in <varname>PREFER_NATIVE</varname>, 
605 otherwise it is simply ignored in that list.</para> 
606 
607 <para><varname>PREFER_PKGSRC</varname> and 
608 <varname>PREFER_NATIVE</varname> should be set during bootstrap 
609 to ensure that the bootstrap process does not use inapropriate 
610 native tools as dependencies for core packages. 
611 
612<screen> 
613&rprompt; <userinput>./bootstrap --prefer-pkgsrc yes --prefer-native openssl</userinput> 
614</screen> 
615 
616 Switching between settings globally at a later date can introduce 
617 complications with dependency resolution. This is caused by packages 
618 built with the opposite preference being installed alongside each 
619 other. Hence, any changes to these variables after bootstrap will 
620 necessitate rebuilding all packages depending on one whose preference 
621 has been changed. This is not trivial and should be avoided.</para> 
622 
623 <para>When using pkgsrc on Linux systems, there is high risk of 
624 "leakage", where programs installed by pkgsrc may inadvertently use a 
625 command or library not installed by pkgsrc, e.g. those installed by 
626 yum or apt. Such foreign dependencies may be installed, removed, or 
627 upgraded to a version incompatible with the pkgsrc package at any 
628 time, causing pkgsrc packages to subsequently malfunction. Pkgsrc 
629 cannot prevent this, as it has no control over other package managers. 
630 Another potential problem is that under Redhat Enterprise and related 
631 Linux systems, yum packages are only patched and never upgraded, so 
632 eventually they may become too outdated for use by pkgsrc. Even 
633 intentionally using foreign dependencies, not considered leakage, can 
634 lead to these problems, so it is generally discouraged. In order to 
635 minimize such problems, PREFER_PKGSRC defaults to "yes" on Linux systems. 
636 This ensures that pkgsrc is aware of any changes to dependency packages 
637 and can rebuild or upgrade the entire dependency tree as needed. This 
638 default can be overridden by setting --prefer-pkgsrc to "no" or a list of 
639 packages, or by setting --prefer-native to "yes".</para> 
640 </sect2> 
641 </sect1> 573 </sect1>
642</chapter> 574</chapter>
cvs diff -r1.59 -r1.60 pkgsrc/doc/guide/files/configuring.xml (expand / switch to unified diff)

--- pkgsrc/doc/guide/files/configuring.xml 2023/01/05 01:42:15 1.59
+++ pkgsrc/doc/guide/files/configuring.xml 2023/05/20 15:43:53 1.60
@@ -1,14 +1,14 @@ @@ -1,14 +1,14 @@
1<!-- $NetBSD: configuring.xml,v 1.59 2023/01/05 01:42:15 gutteridge Exp $ --> 1<!-- $NetBSD: configuring.xml,v 1.60 2023/05/20 15:43:53 rillig Exp $ -->
2 2
3<chapter id="configuring"> 3<chapter id="configuring">
4 <title>Configuring pkgsrc</title> 4 <title>Configuring pkgsrc</title>
5 5
6<anchor id="mk.conf" /> 6<anchor id="mk.conf" />
7 7
8<para>The whole pkgsrc system is configured in a single file, usually 8<para>The whole pkgsrc system is configured in a single file, usually
9called <filename>mk.conf</filename>. In which directory pkgsrc looks for 9called <filename>mk.conf</filename>. In which directory pkgsrc looks for
10that file depends on the installation. On NetBSD, when you use 10that file depends on the installation. On NetBSD, when you use
11&man.make.1; from the base system, it is in the directory 11&man.make.1; from the base system, it is in the directory
12<filename>/etc/</filename>. In all other cases the default location is 12<filename>/etc/</filename>. In all other cases the default location is
13<literal>${PREFIX}/etc/</literal>, depending on where you told the 13<literal>${PREFIX}/etc/</literal>, depending on where you told the
14bootstrap program to install the binary packages.</para> 14bootstrap program to install the binary packages.</para>
@@ -109,26 +109,105 @@ spelling mistakes) takes place. @@ -109,26 +109,105 @@ spelling mistakes) takes place.
109 <listitem><para><varname>PKGMAKECONF</varname>: Location of 109 <listitem><para><varname>PKGMAKECONF</varname>: Location of
110 the &mk.conf; file used by a package's 110 the &mk.conf; file used by a package's
111 BSD-style Makefile. If this is not set, 111 BSD-style Makefile. If this is not set,
112 <varname>MAKECONF</varname> is set to 112 <varname>MAKECONF</varname> is set to
113 <filename>/dev/null</filename> to avoid picking up 113 <filename>/dev/null</filename> to avoid picking up
114 settings used by builds in <filename>/usr/src</filename>.</para> 114 settings used by builds in <filename>/usr/src</filename>.</para>
115 </listitem> 115 </listitem>
116 116
117 <!-- DISTDIR --> 117 <!-- DISTDIR -->
118 <!-- FAILOVER_FETCH --> 118 <!-- FAILOVER_FETCH -->
119 </itemizedlist></para> 119 </itemizedlist></para>
120 </sect1> 120 </sect1>
121 121
 122 <sect1 id="native-or-pkgsrc-preference">
 123 <title>Preferences for native or pkgsrc software</title>
 124
 125 <para>Whenever a package depends on a package that has a
 126 <filename>builtin.mk</filename> file, the dependent package can
 127 either use the built-in (native) version from the base system or the
 128 pkgsrc-provided version. This only affects dependencies, so it is
 129 still possible to build the pkgsrc package <filename
 130 role="pkg">devel/pcre++</filename> even when other packages depend
 131 on the native pcre++ version instead.</para>
 132
 133 <para>To force using the pkgsrc-provided version for a particular
 134 package, define <quote><varname>PREFER_PKGSRC</varname> =
 135 <replaceable>package-ID</replaceable></quote> in &mk.conf;. To force
 136 using the native package, define
 137 <quote><varname>PREFER_NATIVE</varname> =
 138 <replaceable>package-ID</replaceable></quote>. In both cases, the
 139 <replaceable>package-ID</replaceable> is the one from the
 140 <filename>buildlink3.mk</filename> of the package. In most cases,
 141 this ID is the same as the directory name of the package, but for
 142 example, <filename role="pkgpath">devel/pcre++</filename> has the
 143 package ID <quote>pcrexx</quote>.</para>
 144
 145 <para>For the packages that are not listed by their package ID,
 146 pkgsrc uses the pkgsrc-provided version if
 147 <varname>PREFER_PKGSRC</varname> contains the word
 148 <quote>yes</quote>. Otherwise, if <varname>PREFER_NATIVE</varname>
 149 contains the word <quote>yes</quote>, pkgsrc uses the native
 150 version. For example, to require using the pkgsrc-provided versions
 151 for all but the most basic bits on a NetBSD system, you can
 152 set:</para>
 153
 154<programlisting>
 155PREFER_PKGSRC= yes
 156PREFER_NATIVE= getopt skey tcp_wrappers
 157</programlisting>
 158
 159 <para>A package <emphasis>must</emphasis> have a
 160 <filename>builtin.mk</filename> file to be listed in
 161 <varname>PREFER_NATIVE</varname>, otherwise it is simply ignored in
 162 that list.</para>
 163
 164 <para><varname>PREFER_PKGSRC</varname> and
 165 <varname>PREFER_NATIVE</varname> should be set during bootstrap to
 166 ensure that the bootstrap process does not use inapropriate native
 167 tools as dependencies for core packages.</para>
 168
 169<screen>
 170&rprompt; <userinput>./bootstrap --prefer-pkgsrc yes --prefer-native openssl</userinput>
 171</screen>
 172
 173 <para>Switching between settings globally at a later date can
 174 introduce complications with dependency resolution. This is caused
 175 by packages built with the opposite preference being installed
 176 alongside each other. Hence, when changing any of these variables
 177 after bootstrap, you need to rebuild all packages depending on those
 178 whose preference has been changed. This is not trivial and should
 179 be avoided.</para>
 180
 181 <para>When using pkgsrc on Linux systems, there is high risk of
 182 <quote>leakage</quote>, where programs installed by pkgsrc may
 183 inadvertently use a command or library not installed by pkgsrc, e.g.
 184 those installed by yum or apt. Such foreign dependencies may be
 185 installed, removed, or upgraded to a version incompatible with the
 186 pkgsrc package at any time, causing pkgsrc packages to subsequently
 187 malfunction. Pkgsrc cannot prevent this, as it has no control over
 188 other package managers. Another potential problem is that under
 189 Redhat Enterprise and related Linux systems, yum packages are only
 190 patched and never upgraded, so eventually they may become too
 191 outdated for use by pkgsrc. Even intentionally using foreign
 192 dependencies, not considered leakage, can lead to these problems, so
 193 it is generally discouraged. In order to minimize such problems,
 194 PREFER_PKGSRC defaults to <quote>yes</quote> on Linux systems. This ensures that
 195 pkgsrc is aware of any changes to dependency packages and can
 196 rebuild or upgrade the entire dependency tree as needed. This
 197 default can be overridden by setting --prefer-pkgsrc to a
 198 list of packages and --prefer-native to <quote>yes</quote>.</para>
 199 </sect1>
 200
122 <sect1 id="variables-affecting-installation"> 201 <sect1 id="variables-affecting-installation">
123 <title>Variables affecting the installation process</title> 202 <title>Variables affecting the installation process</title>
124 203
125 <itemizedlist> 204 <itemizedlist>
126 <listitem><para><varname>PKGSRC_KEEP_BIN_PKGS</varname>: 205 <listitem><para><varname>PKGSRC_KEEP_BIN_PKGS</varname>:
127 By default, binary packages of built packages are 206 By default, binary packages of built packages are
128 preserved in <filename>${PACKAGES}/All</filename>. Setting 207 preserved in <filename>${PACKAGES}/All</filename>. Setting
129 this variable to "no" prevents this.</para> 208 this variable to "no" prevents this.</para>
130 </listitem> 209 </listitem>
131 </itemizedlist> 210 </itemizedlist>
132 211
133 <para>Packages have to support installation into a 212 <para>Packages have to support installation into a
134 subdirectory of <varname>WRKDIR</varname>. This allows a package 213 subdirectory of <varname>WRKDIR</varname>. This allows a package