 @@ -1,439 +1,449 @@
-<!-- $NetBSD: creating.xml,v 1.14 2014/01/12 09:43:30 wiz Exp $ -->
+<!-- $NetBSD: creating.xml,v 1.15 2014/01/18 20:17:10 wiz Exp $ -->
 <chapter id="creating">
 <title>Creating a new pkgsrc package from scratch</title>
 <para>When you find a package that is not yet in pkgsrc, you
 most likely have a URL from where you can download the source
 code. Starting with this URL, creating a package involves only a
 few steps.</para>
 <procedure>
 <step><para>First, install the packages <filename
 role="pkg">pkgtools/url2pkg</filename> and <filename
 role="pkg">pkgtools/pkglint</filename>.</para></step>
 <step><para>Then, choose one of the top-level directories as the
 category in which you want to place your package. You can also create a
 directory of your own (maybe called <filename>local</filename>). In that
 category directory, create another directory for your package and change
 into it.</para></step>
 <step><para>Run the program <command>url2pkg</command>, which will ask
 you for a URL. Enter the URL of the distribution file (in most cases a
 <filename>.tar.gz</filename> file) and watch how the basic ingredients
 of your package are created automatically. The distribution file is
 extracted automatically to fill in some details in the
 <filename>Makefile</filename> that would otherwise have to be done
 manually.</para></step>
 <step><para>Examine the extracted files to determine the dependencies of
 your package. Ideally, this is mentioned in some
 <filename>README</filename> file, but things may differ. For each of
 these dependencies, look where it exists in pkgsrc, and if there is a
 file called <filename>buildlink3.mk</filename> in that directory, add a
 line to your package <filename>Makefile</filename> which includes that
 file just before the last line. If the
 <filename>buildlink3.mk</filename> file does not exist, it must be
 created first. The <filename>buildlink3.mk</filename> file makes sure that the package's include files and libraries are provided.</para>
 <para>If you just need binaries from a package, add a
 <varname>DEPENDS</varname> line to the Makefile, which specifies the
 version of the dependency and where it can be found in pkgsrc. This line
 should be placed in the third paragraph. If the dependency is only
 needed for building the package, but not when using it, use
 <varname>BUILD_DEPENDS</varname> instead of <varname>DEPENDS</varname>.
 Your package may then look like this:</para>
 <programlisting>
 [...]
 BUILD_DEPENDS+= lua>=5.0:../../lang/lua
 DEPENDS+=       screen-[0-9]*:../../misc/screen
 DEPENDS+=       screen>=4.0:../../misc/screen
 [...]
 .include "../../<replaceable>category</replaceable>/<replaceable>package</replaceable>/buildlink3.mk"
 .include "../../devel/glib2/buildlink3.mk"
 .include "../../mk/bsd.pkg.mk"
 </programlisting>
 </step>
 <step><para>Run <command>pkglint</command> to see what things still need
 to be done to make your package a <quote>good</quote> one. If you don't
 know what pkglint's warnings want to tell you, try <command>pkglint
 --explain</command> or <command>pkglint
 -e</command>, which outputs additional
 explanations.</para></step>
 <step><para>In many cases the package is not yet ready to build. You can
 find instructions for the most common cases in the next section, <xref
 linkend="creating.common"/>. After you have followed the instructions
 over there, you can hopefully continue here.</para></step>
 <step><para>Run <command>bmake clean</command> to clean the working
 directory from the extracted files. Besides these files, a lot of cache
 files and other system information has been saved in the working
 directory, which may become wrong after you edited the
 <filename>Makefile</filename>.</para></step>
 <step><para>Now, run <command>bmake</command> to build the package. For
 the various things that can go wrong in this phase, consult <xref
 linkend="fixes"/>.</para></step>
 <step><para>When the package builds fine, the next step is to install
 the package. Run <command>bmake install</command> and hope that
 everything works.</para></step>
 <step><para>Up to now, the file <filename>PLIST</filename>, which
 contains a list of the files that are installed by the package, is
 nearly empty. Run <command>bmake print-PLIST
 &gt;PLIST</command> to generate a probably correct list. Check
 the file using your preferred text editor to see if the list of
 files looks plausible.</para></step>
 <step><para>Run <command>pkglint</command> again to see if the generated
 <filename>PLIST</filename> contains garbage or not.</para></step>
 <step><para>When you ran <command>bmake install</command>, the package
 has been registered in the database of installed files, but with an
 empty list of files. To fix this, run <command>bmake deinstall</command>
 and <command>bmake install</command> again. Now the package is
 registered with the list of files from
 <filename>PLIST</filename>.</para></step>
 <step><para>Run <command>bmake package</command> to create a binary
 package from the set of installed files.</para></step>
 </procedure>
 <sect1 id="creating.common">
 <title>Common types of packages</title>
 <sect2 id="creating.perl-module">
 <title>Perl modules</title>
 <para>Simple Perl modules are handled automatically by
 <command>url2pkg</command>, including dependencies.</para>
 </sect2>
 <sect2 id="creating.kde-app">
 <title>KDE applications</title>
 <para>KDE applications should always include
 <filename>meta-pkgs/kde3/kde3.mk</filename>, which contains numerous
 settings that are typical of KDE packages.</para>
 </sect2>
 <sect2 id="creating.python-module">
 <title>Python modules and programs</title>
 <para>Python modules and programs packages are easily created using a
 set of predefined variables.</para>
 <para>Most Python packages use either <quote>distutils</quote> or
 easy-setup (<quote>eggs</quote>).
 If the software uses <quote>distutils</quote>, set the
 <varname>PYDISTUTILSPKG</varname> variable to <quote>yes</quote> so
 pkgsrc will make use of this framework.
 <quote>distutils</quote> uses a script called <filename>setup.py</filename>,
 if the <quote>distutils</quote> driver is not called
 <filename>setup.py</filename>, set the <varname>PYSETUP</varname> variable
 to the name of the script.</para>
 <para>
-If the default Python versions are not supported by the software, set the
+If some Python versions are not supported by the software, set the
-<varname>PYTHON_VERSIONS_ACCEPTED</varname> variable to the Python versions
+<varname>PYTHON_VERSIONS_INCOMPATIBLE</varname> variable to the Python versions
-the software is known to work with, from the most recent to the older
+that are not supported, e.g.
 one, e.g.
 <programlisting>
-PYTHON_VERSIONS_ACCEPTED=       33 27 26
+PYTHON_VERSIONS_ACCEPTED=       26
 </programlisting></para>
 <para>
 If the packaged software is a Python module, include
 <quote><filename>../../lang/python/extension.mk</filename></quote>.
 In this case, the package directory should be called
 <quote>py-software</quote> and <varname>PKGNAME</varname> should be set to
 <quote>${PYPKGPREFIX}-${DISTNAME}</quote>, e.g.
 <programlisting>
 DISTNAME=   foopymodule-1.2.10
 PKGNAME=    ${PYPKGPREFIX}-${DISTNAME}
 </programlisting></para>
-<para>If it is an application, also include
+<para>If it is an application, include
-<quote><filename>../../lang/python/application.mk</filename></quote>
+<quote><filename>../../lang/python/application.mk</filename></quote>.
-before <quote>extension.mk</quote>.</para>
+In order to correctly set the path to the Python interpreter, use the
 <para>If the packaged software, either it is an application or a module, is
 egg-aware, you only need to include
 <quote><filename>../../lang/python/egg.mk</filename></quote>.</para>
 <para>In order to correctly set the path to the Python interpreter, use the
 <varname>REPLACE_PYTHON</varname> variable and set it to the list of files
 (paths relative to <varname>WRKSRC</varname>) that must be corrected.
-For example :
+For example:
 <programlisting>
 REPLACE_PYTHON=   *.py
 </programlisting></para>
 <para>Most Python packages use either <quote>distutils</quote> or
 easy-setup (<quote>eggs</quote>).
 If the software uses <quote>distutils</quote>, include
 <quote><filename>../../lang/python/distutils.mk</filename></quote>.
 so pkgsrc will use this framework.
 <quote>distutils</quote> uses a script called <filename>setup.py</filename>,
 if the <quote>distutils</quote> driver is not called
 <filename>setup.py</filename>, set the <varname>PYSETUP</varname> variable
 to the name of the script.</para>
 <para>Otherwise, if the packaged software is egg-aware, you only need
 to include
 <quote><filename>../../lang/python/egg.mk</filename></quote>.</para>
 <para>Some Python modules have separate distributions for Python-2.x
 and Python-3.x support. In pkgsrc this is handled by the
 <filename>versioned_dependencies.mk</filename> file. Set
 <varname>PYTHON_VERSIONED_DEPENDENCIES</varname> to the list of
 packages that should be depended upon and include
 <quote><filename>../../lang/python/versioned_dependencies.mk</filename></quote>,
 then the pkgsrc infrastructure will depend on the appropriate package
 version. For example:
 <programlisting>
 PYTHON_VERSIONED_DEPENDENCIES=dateutil dns
 </programlisting>
 Look inside <filename>versioned_dependencies.mk</filename> for a list
 of supported packages.</para>
 </sect2>
 </sect1>
 <sect1 id="creating.examples">
 <title>Examples</title>
 <sect2 id="creating.nvu">
 <title>How the www/nvu package came into pkgsrc</title>
 <sect3 id="creating.nvu.init">
 <title>The initial package</title>
 <para>Looking at the file <filename>pkgsrc/doc/TODO</filename>, I saw
 that the <quote>nvu</quote> package has not yet been imported into
 pkgsrc. As the description says it has to do with the web, the obvious
 choice for the category is <quote>www</quote>.</para>
 <programlisting>
 &uprompt; mkdir www/nvu
 &uprompt; cd www/nvu
 </programlisting>
 <para>The web site says that the sources are available as a tar file, so
 I fed that URL to the <command>url2pkg</command> program:</para>
 <programlisting>
 &uprompt; url2pkg http://cvs.nvu.com/download/nvu-1.0-sources.tar.bz2
 </programlisting>
 <para>My editor popped up, and I added a <varname>PKGNAME</varname> line
 below the <varname>DISTNAME</varname> line, as the package name should
 not have the word <quote>sources</quote> in it. I also filled in the
 <varname>MAINTAINER</varname>, <varname>HOMEPAGE</varname> and
 <varname>COMMENT</varname> fields. Then the package
 <filename>Makefile</filename> looked like that:</para>
 <programlisting>
 # &#36;NetBSD&#36;
+#
 DISTNAME=       nvu-1.0-sources
 PKGNAME=        nvu-1.0
 CATEGORIES=     www
 MASTER_SITES=   http://cvs.nvu.com/download/
 EXTRACT_SUFX=   .tar.bz2
 MAINTAINER=     rillig@NetBSD.org
 HOMEPAGE=       http://cvs.nvu.com/
 COMMENT=        Web Authoring System
 # url2pkg-marker (please do not remove this line.)
 .include "../../mk/bsd.pkg.mk"
 </programlisting>
 <para>Then, I quit the editor and watched pkgsrc downloading a large
 source archive:</para>
 <programlisting>
 url2pkg> Running "make makesum" ...
 => Required installed package digest>=20010302: digest-20060826 found
 => Fetching nvu-1.0-sources.tar.bz2
 Requesting http://cvs.nvu.com/download/nvu-1.0-sources.tar.bz2
 % |*************************************| 28992 KB  150.77 KB/s00:00 ETA
 29687976 bytes retrieved in 03:12 (150.77 KB/s)
 url2pkg> Running "make extract" ...
 => Required installed package digest>=20010302: digest-20060826 found
 => Checksum SHA1 OK for nvu-1.0-sources.tar.bz2
 => Checksum RMD160 OK for nvu-1.0-sources.tar.bz2
 work.bacc -> /tmp/roland/pkgsrc/www/nvu/work.bacc
 ===> Installing dependencies for nvu-1.0
 ===> Overriding tools for nvu-1.0
 ===> Extracting for nvu-1.0
 url2pkg> Adjusting the Makefile.
 Remember to correct CATEGORIES, HOMEPAGE, COMMENT, and DESCR when you're done!
 Good luck! (See pkgsrc/doc/pkgsrc.txt for some more help :-)
 </programlisting>
 </sect3>
 <sect3 id="creating.nvu.problems">
 <title>Fixing all kinds of problems to make the package work</title>
 <para>Now that the package has been extracted, let's see what's inside
 it. The package has a <filename>README.txt</filename>, but that only
 says something about mozilla, so it's probably useless for seeing what
 dependencies this package has. But since there is a GNU configure script
 in the package, let's hope that it will complain about everything it
 needs.</para>
 <programlisting>
 &uprompt; bmake
 => Required installed package digest>=20010302: digest-20060826 found
 => Checksum SHA1 OK for nvu-1.0-sources.tar.bz2
 => Checksum RMD160 OK for nvu-1.0-sources.tar.bz2
 ===> Patching for nvu-1.0
 ===> Creating toolchain wrappers for nvu-1.0
 ===> Configuring for nvu-1.0
 [...]
 configure: error: Perl 5.004 or higher is required.
 [...]
 WARNING: Please add USE_TOOLS+=perl to the package Makefile.
 [...]
 </programlisting>
 <para>That worked quite well. So I opened the package Makefile in my
 editor, and since it already has a <varname>USE_TOOLS</varname> line, I
 just appended <quote>perl</quote> to it. Since the dependencies of the
 package have changed now, and since a perl wrapper is automatically
 installed in the <quote>tools</quote> phase, I need to build the package
 from scratch.</para>
 <programlisting>
 &uprompt; bmake clean
 ===> Cleaning for nvu-1.0
 &uprompt; bmake
 [...]
 *** /tmp/roland/pkgsrc/www/nvu/work.bacc/.tools/bin/make is not \
 GNU Make.  You will not be able to build Mozilla without GNU Make.
 [...]
 </programlisting>
 <para>So I added <quote>gmake</quote> to the
 <varname>USE_TOOLS</varname> line and tried again (from scratch).</para>
 <programlisting>
 [...]
 checking for GTK - version >= 1.2.0... no
 *** Could not run GTK test program, checking why...
 [...]
 </programlisting>
 <para>Now to the other dependencies. The first question is: Where is the
 GTK package hidden in pkgsrc?</para>
 <programlisting>
 &uprompt; echo ../../*/gtk*
 [many packages ...]
 &uprompt; echo ../../*/gtk
 ../../x11/gtk
 &uprompt; echo ../../*/gtk2
 ../../x11/gtk2
 &uprompt; echo ../../*/gtk2/bui*
 ../../x11/gtk2/buildlink3.mk
 </programlisting>
 <para>The first try was definitely too broad. The second one had exactly
 one result, which is very good. But there is one pitfall with GNOME
 packages. Before GNOME 2 had been released, there were already many
 GNOME 1 packages in pkgsrc. To be able to continue to use these
 packages, the GNOME 2 packages were imported as separate packages, and
 their names usually have a <quote>2</quote> appended. So I checked
 whether this was the case here, and indeed it was.</para>
 <para>Since the GTK2 package has a <filename>buildlink3.mk</filename>
 file, adding the dependency is very easy. I just inserted an
 <literal>.include</literal> line before the last line of the package
 <filename>Makefile</filename>, so that it now looks like this:</para>
 <programlisting>
 [...]
 .include "../../x11/gtk2/buildlink3.mk"
 .include "../../mk/bsd.pkg.mk
 </programlisting>
 <para>After another <command>bmake clean && bmake</command>, the answer
 was:</para>
 <programlisting>
 [...]
 checking for gtk-config... /home/roland/pkg/bin/gtk-config
 checking for GTK - version >= 1.2.0... no
 *** Could not run GTK test program, checking why...
 *** The test program failed to compile or link. See the file config.log for the
 *** exact error that occured. This usually means GTK was incorrectly installed
 *** or that you have moved GTK since it was installed. In the latter case, you
 *** may want to edit the gtk-config script: /home/roland/pkg/bin/gtk-config
 configure: error: Test for GTK failed.
 [...]
 </programlisting>
 <para>In this particular case, the assumption that <quote>every package
 prefers GNOME 2</quote> had been wrong. The first of the lines above
 told me that this package really wanted to have the GNOME 1 version of
 GTK. If the package had looked for GTK2, it would have looked for
 <command>pkg-config</command> instead of <command>gtk-config</command>.
 So I changed the <literal>x11/gtk2</literal> to
 <literal>x11/gtk</literal> in the package <filename>Makefile</filename>,
 and tried again.</para>
 <programlisting>
 [...]
 cc -o xpidl.o -c -DOSTYPE=\"NetBSD3\" -DOSARCH=\"NetBSD\"  [...]
 In file included from xpidl.c:42:
 xpidl.h:53:24: libIDL/IDL.h: No such file or directory
 In file included from xpidl.c:42:
 xpidl.h:132: error: parse error before "IDL_ns"
 [...]
 </programlisting>
 <para>The package still does not find all of its dependencies. Now the
 question is: Which package provides the
 <filename>libIDL/IDL.h</filename> header file?</para>
 <programlisting>
 &uprompt; echo ../../*/*idl*
 ../../devel/py-idle ../../wip/idled ../../x11/acidlaunch
 &uprompt; echo ../../*/*IDL*
 ../../net/libIDL
 </programlisting>
 <para>Let's take the one from the second try. So I included the
 <filename>../../net/libIDL/buildlink3.mk</filename> file and tried
 again. But the error didn't change. After digging through some of the
 code, I concluded that the build process of the package was broken and
 couldn't have ever worked, but since the Mozilla source tree is quite
 large, I didn't want to fix it. So I added the following to the package
 <filename>Makefile</filename> and tried again:</para>
 <programlisting>
 CPPFLAGS+=              -I${BUILDLINK_PREFIX.libIDL}/include/libIDL-2.0
 BUILDLINK_TRANSFORM+=   -l:IDL:IDL-2
 </programlisting>
 <para>The latter line is needed because the package expects the library
 <filename>libIDL.so</filename>, but only
 <filename>libIDL-2.so</filename> is available. So I told the compiler
 wrapper to rewrite that on the fly.</para>
 <para>The next problem was related to a recent change of the FreeType
 interface. I looked up in <filename role="pkg">www/seamonkey</filename>
 which patch files were relevant for this issue and copied them to the
 <filename>patches</filename> directory. Then I retried, fixed the
 patches so that they applied cleanly and retried again. This time,
 everything worked.</para>
 </sect3>
 <sect3 id="creating.nvu.inst">
 <title>Installing the package</title>
 <programlisting>
 &uprompt; bmake CHECK_FILES=no install
 [...]
 &uprompt; bmake print-PLIST >PLIST
 &uprompt; bmake deinstall
 &uprompt; bmake install
 </programlisting>
 </sect3>
 </sect2>
 </sect1>
 </chapter>