Fri Apr 4 18:31:25 2008 UTC ()
Add a note about sharing sandboxes between master and slave based on
discussion with Havard. Add a reference to the bulkbuild-restart script.


(joerg)
diff -r1.3 -r1.4 pkgsrc/doc/HOWTO-pbulk
cvs diff -r1.3 -r1.4 pkgsrc/doc/HOWTO-pbulk (expand / switch to unified diff)

--- pkgsrc/doc/HOWTO-pbulk 2008/02/15 07:37:11 1.3
+++ pkgsrc/doc/HOWTO-pbulk 2008/04/04 18:31:25 1.4
@@ -3,26 +3,30 @@ Step One: Suggested File System Layout @@ -3,26 +3,30 @@ Step One: Suggested File System Layout
3For the DragonFly builds I'm using: 3For the DragonFly builds I'm using:
4- /bulklog for the bulk meta data and log files of each package 4- /bulklog for the bulk meta data and log files of each package
5- /scratch as WRKOBJDIR 5- /scratch as WRKOBJDIR
6- /distfiles as DISTDIR 6- /distfiles as DISTDIR
7- /packages as PACKAGES 7- /packages as PACKAGES
8- separate physical file systems for /var and /usr/pkg 8- separate physical file systems for /var and /usr/pkg
9All file systems but those are read-only, /tmp and /dev need platform 9All file systems but those are read-only, /tmp and /dev need platform
10specific handling either as tmpfs or link. ptyfs helps for /dev. 10specific handling either as tmpfs or link. ptyfs helps for /dev.
11 11
12For parallel builds, /bulklog, /packages and /usr/pkgsrc are supposed to 12For parallel builds, /bulklog, /packages and /usr/pkgsrc are supposed to
13be shared (null mounts, NFS). /usr/pkg and /var/db/pkg needs to be 13be shared (null mounts, NFS). /usr/pkg and /var/db/pkg needs to be
14per-node and /scratch should be per-node. 14per-node and /scratch should be per-node.
15 15
 16The defaults scripts will remove /usr/pkg and /var/db/pkg on the master
 17node as well. When using chroot sandboxes for the build, the master can
 18share a sandbox with one of the build instances.
 19
16Step Two: Preparing pbulk prefix 20Step Two: Preparing pbulk prefix
17 21
18Install pkgtools/pbulk and possibly helper programs like screen into 22Install pkgtools/pbulk and possibly helper programs like screen into
19LOCALBASE=/usr/pkg_bulk, PKG_DBDIR=/usr/pkg_bulk/.pkgdb. A full 23LOCALBASE=/usr/pkg_bulk, PKG_DBDIR=/usr/pkg_bulk/.pkgdb. A full
20bootstrap is the recommended approach. Copy /usr/pkg_bulk to the client 24bootstrap is the recommended approach. Copy /usr/pkg_bulk to the client
21nodes for parallel builds. 25nodes for parallel builds.
22 26
23Step Three: Configure pbulk 27Step Three: Configure pbulk
24 28
25(a) 29(a)
26Setup general build parameters in mk.conf. Below is an example 30Setup general build parameters in mk.conf. Below is an example
27fragment to add. 31fragment to add.
28 32
@@ -45,26 +49,30 @@ Edit /usr/pkg_bulk/etc/pbulk.conf: @@ -45,26 +49,30 @@ Edit /usr/pkg_bulk/etc/pbulk.conf:
45- base_url goes into the report mail, the rsync targets are used 49- base_url goes into the report mail, the rsync targets are used
46 for uploads. 50 for uploads.
47- for limited bulk builds, set limited_list to a file with one 51- for limited bulk builds, set limited_list to a file with one
48 location per line 52 location per line
49 53
50Step Four: Running pbulk 54Step Four: Running pbulk
51 55
52/usr/pkg_bulk/bin/bulkbuild in screen runs the full build. 56/usr/pkg_bulk/bin/bulkbuild in screen runs the full build.
53 57
54/usr/pkg_bulk/libexec/pbulk/{pre-build,scan,build,report,upload} are the 58/usr/pkg_bulk/libexec/pbulk/{pre-build,scan,build,report,upload} are the
55individual build phases. They depend on the execution of the phase 59individual build phases. They depend on the execution of the phase
56before. Exceptions are report and upload which both depend on build. 60before. Exceptions are report and upload which both depend on build.
57 61
 62/usr/pkg_bulk/bin/bulkbuild-restart can be used to resume an aborted
 63build. If it failed after the actual build phase, report and upload should
 64be called individually though.
 65
58--- sample mk.conf fragment --- 66--- sample mk.conf fragment ---
59 67
60WRKOBJDIR = /scratch 68WRKOBJDIR = /scratch
61PKGSRCDIR = /usr/pkgsrc 69PKGSRCDIR = /usr/pkgsrc
62DISTDIR = /distfiles 70DISTDIR = /distfiles
63PACKAGES = /packages 71PACKAGES = /packages
64 72
65FAILOVER_FETCH= yes 73FAILOVER_FETCH= yes
66 74
67X11_TYPE= modular 75X11_TYPE= modular
68 76
69_ACCEPTABLE= yes 77_ACCEPTABLE= yes
70ALLOW_VULNERABLE_PACKAGES= yes 78ALLOW_VULNERABLE_PACKAGES= yes