distbb - distributed bulk builds framework for pkgsrc (www.pkgsrc.org).

  DISTBB main features:
     - DISTBB is not fork of PBULK and original pkgsrc Bulk Build Framework.
       Everything in it was written from scratch.
     - DISTBB can build packages in an isolated environment, e.g. chroots
       virtual machines etc. That is working system is kept intact.
     - DISTBB was written with portability in mind.
       Though, currently only NetBSD, Linux, Interix and Solaris are tested.
     - Several machines in a network or chroots can be used for
       building packages. That is, independent packages can be built
       in parallel using all available power (CPUs and/or machines).
     - Either all available packages (default) or only specified packages
       (distbb -f) can be built. Individual packages may also be built
       easily (distbb -p).
     - Collecting an information about source packages
       (pkg_src_summary stage) is also distributed.
       Different packages are analysed in parallel .
     - DISTBB is _really_ modular. Tool for running things in parallel
       is completely independent from DISTBB.
       It is a separate (parallel/paexec)
       universal tool which is developped and packaged separatedly.
       Another thing separated from DISTBB is a collection of tools
       for manipulating so called packages summary. See pkg_summary(5)
       and pkgtools/pkg_summary-utils package for more information.
       Other helper tools are devel/pipestatus and lang/runawk.
     - DISTBB is small. It is as small as Hubert Feyrer's
       bulk build framework and is more than 30% smaller than
       Joerg Sonnenberger's pbulk. At the same time
       it provides more features.
     - DISTBB is written completely in SHELL and AWK. "C" language is
       not used at all. This is just wrong tool for things like DISTBB.
       PERL is not used too. It is too big and I don't like it :-)
     - DISTBB is very flexible. There are two very important things
       about how distbb works. The first is building options
       (options passed to bmake). The second is a list of targets
       (fetch, configure, build etc.). All building options and targets
       are specified in the configuration file (distbb.conf) and may
       be easily changed. DESTDIR support mentioned above is not something
       special. This is a result of just few changes in distbb.conf.
       Another reason why DISTBB is so flexible is that it is possible
       for users to define their own targets in distbb.local.mk config file.
       For example, see dbb_canbebuilt, dbb_excluded and dbb_vars targets.
       And finally one may set building options and target for individual
       package. This gives you a way to, for example, build bootstrap packages
       with USE_DESTDIR=full or to use different (not default) compiler 
       for individual packages (qemu and  festival don't like gcc-4,
       no problem, build them using gcc-3 if it is available on your system).
       Another example is to set/unset "stack smashing protection"
       for individual packages.
     - DISTBB is able to upload building logs to the external server.
     - DISTBB is able to upload binary packages.
       Uploading packages is aware of NO_BIN_ON_CDROM
       and NO_BIN_ON_FTP restrictions. See UPLOAD_PKGS_FILES variable
       in distbb.default.conf.
     - Bulk build results can be sent to the user via
       email (or jabber, icq etc.).
     - Bulk build logs include (optionally) difference with previous
       bulk build and "global" progress sections. This is very powerful
       feature absent (at the time of writing) in bbf and pbulk.
       Command like tool for comparing results of two bulk builds
       is also available.
     - In distributed mode distbb is immune to client hosts' failures.
       In case client host fails, this host is marked as broken
       and distbb continue to work with the rest clients
       moving broken package (if any) to other alive client.
     - Normally DISTBB receives packages in PKGPATH form, e.g. archivers/gzip,
       pkgtools/distbb etc. One can also use so called enriched PKGPATHs.
       Enriching here means additional building options in the following form
         category/package[:var1=value1[,var2=value2[,...]]]
       For example,
         textproc/dictem:EMACS_TYPE=xemacs214
         textproc/dictem:EMACS_TYPE=emacs22
         www/ap2-wsgi:PKG_APACHE=apache22,PYTHON_VERSION_REQD=25
         sysutils/mc:PKG_OPTIONS.mc=-slang~-charset,PKGNAME=mc-light-4.6.1nb4
         sysutils/mc:PKG_OPTIONS.mc=samba,PKGNAME=mc-full-4.6.1nb4
         sysutils/mc:PKG_OPTIONS.mc=-x11,PKGNAME=mc-nox11-4.6.1nb4
       See NEWS file about 0.19.0 release for details.
     - DISTBB generates comprehensive logs.
        * META/report.html - HTML version of the report
        * META/report.txt  - Plain text version of the report

        * META/packages_built_prev.txt - list of packages that was
          built previously and not rebuilt.
        * META/packages_built.txt - list of packages really rebuilt
          and succeeded.
        * META/packages_built_total.txt - list of packages ready for use.

        * META/packages_failed_scan.txt - list of packages for which
          pkg_src_summary failed, e.g. due to syntax errors in Makefile.
        * META/packages_failed_notavail.txt - list of packages marked
          as not available (NOT_FOR_PLATFORM, ONLY_FOR_COMPILER etc.)
        * META/packages_failed.txt - list of packages really failed.
        * META/packages_failed_deps.txt - list of packages failed
          due to broken dependencies.
        * META/packages_failed_total.txt -
          failed_scan + failed_notavail + faild + failed_deps

        * META/pkg_src_summary.txt - src_summary for all analysed packages

        * build_src_summary_log.txt - bmake stderr collected during
          building the src_summary
          (see META/pkg_src_summary.txt and META/packages_failed_scan.txt)

===========================================================================
CONFIGURATION/TUTORIAL:

1) bootstrap your working pkgsrc (to /usr/pkg, for example)

     Example:
       # cd /usr/pkgsrc/bootstrap/
       # ./bootstrap
       ...
       # edit /usr/pkg/etc/mk.conf
       # cd /
       # tar -czf /usr/pkgsrc/packages/bootstrap.tgz /usr/pkg /var/db/pkg
       #

     Example for mk.conf:
       ...
       SKIP_LICENSE_CHECK=             yes
       # PKG_DEVELOPER=                  yes
       ALLOW_VULNERABLE_PACKAGES=      yes

       # I'd strongly recommend to use local directory for building.
       # Building packages on NFSed directories is slooooooooow.
       WRKOBJDIR=                      /tmp/pkgsrc_obj

       # The following line makes future changes in setting easier
       .sinclude "/usr/pkg_distbb/etc/distbb.local.mk"

2) Prepare distbb
   - bootstrap pkgsrc for distbb (to /usr/pkg_distbb, for example)

     Example:
       # cd /usr/pkgsrc/bootstrap/
       # ./bootstrap --prefix /usr/pkg_distbb

   - install pkgtools/distbb in it

     Example:
       # cd /usr/pkgsrc/pkgtools/distbb
       # /usr/pkg_distbb/bin/bmake install

   - install other software you may need (rsync, nbpatch etc.)

   - edit /usr/pkg_distbb/etc/distbb.conf configuration file

========================================
For "distributed" bulk builds:

3d) On master host:
    - make /usr/pkg_distbb and /usr/pkgsrc directories NFS mountable
      for all your slave hosts.

4d) On all slave hosts:
    - mount -t nfs -o ro master:/usr/pkg_distbb /usr/pkg_distbb
      mount -t nfs -o rw master:/usr/pkgsrc     /usr/pkgsrc

    All slave hosts need they own PREFIX (e.g. /usr/pkg)
    and PKGDBDIR (e.g. /var/db/pkg). These dirs will be deleted
    on slaves.

    NOTE:
       Only /usr/pkgsrc/packages and /usr/pkgsrc/distfiles
       subdirectories need to be writeable and therefore can be
       mounted separately. /usr/pkgsrc can be mounted read-only.

5d) On all slave hosts:

    Make sure PATH environment variable includes /usr/pkg/bin
    (edit .profile or .login files for this).

6d) On the master host:
    - /usr/pkg_distbb/bin/distbb -h

      # for full bulk build
      /usr/pkg_distbb/bin/distbb

      # for partial bulk builds
      /usr/pkg_distbb/bin/distbb -f packages.txt # for partial bulk build

      # for building individual packages
      /usr/pkg_distbb/bin/distbb -p 'meta-pkgs/xfce4 multimedia/mplayer'

      # for (re)building installed packages
      /usr/pkg_distbb/bin/distbb -i

  NOTE:
    - in "distributed" mode master's /usr/pkg is NOT changed.
      Neither pkg_add nor pkg_delete is called on master host.
    - /usr/pkgsrc/packages/All/pkg_summary.txt (PKG_SUMMARY),
      /usr/pkgsrc/packages/All/pkg_all_summary.txt (PKG_ALL_SUMMARY) and
      /usr/pkgsrc/pkg_src_summary.txt (PKG_SRC_SUMMARY) are created/updated
      from master host
    - Everything under ~/bulk_log directory (REPORTS_DIR)
      is managed by master host
    - Uploading logs (UPLOAD_LOGS_CMD), packages (UPLOAD_PKGS_CMD)
      and sending a report (REPORT_CMD) is run on master host.
    - On ALL slave hosts PREFIX (/usr/pkg) and PKGDBDIR (/var/db/pkg)
      will be removed and recreated from bootstrap tarball.
    - DistBB can be run on master host by unprivileged user
      even if packages are built using root privileges
    - Master host and slave hosts don't have to run the same OS.
      Slave hosts should run a target operating operating system.
    - By default ssh is used a transport program (TRANSPORT)
      between master and slaves. You can use any ssh-like program
      for this purpose.
    - chrooted environment (ideally read-only chroots)
      can also be used as slave.

========================================
For single-node bulk build:

3s) Make sure PATH environment variable includes
    /usr/pkg_distbb/bin and /usr/pkg/bin

4s) /usr/pkg_distbb/bin/distbb -h
    /usr/pkg_distbb/bin/distbb                 # for full bulk build
    /usr/pkg_distbb/bin/distbb -f packages.txt # for partial bulk build

  NOTE:
    - /usr/pkg (and PKGDBDIR) will be removed and recreated from
      bootstrap tarball.

===========================================================================

Notes.

A)

There are the following stages of the bulk build (BUILD_STAGES):
   init        - initialization, including running
                 pkg_update_src_summary and pkg_update_summary
   build       - actual package building
   gen_report  - report generation
   update_best - update information about "virtual best" bulk build
                 ($REPORTS_DIR/.best_bb)
   summary     - generating pkg_summary(5)
   checksum    - generating SHA512.txt
   upload_logs - uploading bulk build logs
   upload_pkgs - uploading binary packages
   report      - sending email or other type of notification

They are run in order.

All stages except 'init' are rerunable.  They can be rerun like this

    /usr/pkg_distbb/libexec/distbb/stage_upload_logs <BUILD_ID>

BUILD_ID is printed at the very beginning of bulk build process

B) Full list of configuration variables is available
   in share/distbb/distbb.default.conf

C) ?

===========================================================================
Aleksey Cheusov <vle@gmx.net>
