PkgForge Admin - Package Builder

Before any package builders can be configured the following steps must be taken:

  1. Configure the PkgForge filesystem
  2. Configure client access for the PkgForge Registry database

Installing the Software

The PkgForge, PkgForge-Registry and PkgForge-Server packages must be installed. These are provided as RPMs but it is also possible to build and install them from the .tar.gz format in the same way as any other Perl module.

Perl version 5.8.8 or newer is required. The PkgForge-Server package requires the following modules.

If you wish to build from SRPMs (currently this is the only supported package format) you will also need the Perl RPM2 module installed.

If you are using LCFG this can all be installed by including the lcfg/options/pkgforge-builder.h header.

Configuration

As with the software installation, if you are using LCFG this can all be configured correctly by using the lcfg/options/pkgforge-builder.h header.

The basic layout of the filesystem (as described in the separate filesystem information) should be configured using the /etc/pkgforge/pkgforge.yml YAML file. It should look something like this:

--- 
"accepted": '/afs/example.org/pkgs/pkgforge/accepted/'
"incoming": '/afs/example.org/pkgs/pkgforge/incoming/'
"results": '/afs/example.org/pkgs/pkgforge/results/'

If you are using the LCFG pkgforge component then this is done using the pkgforge.incoming, pkgforge.accepted and pkgforge.results resources.

If AFS is being used to provide the filesystem for the pkgforge service then you will need a Kerberos principal and an associated AFS user. As with Kerberised access to the Registry database the Kerberos keytab can be managed using the LCFG kerberos component. If you already have this setup for DB access you do not need a second keytab for AFS access. The following configuration will create a keytab:

!pkgforge.keytab_buildd   mSET(<%pkgforge.configdir%>/pkgforge_builder.keytab)

!kerberos.keys                       mADD(pkgforge_builder)
kerberos.keytab_pkgforge_builder    <%pkgforge.keytab_builder%>
kerberos.keytabuid_pkgforge_builder <%pkgforge.user%>
kerberos.keytabgid_pkgforge_builder <%pkgforge.group%>

Alternatively the principal and keytab can be created manually using the kadmin tool. You will need to ensure that the UID and GID and the permissions are correct for the keytab or k5start will not work.

A package builder is configured using the /etc/pkgforge/buildd.yml YAML file and also a platform-specific YAML file (e.g. /etc/pkgforge/buildd-bressler_sl6.yml). The registry database access configuration section has already been discussed in the client configuration page. The general buildd configuration should look something like:

--- 
"init": '/etc/init.d/pkgforge-buildd'
"keytab": '/etc/pkgforge/pkgforge_builder.keytab'
"registry": 
  "host": 'pkgforge'
  "name": 'pkgforge'
  "user": 'pkgforge_builder'
"start": 'yes'

The init, keytab and start options are not used by the daemon, they are purely instructions for the LCFG pkgforge component.

The important file is /etc/sysconfig/pkgforge-buildd which controls how the /etc/init.d/pkgforge-buildd init-script starts the daemon. If the LCFG pkgforge component is being used then the sysconfig file will be configured automatically. The sysconfig file should look something like:

NAMES="bressler_sl6 bressler_sl664"
RUNAS="pkgforge"
KRB5_KEYTAB="/etc/pkgforge/pkgforge_builder.keytab"

The following options can be modified via the sysconfig file but mostly they have sensible defaults.

A host can have multiple build daemons running (more than one per processor/core is a bad idea), the names of all the build daemons must be specified and the init-script will start them all in turn.

Note that, by default, some files are stored in the /var/lib/pkgforge/ and /var/run/pkgforge/ directories. Those directories must be writable by whichever unix user is used to run the daemon.

Builder-specific Options

As well as the general configuration which applies to all build daemons on a host each daemon has its own specific configuration, this is stored in a file which references the name of the build daemon (like /etc/pkgforge/buildd-bressler_sl6.yml). The configuration will look something like this:

--- 
"builder": 'RPM'
"checks": 
  - 'RPMLint'
"name": 'bressler_sl6'
"reports": 
  - 'Email'
"submitters": 
  - 'PkgSubmit'
"timeout": '3600'

The build process works in phases and the command(s) issued during each phase are carried out using modules found in the PkgForge::BuildCommand namespace. The phases are configured per-build daemon and are run in the following order:

  1. builder - There must be a single builder command (e.g. PkgForge::BuildCommand::Builder::RPM.
  2. checks - There may be zero or more check commands (e.g. PkgForge::BuildCommand::Check::RPMLint).
  3. submitters - There may be zero or more package submitters (e.g. PkgForge::BuildCommand::Submitter::PkgSubmit).
  4. reports - There may be zero or more reporters which are run at the end of the entire process to report success or fail (e.g. PkgForge::BuildCommand::Reporter::Email).

A timeout can be set for the build process, in the case of the RPM builder this is passed to the rpmbuild command and limits the maximum time the building of a single package may take.

If the LCFG pkgforge component is being used then this will be configured automatically by the lcfg/options/pkgforge-builder.h header. It can be specified like this:

!pkgforge.builders          mADD(f13)
pkgforge.type_f13           RPM
pkgforge.name_f13           <%%sysinfo.node%%>_f13
pkgforge.checks_f13         RPMLint
pkgforge.submitters_f13     PkgSubmit
pkgforge.reports_f13        Email

By default a builder will be added which matches the host platform and architecture combination (e.g. f13/i386). If the architecture of the host is x86_64 then it will also get a default builder for the same platform but for the i386 architecture. It is typically possible to build for earlier platforms using hosts based on newer (e.g. building for SL5 on SL6) but not the other way around.

Temporary Files

At various points the daemon generates temporary files. For security each build daemon uses a specific temporary directory (which is a sub-directory of /var/tmp/pkgforge by default, e.g. /var/tmp/pkgforge/bressler_sl6). This directory must be writable by whichever unix user is used to run the daemon. The location can be changed by setting the tmpdir attribute in the /etc/pkgforge/buildd.yml YAML file (or the builder-specific configuration file), that is not currently modifiable using the LCFG pkgforge component.

Logging

By default the build daemon log is kept in the /var/log/pkgforge/buildd.log file. You need to ensure that the log directory is writable by whichever unix user is used to run the daemon. Logs for each individual build are also stored in a daemon-specific directory (e.g. /var/log/pkgforge/bressler_sl6). Once a build has completed the log is copied into the relevant results directory so you may wish to regularly cull this directory using tmpwatch or similar.

The logs from the build daemon are configured via the /etc/pkgforge/log-buildd.cfg configuration file. For details on how to modify the style see the documentation for the Log::Dispatch and MooseX::LogDispatch Perl modules.

Starting/Stopping the Daemon

Once the configuration files are in place the build daemons can be started using the init-script provided in the package. This is normally /etc/init.d/pkgforge-buildd.

Currently the init-script assumes that Kerberos authentication is required and it will attempt to start the k5start daemon first. Patches welcome to remove this limitation.

If the LCFG pkgforge component is being used then the daemon should be managed using om and the init-script should not be called directly, for example:

% om pkgforge start
% om pkgforge stop

For the component to actually start the daemon the pkgforge.start_buildd resource must be set to yes.

If any configuration or code changes are made then the daemon will need to be completely restarted. This is done with the restart method.

Without the LCFG pkgforge component the init-script should be used directly. The usual start, stop and restart methods are available.