The Package Forge Registry Administrators Guide

The PkgForge Registry requires a PostgreSQL database. It is not currently possible to use any other database as a number of triggers and rules rely on the plpgsql language. This document assumes that you already have a database running and have administrator rights to create new users and databases (normally this is done as the postgres user).

If LCFG is being used to manage the system then the best method for managing the database is to use the LCFG postgresql component. This can be used to configure the ACLs and run backups.

Installing the Software

The PkgForge and PkgForge-Registry packages must be installed. These are provided as RPMs but it is also possible to build and install them from the .tar.gz format in the same way as any other Perl module.

Perl version 5.8.8 or newer is required. The PkgForge-Server package requires the following modules.

If you are using LCFG this can all be installed by including the lcfg/options/pkgforge-registry.h header.

Initialisation

Note that steps 1 and 2 can be done using the registry-init.sh script provided in the PkgForge-Registry package. They are detailed here so that a full explanation can be provided.

Step 1: Create the roles

As the database administrator issue the following commands to create the database roles:

createuser --no-createrole --no-createdb --no-superuser pkgforge_admin
createuser --no-createrole --no-createdb --no-superuser pkgforge_incoming
createuser --no-createrole --no-createdb --no-superuser pkgforge_builder
createuser --no-createrole --no-createdb --no-superuser pkgforge_web

Each distinct part of the service runs as a different database user to make it possible to restrict write access to only the essential tables. This is to prevent deliberate or accidental corruption of the data. The admin user owns the database and has complete access. In normal operation it is rare that admin access is required for modifying the data.

Step 2: Create the database

As the database administrator issue the following commands to create the database and add the necessary language support:

createdb --owner pkgforge_admin pkgforge
createlang plpgsql pkgforge

Step 3: Create the tables and rules

The final step is to create the tables, functions and rules for the database. This is done using the registry-setup.sql script which is provided as part of the PkgForge-Registry package. The SQL can be run using the psql command-line tool, like this:

psql --dbname pkgforge --host pkgforge --username pkgforge_admin --file registry-setup.sql

This assumes that the database server is running on a machine named pkgforge and that you have already configured it appropriately to allow access as the pkgforge_admin database user. If you have not done this then details are provided later in this document. The setup script can instead be run as the postgres database administrator user.

Backups

It is essential to take regular backups of the PkgForge Registry to provide security against loss of data because of a crash. The best way to do this is with pg_dump as part of a nightly cron job.

If the LCFG postgresql component is being used this cron job can be added using the LCFG cron component. For example:

!cron.objects                  mADD(pkgforgedb)
cron.object_pkgforgedb         postgresql
cron.method_pkgforgedb         run
cron.args_pkgforgedb           backupdb pkgforge
cron.run_pkgforgedb            00 22 * * *

That creates a cron job which will run at 22:00 each night and do a dump of the pkgforge database using the backupdb method of the LCFG postgresql component.

Alternatively this can be done by creating a cron job which calls pg_dump directly, for example:

pg_dump --create pkgforge > pkgforge.$(date '+%Y%m%d').dump.sql

This will dump the pkgforge database as plain-text into a file which the date as part its name. The command to create the database will be prepended to the SQL file.

In the event of a problem occurring, the storing of the data from each dump separately is incredibly useful. It makes it possible to walk back in time through the dumps to search for where corruption was first introduced.

With the dumps done like this it is very easy to restore the database. Firstly, do the initialisation step 1, "Create the Roles", which is explained above. After the roles are created the following command (as the database administrator) is all that is necessary.

psql -f pkgforge.20110501.dump.sql

This is also the best approach for moving the pkgforge database to a new host. Just take an additional dump then stop the database on the old server. To bring up the database on the new server do the restore process described above.

Access Control

Access control to PostgreSQL databases is managed through the pg_hba.conf and pg_ident.conf configuration files. On Redhat/Fedora systems these files are usually found in the /var/lib/pgsql/data/ directory along with all the other configuration files for the server. You can usually access this most easily as ~postgres/data/

PostgreSQL supports numerous different authentication mechanisms, for details see the PostgreSQL documentation. PkgForge should support any authentication mechanism which is supported by the Perl DBI module. Only password and kerberos based authentication has been tested though.

The design of the PkgForge system is that each sub-system (i.e. incoming queue processor, build daemon and web interface) have different database roles. This makes it possible to limit the write-access to just the essential set of tables for each role. This improves data security and reduces the risk of accidental or deliberate corruption. Going beyond that it is sensible to also restrict access to the database by IP address and to use SSL for the connections.

SSL Support

If you want to support SSL connections to the database server you need to add an X509 certificate and activate the postgresql ssl option in the postgresql.conf file. Using the LCFG components it can be done like this:

#include <lcfg/options/postgresql-server.h>

!postgresql.options	mADD(ssl)
!postgresql.option_ssl	mSET(on)

#include <lcfg/options/x509-client.h>

!x509.keys             mADD(pgssl)
x509.service_pgssl     <%profile.node%>.<%profile.domain%>
x509.keyfile_pgssl     <%postgresql.pgauthdir%>/server.key
x509.certfile_pgssl    <%postgresql.pgauthdir%>/server.crt
x509.chainfile_pgssl   <%postgresql.pgauthdir%>/root.crt
x509.uid_pgssl         <%postgresql.pgowner%>
x509.gid_pgssl         <%postgresql.pggroup%>
x509.component_pgssl   postgresql

If you are not using the LCFG x509 component you will need to create the SSL certificate manually.

Kerberos Support

If you want support for kerberos authentication in your database you will need a keytab and you will need to set the krb_server_keyfile configuration option in the postgresql.conf file. With LCFG you can do this using the kerberos component:

#include <lcfg/options/kerberos-client.h>

!kerberos.keys                  mADD(postgres)
kerberos.keytab_postgres	<%postgresql.pgauthdir%>/postgres.keytab
kerberos.keytabuid_postgres	<%postgresql.pgowner%>
kerberos.keytabgid_postgres   	<%postgresql.pggroup%>

!postgresql.options                     mADD(krb_server_keyfile)
postgresql.option_krb_server_keyfile	'<%kerberos.keytab_postgres%>'

If you are not using the LCFG x509 component you will need to create the Kerberos keytab file manually. Note also that you would need to ensure that the UID and GID for the file are set correctly.

Example

On an LCFG-managed host the ACLs are handled by the LCFG postgresql component. Here is an example which uses kerberos authentication:

#include <lcfg/options/postgresql-server.h>

!postgresql.ident               mADD(pkgf1)
postgresql.idmap_pkgf1          pkgforge
postgresql.sysname_pkgf1        example1
postgresql.pgname_pkgf1         pkgforge_admin

!postgresql.hostssl             mADD(pkgf1)
postgresql.ssldatabase_pkgf1    pkgforge
postgresql.ssluser_pkgf1        pkgforge_admin
!postgresql.ssladdress_pkgf1    mHOSTIP(host1.example.org)
postgresql.sslnetmask_pkgf1     255.255.255.255
postgresql.sslauthtype_pkgf1    krb5
!postgresql.sslauthopt_pkgf1    mSET(map=<%postgresql.idmap_pkgf1%>)

First, this creates a user mapping from the example1 kerberos principal to the pkgforge_admin database role. That mapping appears in the pg_ident.conf file. The same mapping can be applied from any username to database role.

Secondly, this adds an entry into the pg_hba.conf configuration file to allow kerberised aaccess for that database role from the host1.example.org host address.

With this ACL in place the example1 user can access the pkgforge database as the pkgforge_admin database role from the host1.example.org host using kerberos authentication like this:

% psql --dbname pkgforge --host pkgforge --username pkgforge_admin

This example LCFG configuration creates the following entry in the pg_ident.conf configuration file:

pkgforge    example1   pkgforge_admin

The first entry is the map name, the second is the system username and the third part is the database role.

This example LCFG configuration creates an entry like this in the pg_hba.conf configuration file:

hostssl pkgforge pkgforge_admin 192.168.1.175 255.255.255.255 krb5 map=pkgforge

If you are not using the LCFG postgresql component you will need to edit these files manually. Once the configuration is done you will then also need to reload the database using the init script (usually /etc/init.d/postgresql).