PkgForge Admin - Filesystem Configuration

One of the most important aspects of the design of the Package Forge system is the filesystem used to store all the data associated with build jobs at the various stages in their lifecycle. Basically this breaks down into three sections:

incoming
This is the directory into which new jobs are submitted. All users of the system must be able to create new sub-directories within this directory. The incoming queue processor must be able to read this directory and also be able to delete anything within the directory.
accepted
This is the directory into which valid jobs are transferred once they have been accepted by the incoming queue processor. Therefore the incoming queue processor must be able to create sub-directories within this directory. The build daemons and web interface must be able to read all the contents of this directory, the users may wish to also be able to read the contents but it is not essential. For sanity all users should be prevented from creating or deleting files or directories within this directory (that includes the build daemons and web interface).
results
This is the directory into which the results of build jobs (e.g. packages and log files) are stored. The build daemons must be able to create sub-directories within this directory. The web interface must have read access for this directory, the users may wish to also be able to read the contents but it is not essential. For sanity all users should be prevented from creating or deleting files or directories within this directory (that includes the incoming queue processor and web interface).

These requirements can be converted into a number of different methods of running the system. It could be that each sub-system (incoming processor, build daemons and web interface) runs as a different Unix user with permissions being managed by the traditional Unix model. In the School of Informatics we use OpenAFS so the ACLs were done using that system and they are described below.

Some form of networked filesystem is required for the build daemons to be able to access the input files in the accepted jobs directory and store back files into the results directory. NFS should work but has not been tested.

Using AFS

When using AFS it probably makes most sense to have all the pkgforge directories within a single AFS volume. This makes it easy to move around and make backups. A volume can be created like this:

% vos create -server host1.example.org -partition /vicepd -name pkgforge -maxquota 100000000
% fs mkmount -dir /afs/example.org/pkgs/pkgforge -vol pkgforge -rw

This would be done as an AFS super-user. Note that the maximum quota has been deliberately set to something fairly large, the default limit is fairly small and will be reached quite quickly.

Once this has been done the sub-directories can be created:

% mkdir -p /afs/example.org/pkgs/pkgforge
% mkdir -p /afs/example.org/pkgs/pkgforge/incoming
% mkdir -p /afs/example.org/pkgs/pkgforge/accepted
% mkdir -p /afs/example.org/pkgs/pkgforge/results

Users and Groups

The different sub-systems each run as the same Unix user but as a different AFS user. We currently have the following AFS users:

Rather than managing the ACLs on the various directories by adding and deleting individual users, which is likely to result in errors and oversights, AFS groups are used. This means that the ACLs on the directories should never need to be altered. There are normally three groups:

The purpose of each group should be fairly obvious. Normally the pkgforge_incoming and pkgforge_web AFS groups would each only contain a single AFS user but there are no limits. This is useful when moving services from one host to another and they both need access during a transition period.

Full details on managing AFS users and groups can be found elsewhere but here is an example of the basics:

% pts creategroup pkgforge_builder

% pts createuser -name pkgforge_builder.host1.example.org -id 28251
% pts adduser -user pkgforge_builder.host1.example.org -group pkgforge_builder

If you do not already have an appropriate AFS group you may wish to create another AFS group (e.g. pkgforge_users) which would include all users of the pkgforge system.

Note that the createuser commands must be done as the AFS super-user but all the group management can be done as a normal AFS user.

ACLs

The previous description of requirements is translated into a set of AFS ACLs which can now be applied to the previously created directories with the various groups. Note that to gain access (for either read or write) all the users must be able to at least list the higher-level directories in the tree. Note also that to ensure only the required ACLs are set on each directory the -clear option is being used throughout.

% fs setacl -clear -dir /afs/example.org/pkgs/pkgforge -acl\
                 system:administrators rlidwka\
                 pkgforge_incoming     read\
                 pkgforge_builder      read\
                 pkgforge_web          read\
                 pkgforge_users        read\
                 system:authuser       read

% fs setacl -clear -dir /afs/example.org/pkgs/pkgforge/incoming -acl\
                 system:administrators rlidwka\
                 pkgforge_incoming     write\
                 pkgforge_users        rli\
                 system:authuser       read

% fs setacl -clear -dir /afs/example.org/pkgs/pkgforge/accepted -acl\
                 system:administrators rlidwka\
                 pkgforge_incoming     write\
                 pkgforge_builder      read\
                 pkgforge_web          read\
                 pkgforge_users        read\
                 system:authuser       read

fs setacl -clear -dir /afs/example.org/pkgs/pkgforge/results -acl\
                 system:administrators rlidwka\
                 pkgforge_incoming     write\
                 pkgforge_builder      write\
                 pkgforge_web          read\
                 pkgforge_users        read\
                 system:authuser       read

It is assumed that the /afs/example.org/pkgs directory already existed and was accessible by the users.

Most of the ACLs are fairly self-explanatory but note that for the incoming directory the users only have read, lookup and insert rights, they do not have any ability to delete entries or write into files they do not own (normal write access translates into rlidwk) see the DROPBOXES section of the fs_setacl(1) manual page for more details.

Note also that we have added permissions for all authenticated users to be able to read the contents. Depending on what you are building on the pkgforge system you may wish to remove that access.