Savannah BackEnd Setup

This page describes the configuration needed for Savannah new backend server and Perl code (e.g. the scripts that run on vcs0 which create new repositories for the projects based on database updates from the frontend).

General information

Long ago, Savannah used a single server. This lead to the frontend code (PHP) and backend scripts (Perl) to reside in the same repository, and rely on similar configuration files.

Despite using the same repository, the frontend code does not use the Perl backend scripts, and the backend side (e.g. vcs0) does not use the frontend PHP code. See FrontEndSetup for complimentary details about the frontend.

vcs0 needs to run sv_groups to create new repositories for new projects (or existing projects which enabled new repositories using the "Select Features" menu option on the Savannah web site). download0 needs to run sv_groups to create the 'download/releases' directory for projects. internal0 needs to run sv_mailman, sv_reminder,sv_cleaner as cronjobs. See SavannahInternals for more details about the scripts and cronjobs (although they describe the setup on the current/old VMs, we aim to replicate a similar setup on the new VMs).

The following installation has been executed on all servers: vcs0,frontend0, download0,internal0 - using the scripts should work identically, regardless of which server they are run on.

Install prerequisites:

apt-get install -y imagemagick gettext
cpan -i Mail::Send

As root, create the required directories. /etc/savane is temporarily created with write access (for the 'make install' step as non-root user), and will be made read-only after the installation is complete.

mkdir -p /opt/savannah /etc/savane
chgrp staff /opt/savannah /etc/savane
chmod g+ws /opt/savannah /etc/savane

As non-root (e.g. agn), clone the savane repository.

cd /opt/savannah
git clone https://git.savannah.gnu.org/administration/savane.git
cd savane/
./bootstrap
./configure --prefix=/opt/savannah/ --sysconfdir=/etc
make
make install

The resulting files:

$ ls -l /opt/savannah/
total 8
drwxrwsr-x 2 agn staff 4096 Oct 19 21:23 bin
drwxrwsr-x 3 agn staff 4096 Oct 19 21:09 lib
drwxrwsr-x 3 agn staff 4096 Oct 19 21:09 savane
drwxrwsr-x 3 agn staff 4096 Oct 19 21:09 share

Savannah common backend scripts:

$ ls /opt/savannah/bin
[...]
sv_groups
sv_aliases
sv_homedirs
[...]

Savannah Perl modules:

$ cd /opt/savannah/
$ find ./lib/ -type f
lib/perl/5.18.2/perllocal.pod
lib/perl/5.18.2/auto/Savane/.packlist

$ find ./share/ -type f
share/perl/5.18.2/Savane.pm
share/perl/5.18.2/Savane/Conf.pm
share/perl/5.18.2/Savane/DB.pm
share/perl/5.18.2/Savane/User.pm
share/perl/5.18.2/Savane/Groups.pm
share/perl/5.18.2/Savane/Cvs.pm
share/perl/5.18.2/Savane/Download.pm
share/perl/5.18.2/Savane/Svn.pm
share/perl/5.18.2/Savane/Hg.pm
share/perl/5.18.2/Savane/Git.pm
[...]

UGLY HACK: Symlink the Savane Perl module to the server sitelib. This alleviate the need to set custom PERL5LIB, while keep all of Savannah custom code under /opt/savannah. Improving this is a desired TODO item (by either doing away with Savane.pm or changing the Perl script to self-locate the required modules. See SavannahHackingIdeas).

ln -sf /opt/savannah/share/perl/5.18.2/Savane /usr/local/share/perl/5.18.2
ln -sf /opt/savannah/share/perl/5.18.2/Savane.pm /usr/local/share/perl/5.18.2

Manually prepare configuration files (passwords copied from frontend0). The .pl file is used by Perl scripts (e.g. sv_groups). The .ini file is used by python scripts (e.g. sv_tmpgpg). NOTE: On internal0, set the HOST to localhost instead of 208.118.235.78.

$ cd /etc/savane
$ scp frontend0:/etc/savane/savane.{pass,ini,.conf.pl} .

$ cat /etc/savane/savane.pass
XXXXXXXXX

$ cat /etc/savane/savane.ini
[database]
HOST=208.118.235.78
USER=savannahscripts
PASSWORD=XXXXXXXXXX
NAME=savane

$ cat /etc/savane/savane.conf.pl
[...]
our $sys_dbhost="208.118.235.78"; #internal0.sv.gnu.org
our $sys_dbname="savane";
our $sys_dbuser="savannahscripts";
open(IN, '<'.dirname(__FILE__).'/savane.pass');
our $sys_dbpasswd = <IN>;

On vcs0,download0,internal0 - lock permissions on the configuration files.

NOTE: DO NOT run these commands on frontend0. Frontend0 files require special permissions to ensure apache can access these files. See FrontEndSetup for detail.

cd /etc/savane/
chown root:staff savane.*
chmod 0640 savane.*

# ls -l /etc/savane/
total 16
-rw-r----- 1 root staff 8563 Oct 19 21:29 savane.conf.pl
-rw-r----- 1 root staff   82 Oct 19 22:36 savane.ini
-rw-r----- 1 root staff    9 Oct 19 21:29 savane.pass

# and lock the directory
chmod g-w /etc/savane

Test the scripts, ensuring the Perl modules and DB are working:

### This is equivalent to what OpenSSH is using to find the
### users' keys when they log with SSH to @vcs.sv.gnu.org .
### The users' keys are stored in savane database.

$ /opt/savannah/bin/sv_get_authorized_keys agn | cut -b1-40
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAvsw+
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCe
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDf
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCa

The script sv_groups writes log messages to hard-coded path, created it with 'staff' permissions:

touch /var/log/sv_database2system.log
chown root:adm /var/log/sv_database2system.log

FIXME: sv_groups also uses a lock file named groups-users.lock - not sure where it is stored.

Initial testing of 'sv_groups': As NON-ROOT user (e.g. agn), run:

$ /opt/savannah/bin/sv_groups --only-git --verbose --dry-run

This will show demonstrate how 'sv_groups' scans the groups (=projects) in the savane database, and for each one, checks if the repositories directories exist.

FIXME: Check with new project or modified repositories type (e.g. git,hg/svn).

FIXME: The log file /var/log/sv_database2system.log emits the following message, is this relevant / should we create this user on vcs0?

[sv_groups] Wed Oct 19 21:54:40 2016 ---- groupadd -g 77955 anoncvs (required by the savane backend)