# GNU Savannah Project List This page describes an experimental feature of publishing the list of projects hosted on GNU Savannah. ## Synopsis list of current GNU projects: curl --silent http://savannah.gnu.org/cooperation/v0/projects.tsv \ | awk -F'\t' '$2=="gnu" { print $1 }' and: curl --silent http://savannah.gnu.org/cooperation/v0/projects.json \ | jq -r '.packages[] | select(.type=="gnu") | .package' ## Data files The following files are updated every X hours. ### Tabular format URL: The file contains the following fields: 1. `package` - short package identifier (internally called `unix_group_name`). Contains lower-case English letters, digits, underscore, minus characters. Does not contain spaces. Will be unique. 2. `type` - project type. one of: 1. `gnu` - official GNU package. 2. `nongnu` - non-gnu project hosted on GNU Savannah. 3. `web-portions` - projects relating to content management on www.gnu.org. 4. `gug` - meta-project for user groups. 5. `web-translation` - translations for www.gnu.org. 3. `name` - package name. Free text, will contain spaces. Tabs/Newlines will be escaped with a backslash. Can be an empty string. Might contain UTF-8 characters, characters in other encoding, and even invalid character combination (e.g. invalid/partial UTF-8 sequences). 4. `description` - package description. See `name` for possible content. 5. `registration_date` - Project registration date. Format: `YYYY-MM-DD HH:MM:SS`. Note: several GNU packages predate GNU Savannah by many years. These project will have registration time circa 2000/2001. 6. `registration_epoch` - Project registration timestamp in seconds since epoch. 7. `url_savannah` - URL of project's page on Savannah. 8. `url_homepage` - URL of project's homepage. Can be empty or even invalid. 9. `url_download` - URL of project's download area (e.g. release tarballs for GNU packages). Can be empty or even invalid URL. ### JSON format URL: Format: The JSON file contains one key `packages:[]`, containing the list of packages. Each package is an object (e.g. associative array) with the following structure: { "packages": [ { "package": "coreutils", "type": "gnu", "name": "GNU Core Utilities", "description": "The GNU Core Utilities are the basic file, shell and text manipulation utilities of the GNU Operating System.", "registration_date": "2002-08-04 06:58:58", "registration_epoch": 1028458738, "url_savannah": "https://savannah.gnu.org/projects/coreutils", "url_homepage": "http://www.gnu.org/software/coreutils", "url_download": "http://ftp.gnu.org/gnu/coreutils/" }, { }, ... ] } ## Usage examples NOTES: * Each example downloads the list of projects, and prints the ten most recent GNU packages. * Your output may vary since new GNU packages are routinely being added. * Error checking omitted for brevity. * When processing the list, never assume input is valid - always check, validate and sanitize every field. Some of the fields can be set directly by Savannah users. ### tabular file The following command will print the names of ten most recent GNU packages $ curl --silent http://savannah.gnu.org/cooperation/v0/projects.tsv \ | awk -F'\t' '$2=="gnu"' \ | LC_ALL=C sort -t $'\t' -k6nr,6 \ | head -n10 \ | cut -f1,3,5 \ | awk -F'\t' '{print $3, "-", $2, "(" $1 ")" }' 2014-08-18 01:12:30 - GNU direvent (direvent) 2014-07-05 18:21:58 - GNU Datamash (datamash) 2014-04-11 11:37:50 - GNU libdbh (libdbh) 2014-01-11 15:54:25 - Libre Library of Graphic Elements (llge) 2014-02-12 20:01:04 - cursynth (cursynth) 2014-02-02 23:18:53 - GNU Multi-Precision Rational Interval Arithmetic Library (mpria) 2013-09-24 07:22:02 - GNU APL (apl) 2013-09-24 05:35:02 - GNU dmd (dmd) 2013-10-24 11:02:10 - Faust (fst) 2013-10-01 01:18:45 - GNU Cobol (gnucobol) ### JSON - Shell The program `jq` () is used for JSON processing. #!/bin/sh # DEBUG: print structure of one package # cat JSON-FILE | jq -C '.packages[0]' curl --silent http://savannah.gnu.org/cooperation/v0/projects.json \ | jq --raw-output '.packages | sort_by(.registration_epoch) | reverse | .[] | select(.type=="gnu") | [ .registration_date, " - ", .name, " (", .package, ")" ] | join("")' \ | head -n 10 ### JSON - Perl The Perl module `JSON` () is used to process JSON data. #!/usr/bin/env perl use strict; use warnings; use JSON; use LWP::Simple; use Data::Dumper; my $url = 'http://savannah.gnu.org/cooperation/v0/projects.json'; my $text = get $url or die; my $data = from_json($text); my @packages = @{$data->{packages}}; my @gnu_packages = grep { $_->{type} eq 'gnu' } @packages; # DEBUG: show structure of one package # print Dumper(\$gnu_packages[0]),"\n"; @gnu_packages = sort { $b->{registration_epoch} <=> $a->{registration_epoch} } @gnu_packages; @gnu_packages = @gnu_packages[0..9]; foreach my $pkg (@gnu_packages) { print $pkg->{registration_date}, " - ", $pkg->{name}, " (",$pkg->{package}, ")\n"; } ### JSON - Python #!/usr/bin/env python import json from operator import itemgetter from pprint import pprint from urllib2 import urlopen url = 'http://savannah.gnu.org/cooperation/v0/projects.json' text = urlopen(url) data = json.load(text) packages = data["packages"] gnu_packages = [ x for x in packages if x["type"]=="gnu"]; # DEBUG: show structure of one package # pprint(gnu_packages) gnu_packages = sorted(gnu_packages, key=itemgetter('registration_epoch'), reverse=True) gnu_packages = gnu_packages[0:10] for pkg in gnu_packages: print "%s - %s (%s)" % ( pkg["registration_date"], pkg["name"], pkg["package"] ) ### JSON - Javascript/NodeJS #!/usr/bin/env node var http = require('http'); var url = "http://savannah.gnu.org/cooperation/v0/projects.json"; http.get(url, function(response){ var body = ''; response.on('data', function(d) { body += d; }); response.on('end', function() { data = JSON.parse(body); packages = data.packages; var gnu_packages = packages.filter(function(pkg) { return pkg.type=="gnu"; }); //DEBUG: show structure of one package: //console.log(gnu_packages[0]); gnu_packages.sort(function(a,b){ return (b.registration_epoch - a.registration_epoch); }); gnu_packages = gnu_packages.slice(0,10); gnu_packages.forEach(function(pkg){ console.log(pkg.registration_date,"-",pkg.name,"(",pkg.package,")"); }); }); });