We (GNU Savannah and the FSF) value transparency, so here are the details on this page.
There's been a SQL SELECT injection leading to a leak of unsalted MD5 account passwords, some of them discovered through online passwords recovery services, leading in turn to project membership and admin access, used for vandalism on the 'www' project that backs www.gnu.org.
www.gnu.org normally serves static pages, but PHP was found enabled, so crackers installed a reverse shell there as well.
Root access is separately maintained and no evidence of root access was found.
No other exploitable SQL injection has been found in Savane so far, thanks to the SQL parametrization work done back in 2007.
To be safe, we've reinstalled the system and restored the data from a trusted backup, November 23th circa 12:00 GMT.
- Crack analysis before re-enabling any service
- SQL injection fix and code audit before re-enabling the web front-end
- Removed all passwords (users and system) and sessions
- Use crypt's SHA-512 for passwords, and phpass's entropy code for salt
- Enforced password strength (through passwdqc)
- Added logs analysis reporting tool that keeps us informed of SQL injection attacks
- Disabled PHP entirely at www.gnu.org
- Upgraded friend website gna.org to our version of Savane
- Auditing changes between the 23th and the 27th to see what was committed (no code commits found so far)
Thanks to FSF sysadmins Bernie Innocenti and Ward Vandewege, Savannah Hackers Sylvain Beucler, Brian Gough, Michael J. Flickinger, Jim Meyering; Openwall hacker Solar Designer; Savannah users for their support.
- 2010/11/24 21:30 UTC: SQL SELECT injection attack originated from Tbilisi, Georgia, access to user encrypted passwords
- 2010/11/24 21:27 UTC: one Savannah admin password cracked, account compromised
- 2010/11/26 16:02 UTC: cracker gained membership to the www project
- 2010/11/26 23:51 UTC: cracker tested commit to the www CVS repository
- 2010/11/27 00:51 UTC: cracker defaced www.gnu.org
- 2010/11/27 01:35 UTC: cracker committed a reverse shell using unexpectedly enabled PHP support
- 2010/11/27 01:36 UTC: notification of the intrusion
- 2010/11/27 01:37 UTC: website restored
- 2010/11/27 04:42 UTC: emergency fix to Savane code (unknowing that an admin account was still compromised)
- 2010/11/27 19:05 UTC: new cracker activity on www.gnu.org - we shutdown the machines
- 2010/11/27 21:35 UTC: reinstalled www.gnu.org
- 2010/11/29 15:23 UTC: reinstalled Savannah machines to be safe
- 2010/11/29 21:30 UTC: access to the base host restored, extracting incremental backup from the 23th
- 2010/11/29 23:30 UTC: finished diagnosing original attack
- 2010/11/30 12:30 UTC: data transfers in progress
- 2010/11/30 13:30 UTC: read-only access to source repositories
- 2010/11/30 14:30 UTC: write access to source repositories
- 2010/11/30 16:30 UTC: data transfers finished
- 2010/11/30 18:00 UTC: access to downloads and GNU Arch
- 2010/11/30 21:00 UTC: audited code and found no other SQL injection
- 2010/11/30 22:30 UTC: found trace of earlier attack on Nov 23th 04:00
- 2010/11/30 22:45 UTC: stopped write access
- 2010/11/30 23:45 UTC: found trace of earlier read-only SQL injections as back as January, but none with actual account cracking
- 2010/12/01 00:55 UTC: after fishing through logs, it appears that there was no other account cracking
- 2010/12/01 11:00 UTC: restored write access
- 2010/12/02 08:02 UTC: web front-end improved and re-enabled