This has been half-fixed by the check-in for bug #18692.  We can do better integration later if it proves worthwhile.

>I'm saying that if you pipe through gzip (or any compression
>tool), then it produces nothing until it's input is closed.
>Although "you" (pspp) will be reading cases sequentially, the gzip
>process will have to store them all in memory until after the
>very last one.

That's not at all true.  You can demonstrate this easily: run "gzip < /dev/urandom > /dev/null" in one terminal, and "top" in another.  gzip will use as much CPU as it can get, but its memory consumption is O(1).

>It'll mean for instance that reading/writing will be considerably
>slower,

Possibly.  I bet it wouldn't be any slower on any mid-range to high-end computer you buy off-the-shelf this year, though, because all of those are multi-core.

>and the maximum possible number of cases will be smaller.

I doubt it.

Not sure what you mean by "no longer serializable".  We currently read system and portable files only sequentially, and with one small exception the same is true for writing system and portable files.  I was planning to implement this feature just by piping data to/from a gzip or bzip2 process.  Can you think of why this wouldn't work?

We could easily support automatic compression and decompression for system file I/O, based on file name extensions (.gz .bz2 etc.) or on file magic numbers.  This would be a much better alternative to the "compression" that SPSS system files support.