manDynzip - Cookbook: recipe #255, Seeking into a zipped file

 
 

You are not allowed to post comments on this tracker with your current authentication level.

recipe #255: Seeking into a zipped file

Submitter:  Ole Tange <tange>
Submitted:  Tue 27 Sep 2011 02:08:06 PM UTC
   
 
Category:  None Importance:  3 - Normal
Status:  Draft Privacy:  Public
Assigned to:  None Open/Closed:  Open

Tue 27 Sep 2011 02:08:06 PM UTC, recipe preview:  

The file format should support seeking into the file and decompressing from that location.

The compressed file is already in blocks, so there will be no need to decompress everything.

To know the blocksize the file needs a header like:

magic-string
blocksize (64bit int)

DYNZ
10000000 (64bit int)

When each block is sent the compressed and decompressed size is known so each block could have a header like:

algorithm
absolute position of start of compressed data

lzop -1
302812 (64bit int)

To make seeking easier the headers could be included in the end of the file, too:

DYNZ
10000000 (64bit int)
lzop -1
302812 (64bit int)
lzip -9
709920 (64bit int)
Relative posistion of the start of the index. (64bit int)

This way finding a given byte can be done by:

  • Seek to end of file.
  • Read the relative position of start of index
  • Seek backwards to that position
  • Read the full index creating a table of blockno -> { compression algorithm, start position }
  • Find the block number of the byte wanted (byte position/block size)
  • Decompress that block and seek to the byte (byte position mod block size)



Ole Tange <tange>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by tange (Submitted the item)
  •  

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code