bugThe GNU Hurd - Bugs: bug #28779, libstore/ext2fs issue with sparse...

 
 

bug #28779: libstore/ext2fs issue with sparse store

Submitter:  Samuel Thibault <sthibaul>
Submitted:  Sat 30 Jan 2010 10:47:23 PM UTC
   
 
Category:  Hurd Servers Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  Fixed Privacy:  Public
Assigned to:  None Originator Name: 
Open/Closed:  Closed Reproducibility:  None
Size (loc):  None Planned Release:  None
Effort:  0.00
Wiki-like text discussion box: 


* Mandatory Fields

Add a New Comment Rich Markup
   

Sun 07 Feb 2010 10:48:15 PM UTC, comment #4: 

Applied, thanks!

Samuel Thibault <sthibaul>
Group administrator
Sat 06 Feb 2010 07:11:57 PM UTC, comment #3: 

I have attached a patch that makes diskfs_S_file_get_storage_info in
ext2fs return EOPNOTSUPP if the file is sparse.  It also removes any
code used for hole mapping, which is now dead.

It works for me (TM).


(file #19634)

Carl Fredrik Hammar <hammy>
Group Member
Sat 06 Feb 2010 02:58:20 PM UTC, comment #2: 


> So it is clear that ext2fs (or possibly libstore)
> has messed up encoding the blocks of the file, but I
> can't really tell what's wrong ATM. Possibly this
> could be a general bug in how ext2fs manages holes
> in files, so it should be further investigated.


It turns out the reason was pretty simple.  When iterating
through the blocks of a file, diskfs_S_file_get_storage_info
counts down the number of allocated blocks, and not the
number of potential blocks.

This also explains Samuel's original observation that the
disk size matches the actual size of the file and not the
potential size.

This won't be a problem if we error out if the file contains
holes, which I'll be working on next.

Carl Fredrik Hammar <hammy>
Group Member
Fri 05 Feb 2010 10:28:00 PM UTC, comment #1: 

The situation here is that the ext2fs instance
that contains blip returns a store to the new
ext2fs instance that is a copy of its own store
but with the blocks that contain blip mapped out.
The problem is that ext2fs maps them out wrongly.

First of all, this can be avoided altogether if the
new ext2fs instance is told to use blip through the
file interface, e.g. ext2fs -T file blip.

Now as to the problem, this is what the store ends
up looking like with the commands in the original
submission:

  *store =
    {source = 42, runs = 0x805e3c0, num_runs = 13, end = 8672,
     wrap_src = 8672, wrap_dst = 0, name = 0x805e348 "hd0s1",
     port = 43, block_size = 512, blocks = 6592, size = 4440064,
     log2_block_size = 9, log2_blocks_per_page = 3, flags = 0,
     misc = 0x0, misc_len = 0, class = 0x10782a0,
     children = 0x0, num_children = 0, hook = 0x0}

And the mapping looks like this (unit is a 512
byte block):

  store->runs[0] = {start = 2417496, length = 8}
  store->runs[1] = {start = 2424048, length = 8}
  store->runs[2] = {start = 2424016, length = 24}
  store->runs[3] = {start = 2424072, length = 24}
  store->runs[4] = {start = 2424064, length = 8}
  store->runs[5] = {start = 2424056, length = 8}
  store->runs[6] = {start = 2417504, length = 16}
  store->runs[7] = {start = 2417528, length = 6448}
  store->runs[8] = {start = 2423984, length = 8}
  store->runs[9] = {start = 2423976, length = 8}
  store->runs[10] = {start = 2423992, length = 24}
  store->runs[11] = {start = 2424040, length = 8}
  store->runs[12] = {start = -1, length = 2080}

The last mapping that starts at -1 is a hole, and
it is immediately clear that it is way too small.
There should even be a megabyte worth of non-hole
blocks at the end of the store where dd actually
wrote to blip.

So it is clear that ext2fs (or possibly libstore)
has messed up encoding the blocks of the file, but I
can't really tell what's wrong ATM.  Possibly this
could be a general bug in how ext2fs manages holes
in files, so it should be further investigated.

However, file holes cannot be encoded as store holes
at all!  Reading and writing to a store hole will
result in EIO, which isn't what file holes are for,
rather you just want delayed allocation of file
blocks.  (I'm not really sure what store holes are
good for.)

In fact, it is impossible to map out the blocks of
a file hole without actually allocating the blocks,
which defeats the purpose of the hole.  Instead, I
think the right solution is to simply have ext2fs's
diskfs_S_file_get_storage_info fail if the file
contains holes, so that the client will use it
through the filesystem interface.

Carl Fredrik Hammar <hammy>
Group Member
Sat 30 Jan 2010 10:47:23 PM UTC, original submission:  

$ dd < /dev/zero > blip bs=1M count=1 seek=100
$ mke2fs blip
$ settrans -c foo /hurd/ext2fs blip
ext2fs: blip: panic: get_hypermetadata: disk size (7671808 bytes) too small; superblock says we need 105906176

Apparently libdiskfs gives the actual size to ext2fs, not the potential
size...

Samuel Thibault <sthibaul>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by hammy (Posted a comment)
  • -email is unavailable- added by sthibaul (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-02-07 sthibaul StatusNone Fixed
        Open/ClosedOpen Closed
    2010-02-06 hammy Attached File- Added 0001-Don-t-return-store-for-files-that-contains-holes.patch, #19634

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code