bugThe GNU Hurd - Bugs: bug #28779, libstore/ext2fs issue with sparse...

 
 

bug #28779: libstore/ext2fs issue with sparse store

Submitted by:  Samuel Thibault <sthibaul>
Submitted on:  Sat 30 Jan 2010 10:47:23 PM UTC  
 
Category: Hurd ServersSeverity: 3 - Normal
Priority: 5 - NormalItem Group: None
Status: FixedPrivacy: Public
Assigned to: NoneOriginator Name: 
Open/Closed: ClosedReproducibility: None
Size (loc): NonePlanned Release: None
Effort: 0.00
Wiki-like text discussion box: 

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sun 07 Feb 2010 10:48:15 PM UTC, comment #4:

Applied, thanks!

Samuel Thibault <sthibaul>
Project Member
Sat 06 Feb 2010 07:11:57 PM UTC, comment #3:

I have attached a patch that makes diskfs_S_file_get_storage_info in
ext2fs return EOPNOTSUPP if the file is sparse. It also removes any
code used for hole mapping, which is now dead.

It works for me (TM).

(file #19634)

Carl Fredrik Hammar <hammy>
Project Member
Sat 06 Feb 2010 02:58:20 PM UTC, comment #2:

> So it is clear that ext2fs (or possibly libstore)
> has messed up encoding the blocks of the file, but I
> can't really tell what's wrong ATM. Possibly this
> could be a general bug in how ext2fs manages holes
> in files, so it should be further investigated.


It turns out the reason was pretty simple. When iterating
through the blocks of a file, diskfs_S_file_get_storage_info
counts down the number of allocated blocks, and not the
number of potential blocks.

This also explains Samuel's original observation that the
disk size matches the actual size of the file and not the
potential size.

This won't be a problem if we error out if the file contains
holes, which I'll be working on next.

Carl Fredrik Hammar <hammy>
Project Member
Fri 05 Feb 2010 10:28:00 PM UTC, comment #1:

The situation here is that the ext2fs instance
that contains blip returns a store to the new
ext2fs instance that is a copy of its own store
but with the blocks that contain blip mapped out.
The problem is that ext2fs maps them out wrongly.

First of all, this can be avoided altogether if the
new ext2fs instance is told to use blip through the
file interface, e.g. ext2fs -T file blip.

Now as to the problem, this is what the store ends
up looking like with the commands in the original
submission:

*store =
{source = 42, runs = 0x805e3c0, num_runs = 13, end = 8672,
wrap_src = 8672, wrap_dst = 0, name = 0x805e348 "hd0s1",
port = 43, block_size = 512, blocks = 6592, size = 4440064,
log2_block_size = 9, log2_blocks_per_page = 3, flags = 0,
misc = 0x0, misc_len = 0, class = 0x10782a0,
children = 0x0, num_children = 0, hook = 0x0}

And the mapping looks like this (unit is a 512
byte block):

store->runs[0] = {start = 2417496, length = 8}
store->runs[1] = {start = 2424048, length = 8}
store->runs[2] = {start = 2424016, length = 24}
store->runs[3] = {start = 2424072, length = 24}
store->runs[4] = {start = 2424064, length = 8}
store->runs[5] = {start = 2424056, length = 8}
store->runs[6] = {start = 2417504, length = 16}
store->runs[7] = {start = 2417528, length = 6448}
store->runs[8] = {start = 2423984, length = 8}
store->runs[9] = {start = 2423976, length = 8}
store->runs[10] = {start = 2423992, length = 24}
store->runs[11] = {start = 2424040, length = 8}
store->runs[12] = {start = -1, length = 2080}

The last mapping that starts at -1 is a hole, and
it is immediately clear that it is way too small.
There should even be a megabyte worth of non-hole
blocks at the end of the store where dd actually
wrote to blip.

So it is clear that ext2fs (or possibly libstore)
has messed up encoding the blocks of the file, but I
can't really tell what's wrong ATM. Possibly this
could be a general bug in how ext2fs manages holes
in files, so it should be further investigated.

However, file holes cannot be encoded as store holes
at all! Reading and writing to a store hole will
result in EIO, which isn't what file holes are for,
rather you just want delayed allocation of file
blocks. (I'm not really sure what store holes are
good for.)

In fact, it is impossible to map out the blocks of
a file hole without actually allocating the blocks,
which defeats the purpose of the hole. Instead, I
think the right solution is to simply have ext2fs's
diskfs_S_file_get_storage_info fail if the file
contains holes, so that the client will use it
through the filesystem interface.

Carl Fredrik Hammar <hammy>
Project Member
Sat 30 Jan 2010 10:47:23 PM UTC, original submission:

$ dd < /dev/zero > blip bs=1M count=1 seek=100
$ mke2fs blip
$ settrans -c foo /hurd/ext2fs blip
ext2fs: blip: panic: get_hypermetadata: disk size (7671808 bytes) too small; superblock says we need 105906176

Apparently libdiskfs gives the actual size to ext2fs, not the potential
size...

Samuel Thibault <sthibaul>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by hammy (Posted a comment)
  • -unavailable- added by sthibaul (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sun 07 Feb 2010 10:48:15 PM UTCsthibaulStatusNone=>Fixed
      Open/ClosedOpen=>Closed
    Sat 06 Feb 2010 07:11:57 PM UTChammyAttached File-=>Added 0001-Don-t-return-store-for-files-that-contains-holes.patch, #19634

    Back to the top


    Powered by Savane 3.1-cleanup1