Sat 06 Feb 2010 02:58:20 PM UTC, comment #2:
> So it is clear that ext2fs (or possibly libstore)
> has messed up encoding the blocks of the file, but I
> can't really tell what's wrong ATM. Possibly this
> could be a general bug in how ext2fs manages holes
> in files, so it should be further investigated.
It turns out the reason was pretty simple. When iterating
through the blocks of a file, diskfs_S_file_get_storage_info
counts down the number of allocated blocks, and not the
number of potential blocks.
This also explains Samuel's original observation that the
disk size matches the actual size of the file and not the
potential size.
This won't be a problem if we error out if the file contains
holes, which I'll be working on next.
|
Fri 05 Feb 2010 10:28:00 PM UTC, comment #1:
The situation here is that the ext2fs instance
that contains blip returns a store to the new
ext2fs instance that is a copy of its own store
but with the blocks that contain blip mapped out.
The problem is that ext2fs maps them out wrongly.
First of all, this can be avoided altogether if the
new ext2fs instance is told to use blip through the
file interface, e.g. ext2fs -T file blip.
Now as to the problem, this is what the store ends
up looking like with the commands in the original
submission:
*store =
{source = 42, runs = 0x805e3c0, num_runs = 13, end = 8672,
wrap_src = 8672, wrap_dst = 0, name = 0x805e348 "hd0s1",
port = 43, block_size = 512, blocks = 6592, size = 4440064,
log2_block_size = 9, log2_blocks_per_page = 3, flags = 0,
misc = 0x0, misc_len = 0, class = 0x10782a0,
children = 0x0, num_children = 0, hook = 0x0}
And the mapping looks like this (unit is a 512
byte block):
store->runs[0] = {start = 2417496, length = 8}
store->runs[1] = {start = 2424048, length = 8}
store->runs[2] = {start = 2424016, length = 24}
store->runs[3] = {start = 2424072, length = 24}
store->runs[4] = {start = 2424064, length = 8}
store->runs[5] = {start = 2424056, length = 8}
store->runs[6] = {start = 2417504, length = 16}
store->runs[7] = {start = 2417528, length = 6448}
store->runs[8] = {start = 2423984, length = 8}
store->runs[9] = {start = 2423976, length = 8}
store->runs[10] = {start = 2423992, length = 24}
store->runs[11] = {start = 2424040, length = 8}
store->runs[12] = {start = -1, length = 2080}
The last mapping that starts at -1 is a hole, and
it is immediately clear that it is way too small.
There should even be a megabyte worth of non-hole
blocks at the end of the store where dd actually
wrote to blip.
So it is clear that ext2fs (or possibly libstore)
has messed up encoding the blocks of the file, but I
can't really tell what's wrong ATM. Possibly this
could be a general bug in how ext2fs manages holes
in files, so it should be further investigated.
However, file holes cannot be encoded as store holes
at all! Reading and writing to a store hole will
result in EIO, which isn't what file holes are for,
rather you just want delayed allocation of file
blocks. (I'm not really sure what store holes are
good for.)
In fact, it is impossible to map out the blocks of
a file hole without actually allocating the blocks,
which defeats the purpose of the hole. Instead, I
think the right solution is to simply have ext2fs's
diskfs_S_file_get_storage_info fail if the file
contains holes, so that the client will use it
through the filesystem interface.
|