103 |
================ |
================ |
104 |
|
|
105 |
In our system, Storm (for *storage module*), all data is stored |
In our system, Storm (for *storage module*), all data is stored |
106 |
in *blocks*, byte sequences identified by a |
in *blocks*, byte sequences identified by a SHA-1 cryptographic hash |
107 |
cryptographic hash. Blocks often have a similar granularity |
[ref SHA-1 and our ht'02 paper]. Blocks often have a similar granularity |
108 |
as files, but they are immutable, since any change to the |
as regular files, but they are immutable, since any change to the |
109 |
byte sequence would change the hash (and thus create a different block). |
byte sequence would change the hash (and thus create a different block). |
110 |
|
Mutable data structures are built on top of the immutable blocks |
111 |
|
(see Section 5). |
112 |
|
|
113 |
|
Block storage makes it easy to replicate data between systems. |
114 |
|
Different versions of the same document can easily coexist at this level, |
115 |
|
stored in different blocks. To replicate all data from computer A |
116 |
|
on computer B, it suffices to copy all blocks from A to B that B |
117 |
|
does not already store. |
118 |
|
|
119 |
|
Storm blocks are MIME messages [ref MIME], i.e., objects with |
120 |
|
a header and body as used in Internet mail or HTTP. |
121 |
|
This allows them to carry any metadata that can be carried |
122 |
|
in a MIME header, most importantly a content type. |
123 |
|
|
124 |
|
Collections of Storm blocks are called *pools*. Pools provide |
125 |
|
the following interface:: |
126 |
|
|
127 |
|
getIds() -> list |
128 |
|
get(id) -> block |
129 |
|
add(block) |
130 |
|
delete(block) |
131 |
|
|
132 |
|
Implementations may store blocks in RAM, in individual files, |
133 |
|
in a Zip archive, in a database or through other means. |
134 |
|
We have implemented the first three (using hexadecimal |
135 |
|
representations of the block ids for file names). |
136 |
|
|
137 |
|
When used in a network environment, Storm ids do not provide |
138 |
|
a hint as to where in the network the matching block can be found. |
139 |
|
However, current peer-to-peer systems could be used to |
140 |
|
find blocks in a distributed fashion; for example, Freenet [ref] |
141 |
|
and some Gnutella clients [ref] also use SHA-1-based identifiers. |
142 |
|
|
143 |
|
|
144 |
3. Xanalogical storage |
3. Xanalogical storage |