288 |
|
|
289 |
In Storm, all data is stored |
In Storm, all data is stored |
290 |
as *blocks*, byte sequences identified by a SHA-1 cryptographic content hash |
as *blocks*, byte sequences identified by a SHA-1 cryptographic content hash |
291 |
[ref SHA-1 and our ht'02 paper]. Blocks have a similar granularity |
[ref SHA-1 and our ht'02 paper]. |
292 |
|
Being purely a function of a block's content, block ids |
293 |
|
are completely independent of network location. |
294 |
|
Blocks have a similar granularity |
295 |
as regular files, but they are immutable, since any change to the |
as regular files, but they are immutable, since any change to the |
296 |
byte sequence would change the hash (and thus create a different block). |
byte sequence would change the hash (and thus create a different block). |
297 |
Mutable data structures are built on top of the immutable blocks |
Mutable data structures are built on top of the immutable blocks |
298 |
(see Section 6). |
(see Section 6). |
299 |
|
|
300 |
|
When used in a network environment, such ids do not provide |
301 |
|
a hint as to where a specific block is stored. |
302 |
|
However, many existing peer-to-peer systems could be used to |
303 |
|
find arbitrary blocks in a location-independent fashion; |
304 |
|
for example, Freenet [ref], recent Gnutella-based clients |
305 |
|
(e.g. Shareaza [ref]), and Overnet/eDonkey2000 [ref] |
306 |
|
also use SHA-1-based identifiers [e.g. ref: magnet uri]. |
307 |
|
(However, we have not put a network |
308 |
|
implementation into regular use yet and thus can only describe our |
309 |
|
design, not report on implementation experience. |
310 |
|
We discuss peer-to-peer implementations in Section 7, below.) |
311 |
|
|
312 |
|
Storm blocks are MIME messages [ref MIME], i.e., objects with |
313 |
|
a header and body as used in Internet mail or HTTP. |
314 |
|
This allows them to carry any metadata that can be carried |
315 |
|
in a MIME header, most importantly a content type. |
316 |
|
|
317 |
Storing data in immutable blocks may seem strange at first, but |
Storing data in immutable blocks may seem strange at first, but |
318 |
has a number of advantages. Firstly, it makes it easy |
has a number of advantages. First of all, it makes identifiers |
319 |
|
self-certifying: no matter where we have downloaded a block from, |
320 |
|
we are able to check we have the correct data by checking |
321 |
|
the cryptographic hash in the identifier. Therefore, we can |
322 |
|
safely download blocks from an untrusted peer. |
323 |
|
|
324 |
|
While digital signatures also allow for self-certifying identifiers, |
325 |
|
they raise the need for a public-key infrastructure (PKI) |
326 |
|
and for a timestamping mechanism in order to be reliable |
327 |
|
for more than a short time. |
328 |
|
|
329 |
|
When we make a reference to a block, we can be sure |
330 |
|
that even the original author of the target will not be able |
331 |
|
to change it. For example, if a newspaper refers to a letter |
332 |
|
to the editor this way, the letter's sender won't be able to change |
333 |
|
the reference into an advertisement for a pornographic web page. |
334 |
|
|
335 |
|
Secondly, caching becomes trivial, since it is |
336 |
|
never necessary to check for new versions of blocks. It is easy |
337 |
to replicate data between systems: A replica of a block never |
to replicate data between systems: A replica of a block never |
338 |
needs to be updated; cached copies can be kept as long as desired. |
needs to be updated; cached copies can be kept as long as desired. |
339 |
When a document is replicated, different versions of it can |
When a document is replicated, different versions of it can |
340 |
coexist on the same system without naming conflicts, since |
coexist on the same system without naming conflicts, since |
341 |
each version will be stored in its own block with its own id. |
each version will be stored in its own block with its own id. |
342 |
|
|
343 |
|
If peers make the blocks in their caches available on the network, |
344 |
|
the flash crowd problem could be alleviated: The more users |
345 |
|
request a block, the more locations there are to download it from. |
346 |
|
This resembles e.g. the Squirrel |
347 |
|
web cache [ref] [more refs? -Hermanni]; however, downloads can be |
348 |
|
from *any* peer since the source does not need to be trusted. |
349 |
|
On the other hand, there are privacy |
350 |
|
concerns with exposing one's browser cache to the outside world. |
351 |
|
|
352 |
To replicate all data from computer A |
To replicate all data from computer A |
353 |
on computer B, it suffices to copy all blocks from A to B that B |
on computer B, it suffices to copy all blocks from A to B that B |
354 |
does not already store. This can be done through a simple 'copy' |
does not already store. This can be done through a simple 'copy' |
358 |
when a user wants to incorporate a set of changes, but not |
when a user wants to incorporate a set of changes, but not |
359 |
required at replication time.) |
required at replication time.) |
360 |
|
|
361 |
Secondly, immutable blocks increase *reliability*. |
.. On the other hand for instance, several popular |
362 |
|
database management systems (e.g. Lotus Notes [ref]) have complex |
363 |
|
replication schemes, which may led awkward replication conflicts, |
364 |
|
because of they lack the immutable properties of data. |
365 |
|
[Or does this belong to diff section ? -Hermanni] |
366 |
|
|
367 |
|
The same namespace is used for local data and data |
368 |
|
retrieved from the network. When an online document has been |
369 |
|
permanently downloaded to the local harddisk, it can be found |
370 |
|
by a browser just as data from the network. This is convenient |
371 |
|
for offline browsing, for example in mobile environments: |
372 |
|
After a document has been downloaded, links to it will *never* |
373 |
|
cease to work, online or offline. |
374 |
|
|
375 |
|
Thirdly, immutable blocks increase *reliability*. |
376 |
When saving a document, an application will only *add* blocks, |
When saving a document, an application will only *add* blocks, |
377 |
never overwrite existing data. When a bug causes an application |
never overwrite existing data. When a bug causes an application |
378 |
to write malformed data, only the changes from one session |
to write malformed data, only the changes from one session |
379 |
will be lost; the previous version of the data will still |
will be lost; the previous version of the data will still |
380 |
be accessible. This makes Storm well suited as a basis |
be accessible [#]_. This makes Storm well suited as a basis |
381 |
for implementing experimental projects (such as ours). |
for implementing experimental projects (such as ours, Gzz). |
382 |
Even production systems occasionally corrupt existing data |
Even production systems occasionally corrupt existing data |
383 |
when an overwriting save operation goes awry; for example, |
when an overwriting save operation goes awry; for example, |
384 |
one of the authors has had this problem with |
one of the authors has had this problem with |
385 |
Microsoft Word many times. |
Microsoft Word many times. |
386 |
|
|
387 |
.. On the other hand for instance, several popular |
.. [#] Unfortunately, efficient versioned storage (Section 6) |
388 |
database management systems (e.g. Lotus Notes [ref]) have complex |
makes matters a little more complicated; still, |
389 |
replication schemes, which may led awkward replication conflicts, |
the basic assertion holds. |
390 |
because of they lack the immutable properties of data. |
|
391 |
[Or does this belong to diff section ? -Hermanni] |
Even when a publisher's server fails to serve a block, |
392 |
|
links to it will not cease to work until *no* other peer |
393 |
Storm blocks are MIME messages [ref MIME], i.e., objects with |
holds a copy. This means that providing mirrors is trivial. |
394 |
a header and body as used in Internet mail or HTTP. |
Even after failure of all of the publisher's mirrors, |
395 |
This allows them to carry any metadata that can be carried |
a document may still be available from peers that have |
396 |
in a MIME header, most importantly a content type. |
downloaded it. An archive of published blocks, in the spirit |
397 |
|
of the Web archive [ref], would only be yet another backup; |
398 |
|
normal links to a block would work as long as the archive |
399 |
|
holds a copy. It would also be hard to purposefully remove |
400 |
|
a published document from the network; whether this is |
401 |
|
a good or a bad property we leave for the reader to judge. |
402 |
|
|
403 |
|
These advantages are bought by an utter incompatibility with |
404 |
|
the dominant paradigms of file names and URLs. We hope that |
405 |
|
it would be possible to port existing applications to use Storm |
406 |
|
without too much effort, but we have not investigated |
407 |
|
the issue closely. This is because Storm was developed |
408 |
|
for the experimental Gzz system, a platform explicitly developed |
409 |
|
to overcome the limitations of traditional file-based applications. |
410 |
|
|
411 |
Collections of Storm blocks are called *pools*. Pools provide |
Collections of Storm blocks are called *pools*. Pools provide |
412 |
the following interface:: |
the following interface:: |
417 |
delete(block) |
delete(block) |
418 |
|
|
419 |
Implementations may store blocks in RAM, in individual files, |
Implementations may store blocks in RAM, in individual files, |
420 |
in a Zip archive, in a database or through other means. |
in a Zip archive, in a database, in a p2p network, |
421 |
|
or through other means. |
422 |
We have implemented the first three (using hexadecimal |
We have implemented the first three (using hexadecimal |
423 |
representations of the block ids for file names). |
representations of the block ids for file names). |
424 |
|
|
425 |
When used in a network environment, Storm IDs do not provide |
Sometimes it is useful to think about *zones* blocks are in, |
426 |
a hint as to where a specific block is stored in the network. |
related to distribution policy: for example, a *public* |
427 |
However, many existing peer-to-peer systems could be used to |
zone for blocks served to others in the network, a *private* |
428 |
find arbitrary blocks in a location independent fashion; |
zone for a user's own data, and a *local* zone with blocks |
429 |
for example, Freenet [ref], recent Gnutella-based clients |
served to anyone on the local intranet. It is essential |
430 |
(e.g. Shareaza [ref]), Overnet/eDonkey2000 [ref] also use SHA-1-based identifiers |
that blocks from a private zone are not leaked into a public |
431 |
[e.g. ref: magnet uri]. Footnote:However, we have not put a network |
zone without the user's consent. On the other hand, making |
432 |
implementation into regular use yet and thus can only describe our |
a document available on the network should be as easy as |
433 |
design, not report on implementation experience. |
hitting a 'publish' button moving it to a public zone, |
434 |
We discuss peer-to-peer implementations in Section 7, below. |
possibly also uploading it to a server permanently connected |
435 |
|
to the Internet, if one is available. |
436 |
The immutability of blocks should make caching trivial, since it is |
|
437 |
never necessary to check for new versions of blocks. |
Unfortunately, we have not found a satisfactory representation |
438 |
Since the same namespace is used for local data and data |
of zones yet. In particular, how do we decide which zone |
439 |
retrieved from the network, online documents that have been |
a new block should be in? Probably in the private zone |
440 |
permanently downloaded to the local harddisk can also be found |
in many cases, but if we have been editing a document |
441 |
by the caching mechanism. This is convenient for offline browsing, |
collaboratively edited in a workgroup, we would want |
442 |
for example in mobile environments: Users can download documents |
our changes to be available to the same group. |
443 |
while they are online, store them locally, and be sure that |
|
444 |
their software will be able to access them as if downloaded |
An important open issue with block storage are |
445 |
from the net, without broken links. |
UI conventions for listing, moving and deleting blocks. |
446 |
[Previous sentence doesn't parse to me: more simple :( -Hermanni] |
Currently, the only interface is a file system directory |
447 |
|
containing a set of blocks as files with hexadecimal, |
448 |
Given a peer-to-peer distribution mechanism, it would be possible |
random-looking names. In Gzz, we currently trick our way around |
449 |
to retrieve blocks from any participating peer online that has a copy |
the problem; at startup time, we simply load the most current |
450 |
in its local cache or permanent storage. This is similar to the Squirrel |
version of a document whose identifier is hard-wired into |
451 |
web cache [ref] [more refs? -Hermanni], but does not require trust |
the software (mutable documents are described in section 6.1). |
|
between the peers, since it is possible to check the blocks' integrity by using |
|
|
cryptographic hashes, as used in many peer-to-peer applications |
|
|
(e.g. [ref: ed2k/overnet, shareaza]). Since much-requested blocks would be |
|
|
cached on many systems, a network could deal with hotspots |
|
|
much more easily. On the other hand, there are privacy |
|
|
concerns with exposing one's browser cache to the outside world. |
|
|
|
|
|
|
|
|
[Merge this paragraph with 5) ? -Hermanni] |
|
|
That all data is stored in blocks means that links to it |
|
|
are completely independent of location; when data is moved |
|
|
between servers, references to it do not break. (Footnote: Of course, |
|
|
this requires that the blocks can be found no matter what server |
|
|
they are on. Again, see Section 7.) |
|
|
|
|
|
[Is there disadvantages/issus which we are aware of ? -Hermanni] |
|
452 |
|
|
453 |
|
|
454 |
4. Xanalogical storage |
4. Xanalogical storage |
533 |
that this will not be a major scalability problem. Otherwise, |
that this will not be a major scalability problem. Otherwise, |
534 |
systems that allow range queries, such as skip graphs [ref] |
systems that allow range queries, such as skip graphs [ref] |
535 |
and skipnet [ref], may prove useful. |
and skipnet [ref], may prove useful. |
|
[how does video work here, i.e. are they huge blocks or collections of many |
|
|
small, frames? or a sequence between two keyframes?] |
|
536 |
|
|
537 |
[This might be relevant also: |
.. [how does video work here, i.e. are they huge blocks or collections of many |
538 |
http://www.hpl.hp.com/techreports/2002/HPL-2002-209.pdf |
small, frames? or a sequence between two keyframes?] |
539 |
-Hermanni] |
|
540 |
|
Benja says: Probably large, but the number of links to them |
541 |
|
still won't be that big, I guess. Not sure how to discuss this |
542 |
|
in the text; feel free to propose something :-) |
543 |
|
|
544 |
|
.. [This might be relevant also: |
545 |
|
http://www.hpl.hp.com/techreports/2002/HPL-2002-209.pdf |
546 |
|
-Hermanni] |
547 |
|
|
548 |
One question raised by xanalogical storage is which links to show |
One question raised by xanalogical storage is which links to show |
549 |
for a popular document that has been linked to by many users. |
for a popular document that has been linked to by many users. |
566 |
implementation on top of networking overlay (e.g. distributed hashtable) |
implementation on top of networking overlay (e.g. distributed hashtable) |
567 |
will be trivial. |
will be trivial. |
568 |
|
|
569 |
[Benja, this might be useful for defining Storm APIs for DHTs etc: |
.. [Benja, this might be useful for defining Storm APIs for DHTs etc: |
570 |
http://sahara.cs.berkeley.edu/jan2003-retreat/ravenben_api_talk.pdf |
http://sahara.cs.berkeley.edu/jan2003-retreat/ravenben_api_talk.pdf |
571 |
Full paper will appear in IPTPS 2003 -Hermanni] |
Full paper will appear in IPTPS 2003 -Hermanni] |
572 |
|
|
573 |
|
Benja says: Hm, does that belong in the p2p section? |
574 |
|
|
575 |
In Storm, applications are not allowed to put arbitrary |
In Storm, applications are not allowed to put arbitrary |
576 |
mappings into the index. Instead, applications that want |
mappings into the index. Instead, applications that want |
632 |
are able to scale to the load to store a mapping for each word |
are able to scale to the load to store a mapping for each word |
633 |
occuring in a document. |
occuring in a document. |
634 |
|
|
635 |
[There are two refs about keywords in DHTs-- should we ref these ? -Hermanni] |
.. [There are two refs about keywords in DHTs-- should we ref these ? -Hermanni] |
636 |
|
|
637 |
|
Not sure how applicable they are: our system is *not* |
638 |
|
as general or performant as a DHT (as explained above). |
639 |
|
Should read & find out whether they could be implemented |
640 |
|
through our index system at all... -b |
641 |
|
|
642 |
|
|
643 |
6. Versioning |
6. Versioning |
653 |
mapping pointer identifiers to blocks providing targets for that pointer. |
mapping pointer identifiers to blocks providing targets for that pointer. |
654 |
Through this mechanism, we can keep old versions of documents |
Through this mechanism, we can keep old versions of documents |
655 |
along with the current versions. |
along with the current versions. |
656 |
[Figure ? -Hermanni] |
|
657 |
|
.. [Figure ? -Hermanni] |
658 |
|
|
659 |
Secondly, in the spirit of version control systems like CVS, |
Secondly, in the spirit of version control systems like CVS, |
660 |
we do not store *each version*, but only the differences between versions. |
we do not store *each version*, but only the differences between versions. |
663 |
When we want to access a particular version, we reconstruct it |
When we want to access a particular version, we reconstruct it |
664 |
using the differences, and then check the result using |
using the differences, and then check the result using |
665 |
the cryptographic hash in the full version's block id. |
the cryptographic hash in the full version's block id. |
666 |
[Figure ? -Hermanni] |
|
667 |
|
.. [Figure ? -Hermanni] |
668 |
|
|
669 |
6.1. Pointers |
6.1. Pointers |
670 |
------------- |
------------- |
1005 |
9. Conclusions |
9. Conclusions |
1006 |
============== |
============== |
1007 |
|
|
|
XXX |
|
1008 |
|
XXX |
1009 |
|
|
1010 |
|
|
1011 |
|
10. Acknowledgements |
1012 |
|
==================== |
1013 |
|
|
1014 |
|
We would like to thank Sarah Stehlig for discussions. |
1015 |
|
|
1016 |
|
.. (Add others here. Sarah pointed out the privacy issues with |
1017 |
|
exposing one's cache-- duh, but I for one hadn't thought |
1018 |
|
about it! And the Squirrel paper doesn't mention it either. |
1019 |
|
Probably it's pointed out somewhere in the literature-- |
1020 |
|
too basic a problem not to have been noticed-- but still... -b) |