15 |
However, recent developments in peer-to-peer systems have |
However, recent developments in peer-to-peer systems have |
16 |
rendered this assumption obsolete. Distributed hashtables |
rendered this assumption obsolete. Distributed hashtables |
17 |
[ref chord, can, tapestry, pastry, kademlia, symphony, viceroy] |
[ref chord, can, tapestry, pastry, kademlia, symphony, viceroy] |
18 |
and similar systems [skip graph, swan] allow *location independent* routing |
and similar systems [skip graph, swan, peernet] allow *location independent* |
19 |
based on random identifiers on a global scale. This, we believe, |
routing based on random identifiers on a global scale. This, we believe, |
20 |
may be the most important result of intense peer-to-peer |
may be the most important result of intense peer-to-peer |
21 |
research with regard to hypermedia. |
research with regard to hypermedia. |
22 |
|
|
259 |
|
|
260 |
Immutable blocks has several benefits over existing systems... |
Immutable blocks has several benefits over existing systems... |
261 |
|
|
262 |
1) Storm's block storage makes it easy to replicate data between systems. |
Storm's block storage makes it easy to replicate data between systems. |
263 |
Different versions of the same document can easily coexist at this level, |
Different versions of the same document can easily coexist at this level, |
264 |
stored in different blocks. |
stored in different blocks. |
265 |
[Previous sentence doesn't parse to me (what level ?) :( -Hermanni] |
[Previous sentence doesn't parse to me (what level ?) :( -Hermanni] |
266 |
To replicate all data from computer A |
To replicate all data from computer A |
267 |
on computer B, it suffices to copy all blocks from A to B that B |
on computer B, it suffices to copy all blocks from A to B that B |
268 |
does not already store. |
does not already store. On the other hand, several popular database management |
269 |
[Example of Lotus Notes' replication conficts ? -Hermanni] |
systems (e.g. Lotus Notes [ref]) have complex replication schemes, which may |
270 |
|
led awkward replication conflicts. |
271 |
|
[Or does this belong to diff section ? -Hermanni] |
272 |
|
|
273 |
2) Storm blocks are MIME messages [ref MIME], i.e., objects with |
Storm blocks are MIME messages [ref MIME], i.e., objects with |
274 |
a header and body as used in Internet mail or HTTP. |
a header and body as used in Internet mail or HTTP. |
275 |
This allows them to carry any metadata that can be carried |
This allows them to carry any metadata that can be carried |
276 |
in a MIME header, most importantly a content type. |
in a MIME header, most importantly a content type. |
285 |
|
|
286 |
[analogy to regular Hash Table/DHT ? -Hermanni] |
[analogy to regular Hash Table/DHT ? -Hermanni] |
287 |
|
|
288 |
3) Implementations may store blocks in RAM, in individual files, |
Implementations may store blocks in RAM, in individual files, |
289 |
in a Zip archive, in a database or through other means. |
in a Zip archive, in a database or through other means. |
290 |
We have implemented the first three (using hexadecimal |
We have implemented the first three (using hexadecimal |
291 |
representations of the block ids for file names). |
representations of the block ids for file names). |
292 |
|
|
293 |
4) Storing all data in Storm blocks provides *reliability*: |
Storing all data in Storm blocks provides *reliability*: |
294 |
When saving a document, an application will only *add* blocks, |
When saving a document, an application will only *add* blocks, |
295 |
never overwrite existing data. When a bug causes an application |
never overwrite existing data. When a bug causes an application |
296 |
to write malformed data, only the changes from one session |
to write malformed data, only the changes from one session |
298 |
be accessible. (Footnote: This makes Storm well suited as a basis |
be accessible. (Footnote: This makes Storm well suited as a basis |
299 |
for implementing experimental projects (such as ours).) |
for implementing experimental projects (such as ours).) |
300 |
|
|
301 |
5) When used in a network environment, Storm ids do not provide |
When used in a network environment, Storm ids do not provide |
302 |
a hint as to where in the network the matching block can be found. |
a hint as to where in the network a specific block can be found. |
303 |
However, current peer-to-peer systems could be used to |
However, current peer-to-peer systems could be used to |
304 |
find blocks efficiently in a distributed fashion; for example, |
find blocks efficiently in a distributed fashion; for example, |
305 |
Freenet [ref], a few recent Gnutella clients [e.g. ref: shareaza], |
Freenet [ref], a few recent Gnutella clients (e.g. Shareaza [ref]), |
306 |
Overnet/eDonkey2000 [ref] also use SHA-1-based identifiers |
Overnet/eDonkey2000 [ref] also use SHA-1-based identifiers |
307 |
[e.g. ref: magnet uri]. |
[e.g. ref: magnet uri]. |
308 |
(Footnote:However, we have not put a network implementation into regular use |
(Footnote:However, we have not put a network implementation into regular use |
310 |
implementation experience.) |
implementation experience.) |
311 |
We discuss peer-to-peer implementations in Section 7, below. |
We discuss peer-to-peer implementations in Section 7, below. |
312 |
|
|
313 |
6) The immutability of blocks should make caching trivial, since it is |
The immutability of blocks should make caching trivial, since it is |
314 |
never necessary to check for new versions of blocks. |
never necessary to check for new versions of blocks. |
315 |
Since the same namespace [mention urn-5 ? -Hermanni] is used for local data and data |
Since the same namespace is used for local data and data |
316 |
retrieved from the network, online documents that have been |
retrieved from the network, online documents that have been |
317 |
permanently downloaded to the local harddisk can also be found |
permanently downloaded to the local harddisk can also be found |
318 |
by the caching mechanism. This is convenient for offline browsing, |
by the caching mechanism. This is convenient for offline browsing, |
740 |
network is created and maintained and how queries are performed. DHT is seen as |
network is created and maintained and how queries are performed. DHT is seen as |
741 |
scalable approach and usually provides (poly)logarithmic bounds to *all* internal |
scalable approach and usually provides (poly)logarithmic bounds to *all* internal |
742 |
operations (footnote about 'stable state' ?), while broadcasting can't achieve |
operations (footnote about 'stable state' ?), while broadcasting can't achieve |
743 |
either of these. |
either of these. |
744 |
|
|
745 |
|
[footnote: It's not clear whether all proposed DHT designs can preserve |
746 |
|
(poly)logarithmic properties when nodes join leave the system in a dynamic manner |
747 |
|
|
748 |
In DHT approach, both keys and the addresses of peers are mapped into one virtual |
In DHT approach, both keys and the addresses of peers are mapped into one virtual |
749 |
key space. The form of key space depends on implementation. The mapping makes |
key space. The form of key space depends on implementation (e.g. can be a circle). |
750 |
possible to assign number of data items to a peer, based on how 'close' data |
The mapping makes possible to assign number of data items to a peer, based on how |
751 |
item's and peer's keys are each other. Thus, DHT's overlay connectivity graph |
'close' (e.g. numerical, XOR) data item's and peer's keys are each other. Thus, |
752 |
is structured. On the other hand, the overlay connectivity graph of broadcasting |
DHT's overlay connectivity graph is structured. On the other hand, the overlay |
753 |
approach is formed more or less (depends on implementation) in a random manner. |
connectivity graph of broadcasting approach is formed more or less (depends on |
754 |
|
implementation) in a random manner. |
755 |
|
|
756 |
When performing queries, in broadcasting approach, peer sends a query request to a |
When performing queries, in broadcasting approach, peer sends a query request to a |
757 |
subset of its neighbors and these peers to their subsequent neighbors. The |
subset of its neighbors and these peers to their subsequent neighbors. The |
769 |
In broadcasting approach, implementations' differences mostly lie in the |
In broadcasting approach, implementations' differences mostly lie in the |
770 |
*structural level* of overlay network, i.e. super peers and peer clusters. |
*structural level* of overlay network, i.e. super peers and peer clusters. |
771 |
|
|
|
Recent work [ref: peernet] has concentrated on developing p2p infrastructure |
|
|
at the *network* layer. This is a different alternative to existing p2p |
|
|
infrastructures, which operate at the *application* layer. Initial design |
|
|
of the system is promising, since it has several benefits over application |
|
|
level p2p infrastructures (see paper). However, further research has to be |
|
|
done in order to affirm the applicability of the technique. |
|
|
The storm design presented here is independent of the network layer, i.e. it |
|
|
may be implemented as an overlay on an IP-based network or something else. |
|
772 |
|
|
773 |
Review of the use cases: what does storm in each? |
Review of the use cases: what does storm in each? |
774 |
------------------------------------------------- |
------------------------------------------------- |