15 |
ever to be changed, but that it is not possible to resolve such names |
ever to be changed, but that it is not possible to resolve such names |
16 |
on a global scale. However, recent development in peer-to-peer systems |
on a global scale. However, recent development in peer-to-peer systems |
17 |
has made scalable indexing systems possible, rendering this assumption |
has made scalable indexing systems possible, rendering this assumption |
18 |
obsolete. This, we believe, is the most important result of |
obsolete [refs: chord, can, tapestry, pastry, kademlia, symphony, viceroy]. |
19 |
peer-to-peer research with regard to hypermedia. |
This, we believe, is the most important result of peer-to-peer research with |
20 |
|
regard to hypermedia. |
21 |
[/maybe] |
[/maybe] |
22 |
|
|
23 |
In today's computing world, documents move quite freely between computers, being |
In today's computing world, documents move quite freely between computers, being |
146 |
================ |
================ |
147 |
|
|
148 |
In our system, Storm (for *storage module*), all data is stored |
In our system, Storm (for *storage module*), all data is stored |
149 |
in *blocks*, byte sequences identified by a SHA-1 cryptographic hash |
as *blocks*, byte sequences identified by a SHA-1 cryptographic content-hash |
150 |
[ref SHA-1 and our ht'02 paper]. Blocks often have a similar granularity |
[ref SHA-1 and our ht'02 paper]. Blocks often have a similar granularity |
151 |
as regular files, but they are immutable, since any change to the |
as regular files, but they are immutable, since any change to the |
152 |
byte sequence would change the hash (and thus create a different block). |
byte sequence would change the hash (and thus create a different block). |
153 |
Mutable data structures are built on top of the immutable blocks |
Mutable data structures are built on top of the immutable blocks |
154 |
(see Section 5). |
(see Section 5). |
155 |
|
|
156 |
|
CFS [ref], which is built upon Chord routing layer[ref], store data as blocks. |
157 |
|
However, CFS *splits* files into several miniblocks and spreads blocks over the |
158 |
|
available CFS servers. Freenet [ref] and PAST [ref, pastry ref] doesn't split |
159 |
|
files into blocks, since they store data as whole files. All previously mentioned |
160 |
|
systems lack of the immutable property which is used in Storm blocks. |
161 |
|
|
162 |
|
Immutable blocks has several benefits... |
163 |
|
|
164 |
Block storage makes it easy to replicate data between systems. |
Block storage makes it easy to replicate data between systems. |
165 |
Different versions of the same document can easily coexist at this level, |
Different versions of the same document can easily coexist at this level, |
166 |
stored in different blocks. To replicate all data from computer A |
stored in different blocks. To replicate all data from computer A |
196 |
When used in a network environment, Storm ids do not provide |
When used in a network environment, Storm ids do not provide |
197 |
a hint as to where in the network the matching block can be found. |
a hint as to where in the network the matching block can be found. |
198 |
However, current peer-to-peer systems could be used to |
However, current peer-to-peer systems could be used to |
199 |
find blocks in a distributed fashion; for example, Freenet [ref] |
find blocks in a distributed fashion; for example, Freenet [ref], |
200 |
and some Gnutella clients [ref] also use SHA-1-based identifiers. |
a few recent Gnutella clients [e.g. ref: shareaza] , Overnet/eDonkey2000 [ref] |
201 |
|
also use SHA-1-based identifiers [e.g. ref: magnet uri]. |
202 |
However, we have not put a network implementation into regular use |
However, we have not put a network implementation into regular use |
203 |
yet and thus can only describe our design, not report on |
yet and thus can only describe our design, not report on |
204 |
implementation experience. |
implementation experience. |
290 |
XXX |
XXX |
291 |
|
|
292 |
|
|
293 |
|
<<<<<<< article.rst |
294 |
|
Idea/Plan |
295 |
|
========= |
296 |
|
|
297 |
|
[Notes for the authors, not part of the final document |
298 |
|
though text may be moved from here to there.] |
299 |
|
|
300 |
|
Whenever a document moves on the current web, links to it break, |
301 |
|
be it from an author's computer to a public server, |
302 |
|
from one server to another, from the server to a client, |
303 |
|
or from one personal computer to another. We subsume |
304 |
|
these forms of movement under the term 'data mobility.' |
305 |
|
|
306 |
|
|
307 |
|
Storm goals/benefits: |
308 |
|
|
309 |
|
- Reliability |
310 |
|
- Append-and-delete-only |
311 |
|
- The same data can be stored in many locations, |
312 |
|
allowing it to be easily reconstructed after failure |
313 |
|
- Versioning: Old versions remain accessible |
314 |
|
- Xanalogical storage |
315 |
|
- If a document is accessible, references to it work |
316 |
|
- Links do not break |
317 |
|
- Easy syncing: |
318 |
|
- Just copy a bunch of blocks |
319 |
|
- Documents can be synced & merged |
320 |
|
- Inter-document structures can be synced & merged |
321 |
|
- Syncing can be done without merging immediately, |
322 |
|
leaving two alternative versions current |
323 |
|
(so e.g. an automated process is entirely possible, |
324 |
|
even when there are conflicts) |
325 |
|
- Versioning |
326 |
|
|
327 |
|
|
328 |
|
Grouped differently, |
329 |
|
|
330 |
|
- Reliability (as above) |
331 |
|
- Usablility in the face of intermittent connectivity |
332 |
|
(includes syncing, finding a document if available...) |
333 |
|
- Xanalogical structure |
334 |
|
(includes versioning, non-breaking links etc.) |
335 |
|
|
336 |
|
Storm limitations/weaknesses: |
337 |
|
|
338 |
|
- what, actually? |
339 |
|
|
340 |
|
antont ponders: for files storm is ok, but how about: |
341 |
|
- irc? (latency?) |
342 |
|
- video? (throughput) |
343 |
|
|
344 |
|
and: |
345 |
|
.. multipoint live video? (both latency and throughput demands) |
346 |
|
|
347 |
|
* does it make sense to think of irc messages, and/or video frames, as |
348 |
|
datablocks .. or what? |
349 |
|
|
350 |
|
|
351 |
|
hemppah's comment on syncing term: |
352 |
|
I'd prefer term 'replication' instead of term syncing, when |
353 |
|
updating data to 'the most recent state'. E.g. Lotus Notes uses |
354 |
|
term replication, when one performs locally made updates into |
355 |
|
a centralized server --> 'used within same system'. Syncing term, however, |
356 |
|
is used when importing/exporting e.g. Nokia Communicator calendar data into/from |
357 |
|
Lotus Notes calendar --> 'used between different systems'. |
358 |
|
|
359 |
|
|
360 |
|
hemppah: worth to mention is that Ray Ozzie is a man behind Lotus Notes and Groove; |
361 |
|
Lotus Notes is based on client-server model and, Groove is based on p2p model --> |
362 |
|
possible direction etc. ? |
363 |
|
|
364 |
|
hemppah: I think we should mention that in Gzz one refer to data in non-hierarchial |
365 |
|
way, where as in Notes (and other systems also, references!!), we must use |
366 |
|
hierarchial way. In Notes most important IDs are: |
367 |
|
1) every document has a unique identifier, which is unique among all replicas |
368 |
|
of database |
369 |
|
2) every document/design element has a identifier, named as noteID, which is unique |
370 |
|
in database, but not among all replicas of database |
371 |
|
3) every view has a unique identifier, which is unique among all replicas of |
372 |
|
database |
373 |
|
4) every database has a replica ID, which identifies database's replicas |
374 |
|
among all databases |
375 |
|
|
376 |
|
So, if we want to refer to a document, we use format: |
377 |
|
|
378 |
|
replicaID/viewID/documentID |
379 |
|
|
380 |
|
Also, we can refer to same document, through *many different* views (analogical to Gzz's dimensions ?): |
381 |
|
notes://<server>/replicaID/viewID1/documentID |
382 |
|
notes://<server>/replicaID/viewID2/documentID |
383 |
|
|
384 |
|
Here's a real example: |
385 |
|
Notes://server/D235632D00313587/38D22BF5E8F088348525JK7500129B2C/REWB3FDE0D53807B67C2256CB50026FCVV |
386 |
|
|
387 |
|
For information about IDs in Notes: |
388 |
|
http://www-12.lotus.com/ldd/doc/tools/c/4.5/api45ug.nsf/85255d56004d2bfd85255b1800631684/00d000c1005800c985255e0e00726863?OpenDocument |
389 |
|
|
390 |
|
In Gzz, however, we don't know the location, we know only the *identity* of data what we are looking for, as follows: |
391 |
|
|
392 |
|
urn-5:FAB3FDE0hgfD5kkjj3807B67C2256CfsdB50026FC51 |
393 |
|
|
394 |
|
Above is not a *correct* urn-5, but very similar to last part of notes' syntax. |
395 |
|
|
396 |
|
<<<<<<< article.rst |
397 |
|
benja's reply: |
398 |
|
Hm. Replication to me means, the same data is kept on multiple |
399 |
|
machines. This is not what we are talking about here: We're talking |
400 |
|
about *different versions* of the same data being kept |
401 |
|
on multiple machines, and occasionally being 'brought into sync' |
402 |
|
with each other. If I send you a draft article and you comment on it, |
403 |
|
and I make changes too, and later I merge the two divergent |
404 |
|
versions back together, 'syncing' seems approximately right, |
405 |
|
but 'replication' seems completely wrong to me. |
406 |
|
|
407 |
|
hemppah's reply: |
408 |
|
Hm ;). |
409 |
|
When same data is kept on multiple mahcines, each instance is called replica |
410 |
|
of data. When we merge different *versions* of replicas to same 'version', this |
411 |
|
is called replication. |
412 |
|
|
413 |
|
When we want information to exchanged between different kind of *machines* |
414 |
|
(e.g. between TabletPC's calendar and Notes), we call it syncing. |
415 |
|
|
416 |
|
Above is described how things work in Notes and what terms are used. And, your |
417 |
|
example is just that what replication does in Notes; merge changes, made in |
418 |
|
different version of replicas, into a new *different* version. |
419 |
|
|
420 |
|
|
421 |
|
======= |
422 |
|
(Of course, this is very similar to 'normal' URLs, but our purpose here is to give an example |
423 |
|
of how one refer to a particular data item in colloboration-like tool, like Notes) |
424 |
|
>>>>>>> 1.28 |
425 |
|
|
426 |
|
In Notes, there are servers, which maintain replication of data, opposite to Gzz. What is |
427 |
|
interesting in Notes' replication, is the fact that the replication of database not only |
428 |
|
replicate the *data* but also design of data, which represent the data. |
429 |
|
Worth to mention is also that, even if the data and the design of data (logic etc.) is |
430 |
|
in the *same* (physical) structure, data and design of data is very loosely coupled with |
431 |
|
each other. |
432 |
|
|
433 |
|
Additionally, we should emphatize that how things are going towards non-hierarchical reference models, |
434 |
|
for instance, Notes(hierarchical) and Gzz(non-hierarchical), which both are based on the same |
435 |
|
xanalogical model. |
436 |
|
|
437 |
|
"Usability in the face of intermittent connectivity" is |
438 |
|
more than just mobile applications: It is also copying data |
439 |
|
from one computer to another, where the two computers' |
440 |
|
file systems are not kept in sync through a permanent |
441 |
|
network connection. Hmm, maybe "Usability in the face |
442 |
|
of irregular synchronization" or some such would |
443 |
|
make it clearer? |
444 |
|
|
445 |
|
Ok, let's split that in two: |
446 |
|
|
447 |
|
- Usability in the face of intermittent connectivity |
448 |
|
(we cannot access data stored on the internet) |
449 |
|
- Usability in the face of non-synchronization |
450 |
|
(we can have two independent versions of something |
451 |
|
on two unconnected computers and we can easily |
452 |
|
synchronize the two versions when desired) |
453 |
|
|
454 |
|
======= |
455 |
6. Peer-to-peer implementations |
6. Peer-to-peer implementations |
456 |
=============================== |
=============================== |
457 |
|
>>>>>>> 1.44 |
458 |
|
|
459 |
XXX |
XXX |
460 |
|
|