/[gzz]/manuscripts/storm/article.rst
ViewVC logotype

Diff of /manuscripts/storm/article.rst

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.44 by benja, Tue Jan 28 03:44:00 2003 UTC revision 1.45 by hemppah, Tue Jan 28 09:16:47 2003 UTC
# Line 15  as unlike for a URL, there is little rea Line 15  as unlike for a URL, there is little rea
15  ever to be changed, but that it is not possible to resolve such names  ever to be changed, but that it is not possible to resolve such names
16  on a global scale. However, recent development in peer-to-peer systems  on a global scale. However, recent development in peer-to-peer systems
17  has made scalable indexing systems possible, rendering this assumption  has made scalable indexing systems possible, rendering this assumption
18  obsolete. This, we believe, is the most important result of  obsolete [refs: chord, can, tapestry, pastry, kademlia, symphony, viceroy].
19  peer-to-peer research with regard to hypermedia.  This, we believe, is the most important result of peer-to-peer research with
20    regard to hypermedia.
21  [/maybe]  [/maybe]
22    
23  In today's computing world, documents move quite freely between computers, being  In today's computing world, documents move quite freely between computers, being
# Line 145  Section 8 concludes the paper. Line 146  Section 8 concludes the paper.
146  ================  ================
147    
148  In our system, Storm (for *storage module*), all data is stored  In our system, Storm (for *storage module*), all data is stored
149  in *blocks*, byte sequences identified by a SHA-1 cryptographic hash  as *blocks*, byte sequences identified by a SHA-1 cryptographic content-hash
150  [ref SHA-1 and our ht'02 paper]. Blocks often have a similar granularity  [ref SHA-1 and our ht'02 paper]. Blocks often have a similar granularity
151  as regular files, but they are immutable, since any change to the  as regular files, but they are immutable, since any change to the
152  byte sequence would change the hash (and thus create a different block).  byte sequence would change the hash (and thus create a different block).
153  Mutable data structures are built on top of the immutable blocks  Mutable data structures are built on top of the immutable blocks
154  (see Section 5).  (see Section 5).
155    
156    CFS [ref], which is built upon Chord routing layer[ref], store data as blocks.
157    However, CFS *splits* files into several miniblocks and spreads blocks over the
158    available CFS servers. Freenet [ref] and PAST [ref, pastry ref] doesn't split
159    files into blocks, since they store data as whole files. All previously mentioned
160    systems lack of the immutable property which is used in Storm blocks.
161    
162    Immutable blocks has several benefits...
163    
164  Block storage makes it easy to replicate data between systems.  Block storage makes it easy to replicate data between systems.
165  Different versions of the same document can easily coexist at this level,  Different versions of the same document can easily coexist at this level,
166  stored in different blocks. To replicate all data from computer A  stored in different blocks. To replicate all data from computer A
# Line 187  for implementing experimental projects ( Line 196  for implementing experimental projects (
196  When used in a network environment, Storm ids do not provide  When used in a network environment, Storm ids do not provide
197  a hint as to where in the network the matching block can be found.  a hint as to where in the network the matching block can be found.
198  However, current peer-to-peer systems could be used to  However, current peer-to-peer systems could be used to
199  find blocks in a distributed fashion; for example, Freenet [ref]  find blocks in a distributed fashion; for example, Freenet [ref],
200  and some Gnutella clients [ref] also use SHA-1-based identifiers.  a few recent Gnutella clients [e.g. ref: shareaza] , Overnet/eDonkey2000 [ref]
201    also use SHA-1-based identifiers [e.g. ref: magnet uri].
202  However, we have not put a network implementation into regular use  However, we have not put a network implementation into regular use
203  yet and thus can only describe our design, not report on  yet and thus can only describe our design, not report on
204  implementation experience.  implementation experience.
# Line 280  as the system matures. Line 290  as the system matures.
290  XXX  XXX
291    
292    
293    <<<<<<< article.rst
294    Idea/Plan
295    =========
296    
297    [Notes for the authors, not part of the final document
298    though text may be moved from here to there.]
299    
300    Whenever a document moves on the current web, links to it break,
301    be it from an author's computer to a public server,
302    from one server to another, from the server to a client,
303    or from one personal computer to another. We subsume
304    these forms of movement under the term 'data mobility.'
305    
306    
307    Storm goals/benefits:
308    
309    - Reliability
310      - Append-and-delete-only
311      - The same data can be stored in many locations,
312        allowing it to be easily reconstructed after failure
313      - Versioning: Old versions remain accessible
314    - Xanalogical storage
315    - If a document is accessible, references to it work
316    - Links do not break
317    - Easy syncing:
318      - Just copy a bunch of blocks
319      - Documents can be synced & merged
320      - Inter-document structures can be synced & merged
321      - Syncing can be done without merging immediately,
322        leaving two alternative versions current
323        (so e.g. an automated process is entirely possible,
324        even when there are conflicts)
325    - Versioning
326    
327    
328    Grouped differently,
329    
330    - Reliability (as above)
331    - Usablility in the face of intermittent connectivity
332      (includes syncing, finding a document if available...)
333    - Xanalogical structure
334      (includes versioning, non-breaking links etc.)
335    
336    Storm limitations/weaknesses:
337    
338    - what, actually?
339    
340    antont ponders: for files storm is ok, but how about:
341    - irc? (latency?)
342    - video? (throughput)
343    
344    and:
345    .. multipoint live video? (both latency and throughput demands)
346    
347    * does it make sense to think of irc messages, and/or video frames, as
348    datablocks .. or what?
349    
350      
351    hemppah's comment on syncing term:
352    I'd prefer term 'replication' instead of term syncing, when
353    updating data to 'the most recent state'. E.g. Lotus Notes uses
354    term replication, when one performs locally made updates into
355    a centralized server --> 'used within same system'. Syncing term, however,
356    is used when importing/exporting e.g. Nokia Communicator calendar data into/from
357    Lotus Notes calendar --> 'used between different systems'.
358    
359    
360    hemppah: worth to mention is that Ray Ozzie is a man behind Lotus Notes and Groove;
361    Lotus Notes is based on client-server model and, Groove is based on p2p model -->
362    possible direction etc. ?
363    
364    hemppah: I think we should mention that in Gzz one refer to data in non-hierarchial
365    way, where as in Notes (and other systems also, references!!), we must use
366    hierarchial way. In Notes most important IDs are:
367    1) every document has a unique identifier, which is unique among all replicas
368    of database
369    2) every document/design element has a identifier, named as noteID, which is unique
370    in database, but not among all replicas of database
371    3) every view has a unique identifier,  which is unique among all replicas of
372    database
373    4) every database has a replica ID, which identifies database's replicas
374    among all databases
375    
376    So, if we want to refer to a document, we use format:
377    
378    replicaID/viewID/documentID
379    
380    Also, we can refer to same document, through *many different* views (analogical to Gzz's dimensions ?):
381    notes://<server>/replicaID/viewID1/documentID
382    notes://<server>/replicaID/viewID2/documentID
383    
384    Here's a real example:
385    Notes://server/D235632D00313587/38D22BF5E8F088348525JK7500129B2C/REWB3FDE0D53807B67C2256CB50026FCVV
386    
387    For information about IDs in Notes:
388    http://www-12.lotus.com/ldd/doc/tools/c/4.5/api45ug.nsf/85255d56004d2bfd85255b1800631684/00d000c1005800c985255e0e00726863?OpenDocument
389    
390    In Gzz, however, we don't know the location, we know only the *identity* of data what we are looking for, as follows:
391    
392    urn-5:FAB3FDE0hgfD5kkjj3807B67C2256CfsdB50026FC51
393    
394    Above is not a *correct* urn-5, but very similar to last part of notes' syntax.
395    
396    <<<<<<< article.rst
397    benja's reply:
398    Hm. Replication to me means, the same data is kept on multiple
399    machines. This is not what we are talking about here: We're talking
400    about *different versions* of the same data being kept
401    on multiple machines, and occasionally being 'brought into sync'
402    with each other. If I send you a draft article and you comment on it,
403    and I make changes too, and later I merge the two divergent
404    versions back together, 'syncing' seems approximately right,
405    but 'replication' seems completely wrong to me.
406    
407    hemppah's reply:
408    Hm ;).
409    When same data is kept on multiple mahcines, each instance is called replica
410    of data. When we merge different *versions* of replicas to same 'version', this
411    is called replication.
412    
413    When we want information to exchanged between different kind of *machines*
414    (e.g. between TabletPC's calendar and Notes), we call it syncing.
415    
416    Above is described how things work in Notes and what terms are used. And, your
417    example is just that what replication does in Notes; merge changes, made in
418    different version of replicas, into a new *different* version.
419    
420    
421    =======
422    (Of course, this is very similar to 'normal' URLs, but our purpose here is to give an example
423    of how one refer to a particular data item in colloboration-like tool, like Notes)
424    >>>>>>> 1.28
425    
426    In Notes, there are servers, which maintain replication of data, opposite to Gzz. What is
427    interesting in Notes' replication, is the fact that the replication of database not only
428    replicate the *data* but also design of data, which represent the data.
429    Worth to mention is also that, even if the data and the design of data (logic etc.) is
430    in the *same* (physical) structure, data and design of data is very loosely coupled with
431    each other.
432    
433    Additionally, we should emphatize that how things are going towards non-hierarchical reference models,
434    for instance, Notes(hierarchical) and Gzz(non-hierarchical), which both are based on the same
435    xanalogical model.
436    
437    "Usability in the face of intermittent connectivity" is
438    more than just mobile applications: It is also copying data
439    from one computer to another, where the two computers'
440    file systems are not kept in sync through a permanent
441    network connection. Hmm, maybe "Usability in the face
442    of irregular synchronization" or some such would
443    make it clearer?
444    
445    Ok, let's split that in two:
446    
447    - Usability in the face of intermittent connectivity
448      (we cannot access data stored on the internet)
449    - Usability in the face of non-synchronization
450      (we can have two independent versions of something
451      on two unconnected computers and we can easily
452      synchronize the two versions when desired)
453      
454    =======
455  6. Peer-to-peer implementations  6. Peer-to-peer implementations
456  ===============================  ===============================
457    >>>>>>> 1.44
458    
459  XXX  XXX
460    

Legend:
Removed from v.1.44  
changed lines
  Added in v.1.45

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26