/[gzz]/manuscripts/storm/article.rst
ViewVC logotype

Diff of /manuscripts/storm/article.rst

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.76 by hemppah, Mon Feb 3 10:01:54 2003 UTC revision 1.77 by hemppah, Mon Feb 3 13:40:20 2003 UTC
# Line 300  When saving a document, an application w Line 300  When saving a document, an application w
300  never overwrite existing data. When a bug causes an application  never overwrite existing data. When a bug causes an application
301  to write malformed data, only the changes from one session  to write malformed data, only the changes from one session
302  will be lost; the previous version of the data will still  will be lost; the previous version of the data will still
303  be accessible. (Footnote: This makes Storm well suited as a basis  be accessible. This makes Storm well suited as a basis
304  for implementing experimental projects (such as ours).)  for implementing experimental projects (such as ours).
305    
306  When used in a network environment, Storm ids do not provide  When used in a network environment, Storm ids do not provide
307  a hint as to where in the network a specific block can be found.  a hint as to where in the network a specific block can be found.
308  However, current peer-to-peer systems could be used to  However, current peer-to-peer systems could be used to
309  find blocks efficiently in a distributed fashion; for example,  find blocks efficiently in a location independent fashion; for example,
310  Freenet [ref], a few recent Gnutella clients (e.g. Shareaza [ref]),  Freenet [ref], recent Gnutella-based clients (e.g. Shareaza [ref]),
311  Overnet/eDonkey2000 [ref] also use SHA-1-based identifiers  Overnet/eDonkey2000 [ref] also use SHA-1-based identifiers
312  [e.g. ref: magnet uri].  [e.g. ref: magnet uri]. Footnote:However, we have not put a network
313  (Footnote:However, we have not put a network implementation into regular use  implementation into regular use yet and thus can only describe our
314  yet and thus can only describe our design, not report on  design, not report on implementation experience.
 implementation experience.)  
315  We discuss peer-to-peer implementations in Section 7, below.  We discuss peer-to-peer implementations in Section 7, below.
316    
317  The immutability of blocks should make caching trivial, since it is  The immutability of blocks should make caching trivial, since it is
# Line 362  A link is shown between any two document Line 361  A link is shown between any two document
361  that the link connects. Xanalogical links are external and bidirectional.  that the link connects. Xanalogical links are external and bidirectional.
362    
363  .. [#] Xanalogical storage is not limited to text. We speak about  .. [#] Xanalogical storage is not limited to text. We speak about
364     *characters* because it simplifies the explanation; pixels     *characters* because it simplifies the explanation; picture's pixels
365     or frames of video could be substituted.     or frames of video could be substituted.
366    
367  In addition to content links, xanalogical storage keeps an index of  In addition to content links, xanalogical storage keeps an index of
# Line 376  they refer to. Thus, for each character Line 375  they refer to. Thus, for each character
375  queries the index for other documents containing this character,  queries the index for other documents containing this character,
376  and shows them as transclusions. Resolving links is a multi-step process.  and shows them as transclusions. Resolving links is a multi-step process.
377  Each link is modeled as two collections of characters: the two  Each link is modeled as two collections of characters: the two
378  endpoints of the link. To show links to a document,  endpoints of the link. To show links to a specific document,
379  the system firstly uses the link index to find links  the system firstly uses the link index to find links
380  to each character in the documment. Secondly, for each link,  to each character in the document. Secondly, for each link,
381  it looks at the *other* set of characters in the link-- the target  it looks at the *other* set of characters in the link-- the target
382  of the link, if the original character was the source, and vice versa.  of the link, if the original character was the source, and vice versa.
383  Thirdly, it looks for documents containing these target characters.  Thirdly, it looks for documents containing these target characters.
384  This way, even if both the source and target cjaracters of the link  This way, even if both the source and target characters of the link
385  are moved to a different document, the link stays connected to them.  are moved to a different document, the link stays connected to them.
386    
387  Of course, doing any expensive operation for *every* character  Of course, doing any expensive operation for *every* character
# Line 397  from that session, we use the block's id Line 396  from that session, we use the block's id
396  character, and the number of characters in the span.  character, and the number of characters in the span.
397  This technique was first introduced in [ref ht02 paper].  This technique was first introduced in [ref ht02 paper].
398    
399  In Xanadu, characters are written to append-only *scrolls*  In Xanadu, characters are stored to append-only *scrolls*
400  when they are typed [ref]. Because of this, we call the blocks  when they are typed [ref]. Because of this, we call the blocks
401  containing the actual characters *scroll blocks*. The documents  containing the actual characters *scroll blocks*. The documents
402  do not actually contain the characters; instead, they are  do not actually contain the characters; instead, they are
# Line 415  and the characters retrieved from there Line 414  and the characters retrieved from there
414     the data is transcluded.     the data is transcluded.
415    
416  Our current implementation shows only links between documents  Our current implementation shows only links between documents
417  that are in memory at the same time [screenshot of xupdf].  that are in memory at the same time [screenshot of xupdf, perhaps ref too
418  In the future, we will implement a global index atop of   (submitted) ?].
419    In the future, we will implement a global distributed index at top of
420  a distributed hashtable, with the scroll blocks' ids as the keys.  a distributed hashtable, with the scroll blocks' ids as the keys.
421  To find the transclusions of a span, the system will retrieve  To find the transclusions of a span, the system will retrieve
422  all transclusions of any span in the scroll block, then  all transclusions of any span in the scroll block, then
# Line 427  the spans cannot be used as hashtable ke Line 427  the spans cannot be used as hashtable ke
427  will be relatively small (limited by the amount of text  will be relatively small (limited by the amount of text
428  the user enters between two saves of a document), we hope  the user enters between two saves of a document), we hope
429  that this will not be a major scalability problem. Otherwise,  that this will not be a major scalability problem. Otherwise,
430  systems that allow range queries, such as skip graphs [ref],  systems that allow range queries, such as skip graphs [ref]
431  may prove useful.  and skipnet [ref], may prove useful.
432    
433    [This might be relevant:
434    http://www.hpl.hp.com/techreports/2002/HPL-2002-209.pdf
435    -Hermanni]
436    
437  One question raised by xanalogical storage is which links to show  One question raised by xanalogical storage is which links to show
438  for a popular document that has been linked to by many users.  for a popular document that has been linked to by many users.
# Line 451  on a local machine, but the interface is Line 455  on a local machine, but the interface is
455  implementation on top of a distributed hashtable  implementation on top of a distributed hashtable
456  will be trivial.  will be trivial.
457    
458    [Benja, this might be useful for defining APIs for DHTs etc:
459    http://sahara.cs.berkeley.edu/jan2003-retreat/ravenben_api_talk.pdf
460    Full paper will appear in IPTPS 2003 -Hermanni]
461    
462  In Storm, applications are not allowed to put arbitrary  In Storm, applications are not allowed to put arbitrary
463  mappings into the index. Instead, applications that want  mappings into the index. Instead, applications that want
464  to index blocks provide the following callback  to index blocks provide the following callback
# Line 509  interface is well suited to this, or whe Line 517  interface is well suited to this, or whe
517  are able to scale to the load to store a mapping for each word  are able to scale to the load to store a mapping for each word
518  occuring in a document.  occuring in a document.
519    
520    [There are two refs about keywords in DHTs-- should we ref these ? -Hermanni]
521    
522    
523  6. Versioning  6. Versioning
524  =============  =============
# Line 601  pointer block for that pointer would the Line 611  pointer block for that pointer would the
611  In summary, the current pointer system seems promising, but  In summary, the current pointer system seems promising, but
612  there are a number of unresolved issues with it:  there are a number of unresolved issues with it:
613    
614  - signatures needed online  - signatures needed online [possible refs: ConChord, SDSI/SPKI ? -Hermanni]
615  - multiple 'current' versions annoying  - multiple 'current' versions annoying
616  - not suited to Web-like publishing  - not suited to Web-like publishing
617    
# Line 747  scalable approach and usually provides ( Line 757  scalable approach and usually provides (
757  operations (footnote about 'stable state' ?), while broadcasting can't achieve  operations (footnote about 'stable state' ?), while broadcasting can't achieve
758  either of these.  either of these.
759    
760  [footnote: It's not clear  whether all proposed DHT designs can preserve  [footnote: It's not clear whether *all* proposed DHT designs can preserve
761  (poly)logarithmic properties when participants are heterogeneous and they  (poly)logarithmic properties when participants are heterogeneous and they
762  join and leave the system in a dynamic manner.  join and leave the system in a dynamic manner.
763    

Legend:
Removed from v.1.76  
changed lines
  Added in v.1.77

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26