/[gzz]/manuscripts/storm/short-paper.rst
ViewVC logotype

Diff of /manuscripts/storm/short-paper.rst

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.3 by benja, Thu May 22 20:31:14 2003 UTC revision 1.4 by benja, Thu May 22 20:47:48 2003 UTC
# Line 72  research with regard to hypermedia. Line 72  research with regard to hypermedia.
72  Storm block storage  Storm block storage
73  ===================  ===================
74    
75    .. as blocks, independent of network location:
76    
77  In Storm, all data is stored  In Storm, all data is stored
78  as *blocks*, immutable byte sequences identified by a SHA-1  as *blocks*, immutable byte sequences identified by a SHA-1
79  cryptographic content hash [fips-sha1]_.  cryptographic content hash [fips-sha1]_.
80  Purely a function of a block's content, block ids  Purely a function of a block's content, block ids
81  are completely independent of network location.  are completely independent of network location.
82  Blocks have a similar or somewhat finer granularity  
83  as regular files, but they are immutable, since any change to the  .. similar to files, but immutable:
84  byte sequence would change the hash (and thus create a different block).  
85  Mutable data structures are built on top of the immutable blocks  Blocks are similar to files, but they cannot be modified.
86  (see Section 6).  Any change in the data would cause the identifier to change too.
87    
88  Storing data in immutable blocks may seem strange at first, but  .. identifiers self-certifying:
89  has a number of advantages. First of all, it makes identifiers  
90  self-certifying: no matter where we have downloaded a block from,  Storing data in immutable blocks
91  we are able to check we have the correct data by checking  has a number of advantages. Firstly, it makes identifiers
92  the cryptographic hash in the identifier. Therefore, we can  self-certifying.
93  safely download blocks from an untrusted peer.  
94    After downloading a block, we are can check whether the data
95    matches the cryptographic hash in the identifier.
96    Therefore, we can safely download blocks from an untrusted peer.
97    
98    .. link targets cannot be changed on us:
99    
100  When we make a reference to a block, we can be sure  When we make a reference to a block, we can be sure
101  that even the original author of the target will not be able  that even the original author of the target will not be able
# Line 97  For example, if a newspaper refers to a Line 104  For example, if a newspaper refers to a
104  to the editor this way, the letter's sender won't be able to change  to the editor this way, the letter's sender won't be able to change
105  the reference into an advertisement for a pornographic web page.  the reference into an advertisement for a pornographic web page.
106    
107    .. caching trivial:
108    
109  Secondly, caching becomes trivial, since it is  Secondly, caching becomes trivial, since it is
110  never necessary to check for new versions of blocks. It is easy  never necessary to check for new versions of blocks.
111  to replicate data between systems: A replica of a block never  
112  needs to be updated; cached copies can be kept as long as desired.  .. flash crowds alleviated:
113    
114  If peers make the blocks in their caches available on the network,  If peers make the blocks in their caches available on the network,
115  the flash crowd problem could be alleviated: The more users  the flash crowd problem could be alleviated: The more users
# Line 111  from *any* peer since the source does no Line 120  from *any* peer since the source does no
120  On the other hand, there are privacy  On the other hand, there are privacy
121  concerns with exposing one's cache to the outside world.  concerns with exposing one's cache to the outside world.
122    
123    .. replication easy:
124    
125  To replicate all data from computer A  To replicate all data from computer A
126  on computer B, it suffices to copy all blocks from A to B that B  on computer B, it suffices to copy all blocks from A to B that B
127  does not already store. This can be done through a simple 'copy'  does not already store. This can be done through a simple 'copy'
128  command. Different versions of a single document  command. Different versions of a single document
129  can coexist on the same system without naming conflicts, since  can coexist on the same system without naming conflicts, since
130  each version will be stored in its own block with its own id.  each version will be stored in its own block with its own id.
131  In contrast, a system based on mutable resources  
132  has to use more advanced schemes, for example merging the changes  .. web links resolvable to local copies:
 done to a document at A or B. (Merging is still necessary  
 when a user wants to incorporate a set of changes, but not  
 required at replication time.)  
   
 .. On the other hand for instance, several popular  
    database management systems (e.g. Lotus Notes [ref]) have complex  
    replication schemes, which may led awkward replication conflicts,  
    because of they lack the immutable properties of data.  
    [Or does this belong to diff section ? -Hermanni]  
133    
134  The same namespace is used for local data and data  The same namespace is used for local data and data
135  retrieved from the network. When an online document has been  retrieved from the network. When an online document has been
# Line 137  for offline browsing, for example in mob Line 139  for offline browsing, for example in mob
139  After a block has been downloaded, references to it will *never*  After a block has been downloaded, references to it will *never*
140  cease to work, online or offline.  cease to work, online or offline.
141    
142  .. tried to think this a bit, as 'never' is always a strong word.  .. append-only, bugs don't lose old data:
    can it be that data in the cache is 'out-of-sync', i.e.  
    there are different versions so that links indeed break for  
    off-line browsing? e.g. when there is document A linking  
    to document B, both v1.0, and the user downloads them. then,  
    both are updated on the server, to versions 1.1. user dowloads  
    A, goes off-line, and tries to follow the link to B(1.1?). ?  
    i know this is not what is meant above, but yet an example of how  
    evil reviewers might react to the strong notion 'never' there ;o  
    or is it so that the link from A to B is not to a particular version,  
    but to any B, so that in that case the user would get B1.0 and be happy?  
    (or confused, if diff from A,B1.0->1.1 was so major that how A1.1 links to  
    B1.1 does not make any sense when the user ends up in 1.0 instead?)  
    -- antont  
   
 .. Changed to read 'block.' That blocks are immutable should be clear  
    at this point ;-) -b  
143    
144  Thirdly, immutable blocks increase *reliability*.  Thirdly, immutable blocks increase *reliability*.
145  When saving a document, an application will only *add* blocks,  When saving a document, an application will only *add* blocks,
# Line 171  Microsoft Word many times. Line 157  Microsoft Word many times.
157     makes matters a little more complicated; still,     makes matters a little more complicated; still,
158     the basic assertion holds.     the basic assertion holds.
159    
160  Even when a publisher's server fails to serve a block,  .. mirrors trivial:
161  links to it will work until *no* other peer  
162    Links to a block will work as long as *any* peer
163  holds a copy. Thus, providing mirrors is trivial.  holds a copy. Thus, providing mirrors is trivial.
164  Even after failure of all of the publisher's mirrors,  Even after failure of all dedicated mirrors,
165  a document may still be available from peers that have  a document may still be available from peers that have
166  downloaded it. An archive of published blocks, in the spirit  downloaded it. An archive of published blocks, in the spirit
167  of the Web archive [waybackmachine]_, would only be yet another backup:  of the Web archive [waybackmachine]_, would only be yet another backup:
168  normal links to a block would work as long as the archive  normal links to a block would work as long as the archive
169  holds a copy. It would also be hard to purposefully remove  holds a copy.
170  a published document from the network; whether this is  
171  a good or a bad property we leave for the reader to judge.  .. more durable:
   
 .. XXX When there are problems with network connectivity to a central  
    repository, peers can still work with each other.  
    According to Murphy's law, problems always occur on tight deadlines  
    (our project is empirical proof :-) )  
172    
173  Finally, because blocks are easy to move from system  Finally, because blocks are easy to move from system
174  to system, we hope that block storage will be more *durable* than files.  to system, we hope that block storage will be more *durable* than files.
# Line 203  to keep them in. By making it easy to co Line 185  to keep them in. By making it easy to co
185  blocks produced on a diverse number of systems, it would be easier  blocks produced on a diverse number of systems, it would be easier
186  to keep old data around.  to keep old data around.
187    
188    .. persistency commitment:
189    
190  Of course, to meet this goal it is necessary that the block  Of course, to meet this goal it is necessary that the block
191  system remains backwards compatible at all times. We have therefore  system remains backwards compatible at all times. We have therefore
192  decided to enter a *persistency commitment* when we finalize  decided to enter a *persistency commitment* when we finalize
# Line 212  to handle any block created according to Line 196  to handle any block created according to
196  This means that no matter how much we'll regret our current choices  This means that no matter how much we'll regret our current choices
197  in the future, we commit to providing backward compatibility for them.  in the future, we commit to providing backward compatibility for them.
198    
199    .. incompatibility with existing systems:
200    
201  The advantages we have outlined are bought by an utter incompatibility with  The advantages we have outlined are bought by an utter incompatibility with
202  the dominant paradigms of file names and URLs. We hope that  the dominant paradigms of file names and URLs. We hope that
203  it would be possible to port existing applications to use Storm  it would be possible to port existing applications to use Storm

Legend:
Removed from v.1.3  
changed lines
  Added in v.1.4

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26