752 |
|
|
753 |
In a networked implementation, each peer is responsible |
In a networked implementation, each peer is responsible |
754 |
for indexing the blocks it stores. Since no peer can |
for indexing the blocks it stores. Since no peer can |
755 |
feasibly know all applications that may want indexing, |
feasibly know all applications that may need indexing, |
756 |
there may be blocks available on the network that have |
there may be blocks available on the network that have |
757 |
not been indexed by a particular application. |
not been indexed by a particular application. |
758 |
We do not see this as a problem-- it's just like a block |
We do not see this as a problem --- it's just like a block |
759 |
being available only if there's a peer which wants it to be-- |
being available only if there's a peer which wants it to be --- |
760 |
but applications must be prepared to deal with it. |
but applications must be prepared to deal with it. |
761 |
|
|
762 |
Locally, on the other hand, it is guaranteed that |
Locally, on the other hand, it is guaranteed that |
763 |
all blocks in a pool are indexed by all applications |
all blocks in a pool are indexed by all applications |
764 |
known by the pool, even if the block was added |
known by the pool. To ensure this, we check that all blocks |
765 |
to the pool in a previous session in which the pool |
are indexed when a pool is loaded, and add missing items to the index. |
|
did not know about the indexing application. To ensure this, |
|
|
we maintain for each application a table of blocks |
|
|
that have already been indexed. When the Storm pool |
|
|
implementation is initialized, it compares the list |
|
|
of indexed blocks with the list of all available blocks, |
|
|
and asks the application for unindexed blocks' items. |
|
766 |
|
|
767 |
One indexing application that may seem obvious is keyword-based |
One indexing application that may seem obvious is keyword-based |
768 |
full-text search. However, no research has been done |
full-text search. However, no research has been done |
782 |
6. Versioning |
6. Versioning |
783 |
============= |
============= |
784 |
|
|
785 |
Clearly, for block storage to be useful, there has to be a way to |
We implement mutable documents on top of block storage |
786 |
efficiently update documents/maintain different versions of documents. |
using a combination of two mechanisms. |
787 |
We achieve this by a combination of two mechanisms. Firstly, a |
Firstly, a *pointer* is an updatable reference to a block. |
|
*pointer* is an updatable reference to a block. |
|
788 |
Secondly, similar to version control systems like CVS, |
Secondly, similar to version control systems like CVS, |
789 |
we do not store each version, but only the differences between versions. |
we do not store each version, but only the differences between versions. |
790 |
|
|