302 |
Primitive block storage abstraction |
Primitive block storage abstraction |
303 |
=================================== |
=================================== |
304 |
|
|
305 |
In this section, we introduce a simple data model that |
In this section, we introduce the Storm [#]_ model, |
306 |
|
a simple but powerful data model that can represent |
307 |
|
all data on the P2P Web and that maps well to |
308 |
explicitly state the capabilities |
the functionality of current P2P filesharing systems. |
309 |
that our pointer scheme requires from the underlying system. |
|
310 |
|
.. [#] For STORage Module. Storm is the name of our |
311 |
|
Free Software implementation, discussed in |
312 |
Add permanent block |
Section XXX. |
313 |
Given a bit string and a MIME type, |
|
314 |
adds the block into |
The Storm model is comprised of the following components: |
315 |
the system and return a *permanent id* for |
|
316 |
that exact bitstring, |
Blocks, identified by hash |
317 |
for example a cryptographic hash of the data. |
All data is stored in *blocks*, which are file-like, |
318 |
|
immutable byte sequences. Blocks have global identifiers |
319 |
Get the permanent data for a given id |
composed of a cryptographic hash of their content |
320 |
Given a permanent id, locate and return the bitstring |
plus a MIME content type. (The MIME type is the minimal amount |
321 |
stored with that permanent id, or inform the caller |
of metadata necessary in addition to the content |
322 |
that the data was not currently available. |
for showing a block in a Web browser.) |
323 |
|
|
324 |
Global reverse indices |
Pools |
325 |
The ability to define (on a system-wide basis) |
A *pool* is a collection of blocks. The data published |
326 |
partial functions of blocks (of the data in the bitstrings) |
by a particular P2P Web server, the data downloaded |
327 |
and to query for the ids |
by a client, the data in a client's cache, and all the |
328 |
(and thus the data) |
data published on the whole Web can all be seen as a pool. |
329 |
of the blocks with the given values for the functions. |
In an API, a pool provides the functionality to |
330 |
|
fetch a block by its global id; local pools also allow |
331 |
For scalability, the ability to also define another |
new blocks to be added. |
332 |
function, which takes a set of blocks and determines which |
|
333 |
of the blocks are "relevant" locally on each peer can yield |
Reverse indices |
334 |
great improvements. |
Reverse indices are application-specific plug-ins |
335 |
|
which index blocks by properties of their content |
336 |
Managing where the blocks are stored is excluded |
rather than by hash. Reverse indices examine each |
337 |
from the scope of this abstraction. The versioning system |
block in a local pool and return hashtable keys |
338 |
must work as best it can even if only a subset of the |
under which this block should be found. |
339 |
blocks is available. |
|
340 |
|
For example, an index of Ogg Vorbis audio blocks running |
341 |
|
on an artist's Web server could return each word |
342 |
|
occuring in the album and title fields of each song. |
343 |
|
The server would publish the resulting mappings |
344 |
|
from words to block ids on the P2P network. |
345 |
|
|
346 |
|
An index can also return additional information |
347 |
|
to be placed in the hashtable along with the |
348 |
|
block id. In the example, this could be the complete |
349 |
|
metadata of the song, so that a client does not need |
350 |
|
to download the complete audio file before being able |
351 |
|
to show its artist, title etc. to its user. |
352 |
|
|
353 |
|
The pointer functionality is trivial to implement |
354 |
|
*on top of* this basic model. A pointer record |
355 |
|
is simply a special kind of block. A pointer record index |
356 |
|
is used to publish mappings from pointer ids |
357 |
|
to the block ids of pointer records. |
358 |
|
|
359 |
The benefits of starting from this basic model are numerous: |
The benefits of starting from this basic model are numerous: |
360 |
|
|
365 |
|
|
366 |
- robust - all state of the system is in the blocks |
- robust - all state of the system is in the blocks |
367 |
|
|
|
In this framework, we shall define a system of *pointers*: |
|
|
some types of identifiers that refer to a block but whose |
|
|
referent may be changed. In the following Sections, we show |
|
|
how a reasonably efficient system can be constructed |
|
|
using only the primitive datablock model given above. |
|
|
|
|
|
.. - Basic model of underlying system: Only blocks |
|
|
- Benefits of that, in a nutshell |
|
|
- Reverse indexing (nicely provided through structured overlays), |
|
|
*define* reverse indexing |
|
|
- Def.: Pointers are abstract references that *point* |
|
|
to different blocks (versions) over time |
|
|
- Storm is designed to be modular; different routing algorithms |
|
|
can be used with Storm for locating data with loc.ind. ids ( |
|
|
(e.g., PAST and CFS are tightly binded to their routing |
|
|
algorithm) |
|
|
|
|
368 |
|
|
369 |
Applications |
Applications |
370 |
============ |
============ |
425 |
pointer records could be used to implement |
pointer records could be used to implement |
426 |
a p2p version of software upgrades |
a p2p version of software upgrades |
427 |
such as Debian's ``apt`` or Windows update, |
such as Debian's ``apt`` or Windows update, |
428 |
allowing large bandwidth savings. |
allowing for large bandwidth savings because |
429 |
|
updates could be downloaded from the closest |
430 |
|
host that has a copy. |
431 |
|
|
432 |
An important advantage of pointer records, due |
An important advantage of pointer records, due |
433 |
to the simple underlying block abstraction, is |
to the simple underlying block abstraction, is |