148 |
usable yet. |
usable yet. |
149 |
|
|
150 |
This paper is structured as follows. In the next section, we describe |
This paper is structured as follows. In the next section, we describe |
151 |
related work. In section 3, we introduce the basic storage unit of our |
related work. In section 3, we give an overview of xanalogical model. |
152 |
system, file-like blocks identified by cryptographic hashes. |
In section 4, we introduce the basic storage unit of our |
153 |
In section 4, we discuss our implementation of Xanalogical storage |
system, i.e. file-like blocks identified by cryptographic hashes. In section 5, |
154 |
on top of the block system. In section 5, we discuss application-specific |
we discuss application-specific reverse indexing of blocks by their |
155 |
reverse indexing of blocks by their content, essential for many applications. |
content, essential for many applications. In section 6, we present |
156 |
In section 6, we present techiques for efficient versioned storage |
techiques for efficient versioned storage of mutable data on top of blocks. |
157 |
of mutable data on top of blocks. In section 7, |
In section 7, we report on implementation experience and future directions. |
|
we report on implementation experience and future directions. |
|
158 |
Section 8 concludes the paper. |
Section 8 concludes the paper. |
159 |
|
|
160 |
.. uml:: storm_layers |
.. uml:: storm_layers |
418 |
which are permanently on-line, but act as ordinary peers |
which are permanently on-line, but act as ordinary peers |
419 |
in the indexing overlay network. |
in the indexing overlay network. |
420 |
|
|
421 |
|
|
422 |
|
3. Overview of Xanalogical storage |
423 |
|
================================== |
424 |
|
|
425 |
|
In the xanalogical storage model [ref], |
426 |
|
pioneered by the unfinished Project Xanadu [ref], |
427 |
|
links are not between documents, but individual characters. |
428 |
|
When a character is first typed in, it acquires a permanent id |
429 |
|
("the character 'D' typed by Janne Kujala on 10/8/97 8:37:18"), |
430 |
|
which it retains when copied to a different document, distinguishing |
431 |
|
it from all similar characters typed in independently [#]_. |
432 |
|
A link is shown between any two documents containing the characters |
433 |
|
that the link connects. Xanalogical links are external and bidirectional. |
434 |
|
|
435 |
|
.. [#] Xanalogical storage is not limited to text. We speak about |
436 |
|
*characters* because it simplifies the explanation; picture's pixels |
437 |
|
or frames of video could be substituted. |
438 |
|
|
439 |
3. Block storage |
In addition to content links, xanalogical storage keeps an index of |
440 |
================ |
transclusions: identical characters copied into different documents. |
441 |
|
Through this mechanism, the system can show to the user all documents |
442 |
|
that share text with the current document. |
443 |
|
|
444 |
|
To keep track of links and transclusions, the system keeps a global index |
445 |
|
of documents by the characters they contain, and of links by the characters |
446 |
|
they refer to. Thus, for each character in the document, the system |
447 |
|
queries the index for other documents containing this character, |
448 |
|
and shows them as transclusions. Resolving links is a multi-step process. |
449 |
|
Each link is modeled as two collections of characters: the two |
450 |
|
endpoints of the link. To show links to a specific document, |
451 |
|
the system firstly uses the link index to find links |
452 |
|
to each character in the document. Secondly, for each link, |
453 |
|
it looks at the *other* set of characters in the link-- the target |
454 |
|
of the link, if the original character was the source, and vice versa. |
455 |
|
Thirdly, it looks for documents containing these target characters. |
456 |
|
This way, even if both the source and target characters of the link |
457 |
|
are moved to a different document, the link stays connected to them. |
458 |
|
|
459 |
|
Of course, doing any expensive operation for *every* character |
460 |
|
in a document does not scale very well. In practice, |
461 |
|
characters typed in consecutively are given consecutive ids, |
462 |
|
such as ``...:4``, ``...:5``, ``...:6`` and so on, and |
463 |
|
operations are on *spans*, i.e. consecutive ranges of characters |
464 |
|
(``...:4-6``) in a document. In Storm, in each editor session we |
465 |
|
create a block with all characters entered in this session (the content type |
466 |
|
being ``text/plain``). To designate a span of characters |
467 |
|
from that session, we use the block's id, the offset of the first |
468 |
|
character, and the number of characters in the span. |
469 |
|
This technique was first introduced in [ref ht02 paper]. |
470 |
|
|
471 |
|
In Xanadu, characters are stored to append-only *scrolls* |
472 |
|
when they are typed [ref]. Because of this, in Storm, we call the |
473 |
|
blocks containing the actual characters *scroll blocks*. The documents |
474 |
|
do not actually contain the characters; instead, they are |
475 |
|
*virtual files* containing span references as described above. |
476 |
|
To show a document, the scroll blocks it references are loaded |
477 |
|
and the characters retrieved from there [#]_. |
478 |
|
|
479 |
|
.. [#] It is unclear whether this approach is efficient for text |
480 |
|
in the Storm framework; in the future, we may try storing |
481 |
|
the characters in the documents themselves, along with their |
482 |
|
permanent identifiers. For images or video, on the other hand, |
483 |
|
it is clearly beneficial if content appearing in different |
484 |
|
documents-- or different versions of a document-- is only |
485 |
|
stored once, in a block only referred to wherever |
486 |
|
the data is transcluded. |
487 |
|
|
488 |
|
|
489 |
|
|
490 |
|
4. Storm block storage |
491 |
|
====================== |
492 |
|
|
493 |
In Storm, all data is stored |
In Storm, all data is stored |
494 |
as *blocks*, byte sequences identified by a SHA-1 |
as *blocks*, byte sequences identified by a SHA-1 |
638 |
to overcome the limitations of traditional file-based applications. |
to overcome the limitations of traditional file-based applications. |
639 |
|
|
640 |
|
|
641 |
3.1. Implementation |
4.1. Implementation |
642 |
------------------- |
------------------- |
643 |
|
|
644 |
Storm blocks are MIME messages [ref MIME], i.e., objects with |
Storm blocks are MIME messages [ref MIME], i.e., objects with |
708 |
the software (mutable documents are described in section 6.1). |
the software (mutable documents are described in section 6.1). |
709 |
|
|
710 |
|
|
|
4. Xanalogical storage |
|
|
====================== |
|
|
|
|
|
In the xanalogical storage model [ref], |
|
|
pioneered by the unfinished Project Xanadu [ref], |
|
|
links are not between documents, but individual characters. |
|
|
When a character is first typed in, it acquires a permanent id |
|
|
("the character 'D' typed by Janne Kujala on 10/8/97 8:37:18"), |
|
|
which it retains when copied to a different document, distinguishing |
|
|
it from all similar characters typed in independently [#]_. |
|
|
A link is shown between any two documents containing the characters |
|
|
that the link connects. Xanalogical links are external and bidirectional. |
|
|
|
|
|
.. [#] Xanalogical storage is not limited to text. We speak about |
|
|
*characters* because it simplifies the explanation; picture's pixels |
|
|
or frames of video could be substituted. |
|
|
|
|
|
In addition to content links, xanalogical storage keeps an index of |
|
|
transclusions: identical characters copied into different documents. |
|
|
Through this mechanism, the system can show to the user all documents |
|
|
that share text with the current document. |
|
|
|
|
|
To keep track of links and transclusions, the system keeps a global index |
|
|
of documents by the characters they contain, and of links by the characters |
|
|
they refer to. Thus, for each character in the document, the system |
|
|
queries the index for other documents containing this character, |
|
|
and shows them as transclusions. Resolving links is a multi-step process. |
|
|
Each link is modeled as two collections of characters: the two |
|
|
endpoints of the link. To show links to a specific document, |
|
|
the system firstly uses the link index to find links |
|
|
to each character in the document. Secondly, for each link, |
|
|
it looks at the *other* set of characters in the link-- the target |
|
|
of the link, if the original character was the source, and vice versa. |
|
|
Thirdly, it looks for documents containing these target characters. |
|
|
This way, even if both the source and target characters of the link |
|
|
are moved to a different document, the link stays connected to them. |
|
|
|
|
|
Of course, doing any expensive operation for *every* character |
|
|
in a document does not scale very well. In practice, |
|
|
characters typed in consecutively are given consecutive ids, |
|
|
such as ``...:4``, ``...:5``, ``...:6`` and so on, and |
|
|
operations are on *spans*, i.e. consecutive ranges of characters |
|
|
(``...:4-6``) in a document. In Storm, in each editor session we |
|
|
create a block with all characters entered in this session (the content type |
|
|
being ``text/plain``). To designate a span of characters |
|
|
from that session, we use the block's id, the offset of the first |
|
|
character, and the number of characters in the span. |
|
|
This technique was first introduced in [ref ht02 paper]. |
|
|
|
|
|
In Xanadu, characters are stored to append-only *scrolls* |
|
|
when they are typed [ref]. Because of this, in Storm, we call the |
|
|
blocks containing the actual characters *scroll blocks*. The documents |
|
|
do not actually contain the characters; instead, they are |
|
|
*virtual files* containing span references as described above. |
|
|
To show a document, the scroll blocks it references are loaded |
|
|
and the characters retrieved from there [#]_. |
|
|
|
|
|
.. [#] It is unclear whether this approach is efficient for text |
|
|
in the Storm framework; in the future, we may try storing |
|
|
the characters in the documents themselves, along with their |
|
|
permanent identifiers. For images or video, on the other hand, |
|
|
it is clearly beneficial if content appearing in different |
|
|
documents-- or different versions of a document-- is only |
|
|
stored once, in a block only referred to wherever |
|
|
the data is transcluded. |
|
|
|
|
|
Our current implementation shows only links between documents |
|
|
that are in memory at the same time [screenshot of xupdf, perhaps ref too |
|
|
(submitted) antont: was thinking the same, it would illustrate this well]. |
|
|
In the future, we will implement a global distributed index at top of |
|
|
a distributed hashtable, with the scroll blocks' ids as the keys. |
|
|
To find the transclusions of a span, the system will retrieve |
|
|
all transclusions of any span in the scroll block, then |
|
|
sort out those that do not overlap the span in question. |
|
|
|
|
|
Since the problem is to search for overlapping ranges, |
|
|
the spans cannot be used as hashtable keys. However, as the blocks |
|
|
will be relatively small (limited by the amount of text |
|
|
the user enters between two saves of a document), we hope |
|
|
that this will not be a major scalability problem. Otherwise, |
|
|
systems that allow range queries, such as skip graphs [ref] |
|
|
and skipnet [ref], may prove useful. |
|
|
|
|
|
.. [how does video work here, i.e. are they huge blocks or collections of many |
|
|
small, frames? or a sequence between two keyframes?] |
|
|
|
|
|
Benja says: Probably large, but the number of links to them |
|
|
still won't be that big, I guess. Not sure how to discuss this |
|
|
in the text; feel free to propose something :-) |
|
|
|
|
|
Toni: will try.. wondering also about who to consult :o |
|
|
|
|
|
.. [This might be relevant also: |
|
|
http://www.hpl.hp.com/techreports/2002/HPL-2002-209.pdf |
|
|
-Hermanni] |
|
|
|
|
|
One question raised by xanalogical storage is which links to show |
|
|
for a popular document that has been linked to by many users. |
|
|
We hope to address this problem by collaborative filtering |
|
|
of links [explain, ref (grouplens.org pubs?)]. There has been research on |
|
|
collaborative filtering in peer-to-peer systems |
|
|
without compromising participants' privacy [ref John Canny]. |
|
|
For some purposes simple rules based on e.g. belonging to a group may be |
|
|
applicable as well: e.g. when working on a project with a project group, it |
|
|
may be beneficial for the members of the group to see other members' |
|
|
comments of articles etc. |
|
|
|
|
|
|
|
711 |
5. Application-specific reverse indexing |
5. Application-specific reverse indexing |
712 |
======================================== |
======================================== |
713 |
|
|
1125 |
Comments may be new entities(?) linking to it |
Comments may be new entities(?) linking to it |
1126 |
|
|
1127 |
|
|
1128 |
|
|
1129 |
|
Our current implementation shows only links between documents |
1130 |
|
that are in memory at the same time [screenshot of xupdf, perhaps ref too |
1131 |
|
(submitted) antont: was thinking the same, it would illustrate this well]. |
1132 |
|
In the future, we will implement a global distributed index at top of |
1133 |
|
a distributed hashtable, with the scroll blocks' ids as the keys. |
1134 |
|
To find the transclusions of a span, the system will retrieve |
1135 |
|
all transclusions of any span in the scroll block, then |
1136 |
|
sort out those that do not overlap the span in question. |
1137 |
|
|
1138 |
|
Since the problem is to search for overlapping ranges, |
1139 |
|
the spans cannot be used as hashtable keys. However, as the blocks |
1140 |
|
will be relatively small (limited by the amount of text |
1141 |
|
the user enters between two saves of a document), we hope |
1142 |
|
that this will not be a major scalability problem. Otherwise, |
1143 |
|
systems that allow range queries, such as skip graphs [ref] |
1144 |
|
and skipnet [ref], may prove useful. |
1145 |
|
|
1146 |
|
.. [how does video work here, i.e. are they huge blocks or collections of many |
1147 |
|
small, frames? or a sequence between two keyframes?] |
1148 |
|
|
1149 |
|
Benja says: Probably large, but the number of links to them |
1150 |
|
still won't be that big, I guess. Not sure how to discuss this |
1151 |
|
in the text; feel free to propose something :-) |
1152 |
|
|
1153 |
|
Toni: will try.. wondering also about who to consult :o |
1154 |
|
|
1155 |
|
.. [This might be relevant also: |
1156 |
|
http://www.hpl.hp.com/techreports/2002/HPL-2002-209.pdf |
1157 |
|
-Hermanni] |
1158 |
|
|
1159 |
|
One question raised by xanalogical storage is which links to show |
1160 |
|
for a popular document that has been linked to by many users. |
1161 |
|
We hope to address this problem by collaborative filtering |
1162 |
|
of links [explain, ref (grouplens.org pubs?)]. There has been research on |
1163 |
|
collaborative filtering in peer-to-peer systems |
1164 |
|
without compromising participants' privacy [ref John Canny]. |
1165 |
|
For some purposes simple rules based on e.g. belonging to a group may be |
1166 |
|
applicable as well: e.g. when working on a project with a project group, it |
1167 |
|
may be beneficial for the members of the group to see other members' |
1168 |
|
comments of articles etc. |
1169 |
|
|
1170 |
|
|
1171 |
Open issue: Tree hashing, also possibly needed for (OceanStore-like?) |
Open issue: Tree hashing, also possibly needed for (OceanStore-like?) |
1172 |
distribution of shares |
distribution of shares |
1173 |
|
|