34 |
either have to include location information (as in URLs, which break |
either have to include location information (as in URLs, which break |
35 |
when documents are moved), or can only be resolved locally (as in |
when documents are moved), or can only be resolved locally (as in |
36 |
link services that can only find links stored on a select set |
link services that can only find links stored on a select set |
37 |
of link servers [ref Microcosm, DLS, ...]). Berners-Lee [name-myth]_ |
of link servers [hill94extending-andalso-carr95dls]_). Berners-Lee [name-myth]_ |
38 |
argues that unique random identifiers are not globally feasible |
argues that unique random identifiers are not globally feasible |
39 |
for this reason. |
for this reason. |
40 |
|
|
41 |
However, recent developments in peer-to-peer systems have |
However, recent developments in peer-to-peer systems have |
42 |
rendered this assumption obsolete. Structured overlay networks |
rendered this assumption obsolete. Structured overlay networks |
43 |
[ref chord, can, tapestry, pastry, kademlia, symphony, viceroy, |
[stoica01chord-andalso-ratnasamy01can-andalso-zhao01tapestry-andalso-rowston01pastry-andalso-maymounkov02kademlia-andalso-malkhi02viceroy-andalso-AspnesS2003-andalso-bonsma02swan]_ |
44 |
skip graph, swan] allow location-independent identifiers |
allow location-independent identifiers |
45 |
to be resolved on a global scale. |
to be resolved on a global scale. |
46 |
It is now feasible to do a global search to find all information |
It is now feasible to do a global search to find all information |
47 |
about a given identifier, on any peer in the network. |
about a given identifier, on any peer in the network. |
87 |
to documents stored on either one's computer [thompson01coincidence]_. |
to documents stored on either one's computer [thompson01coincidence]_. |
88 |
|
|
89 |
Advanced hypermedia systems such as Microcosm and Hyper-G |
Advanced hypermedia systems such as Microcosm and Hyper-G |
90 |
address dangling links through a notification system [ref]: |
address dangling links through a notification system |
91 |
|
[hill94extending-andalso-kappe95scalable]_: |
92 |
When a document is moved, a message is sent to servers storing links to it. |
When a document is moved, a message is sent to servers storing links to it. |
93 |
Hyper-G uses an efficient protocol for delivering such notifications |
Hyper-G uses an efficient protocol for delivering such notifications |
94 |
on the public Internet. |
on the public Internet. |
110 |
modifying them on each; when two people collaborate on a document, |
modifying them on each; when two people collaborate on a document, |
111 |
sending each other versions of the document by email; |
sending each other versions of the document by email; |
112 |
when someone downloads a document, modifies it, and publishes |
when someone downloads a document, modifies it, and publishes |
113 |
the modified version (e.g., a manual licensed under the Gnu FDL [ref]), |
the modified version (e.g., a manual licensed under the Gnu FDL [gnu-fdl]_), |
114 |
or when a group of people collaborate on a set of documents, |
or when a group of people collaborate on a set of documents, |
115 |
synchronizing irregularly with a central server (as in CVS [ref]), |
synchronizing irregularly with a central server (as in CVS [cvs]_), |
116 |
a network (as in Lotus Notes [ref]) or directly with each other |
with a network (as in Lotus Notes) or directly with each other. |
117 |
(as in Groove[?] [ref]). In each of these cases, a user should be able |
In each of these cases, a user should be able |
118 |
to work on the version at hand and then either merge it with others |
to work on the version at hand and then either merge it with others |
119 |
or fork to a different branch. |
or fork to a different branch. |
120 |
|
|
123 |
for storing and retrieving data as *blocks*, immutable |
for storing and retrieving data as *blocks*, immutable |
124 |
byte sequences identified by cryptographic content hashes |
byte sequences identified by cryptographic content hashes |
125 |
[lukka02guids]_. Additionally, Storm provides services |
[lukka02guids]_. Additionally, Storm provides services |
126 |
for versioned data and Xanalogical storage [ref]. |
for versioned data and Xanalogical storage [ted-xanalogical-structure-needed]_. |
127 |
We address the mobility of documents by block storage |
We address the mobility of documents by block storage |
128 |
and versioning, while we use Xanalogical storage |
and versioning, while we use Xanalogical storage |
129 |
to address the movement of content between documents (copy&paste). |
to address the movement of content between documents (copy&paste). |
133 |
peer-to-peer search technologies to enhance data mobility. |
peer-to-peer search technologies to enhance data mobility. |
134 |
Additionally, we hope to |
Additionally, we hope to |
135 |
provide an input to the ongoing discussion about peer-to-peer |
provide an input to the ongoing discussion about peer-to-peer |
136 |
hypermedia systems [ref ht01, ht02]. |
hypermedia systems [thompson01coincidence-andalso-bouvin02open-andalso-p2p-hypertext-panel]_. |
137 |
|
|
138 |
Currently, Storm is partially implemented as a part of the Gzz |
Currently, Storm is partially implemented as a part of the Gzz |
139 |
project [gzz]_, which uses Storm exclusively for all disk storage. |
project [gzz]_, which uses Storm exclusively for all disk storage. |
188 |
--------------------------------------- |
--------------------------------------- |
189 |
|
|
190 |
The dangling link problem has received a lot of attention |
The dangling link problem has received a lot of attention |
191 |
in hypermedia research [refs]. As examples, we examine the ways |
in hypermedia research (e.g. [davis98referential]_). As examples, we examine the ways |
192 |
in which HTTP, Microcosm [ref] and Hyper-G [ref] |
in which HTTP, Microcosm [fountain90microcosm]_ and Hyper-G [andrews95hyperg]_ |
193 |
deal with the problem. |
deal with the problem. |
194 |
|
|
195 |
In HTTP, servers are able to notify a client that a document |
In HTTP, servers are able to notify a client that a document |
196 |
has been moved, and redirect it accordingly [ref spec?]. However, |
has been moved, and redirect it accordingly [rfc2068]_. However, |
197 |
this is not required, and there are no facilities for |
this is not required, and there are no facilities for |
198 |
updating a link automatically when its target is moved. |
updating a link automatically when its target is moved. |
199 |
Consequently, broken links are a common experience for Web users. |
Consequently, broken links are a common experience for Web users. |
202 |
through *filters*, which react to arbitrary messages |
through *filters*, which react to arbitrary messages |
203 |
(such as 'find links to this anchor') generated by |
(such as 'find links to this anchor') generated by |
204 |
a client application. Filters are processes on the local system |
a client application. Filters are processes on the local system |
205 |
or on a remote host [ref distributed microcosm]. When |
or on a remote host [hill94extending]_. When |
206 |
a document is moved or deleted, a message is sent |
a document is moved or deleted, a message is sent |
207 |
to the filters. Linkbases implemented as filters can |
to the filters. Linkbases implemented as filters can |
208 |
update their links accordingly. A client selects a set |
update their links accordingly. A client selects a set |
231 |
it would be possible to find both the document and links to it, |
it would be possible to find both the document and links to it, |
232 |
no matter which peer in the network they are stored on. |
no matter which peer in the network they are stored on. |
233 |
|
|
234 |
Version control systems like CVS or RCS [ref] usually assume |
Version control systems like CVS or RCS [tichy85rcs]_ usually |
235 |
|
run on a single system or assume |
236 |
a central server hosting a repository. The WebDAV/DeltaV protocols, |
a central server hosting a repository. The WebDAV/DeltaV protocols, |
237 |
designed for interoperability between version control systems, inherit |
designed for interoperability between version control systems, inherit |
238 |
this assumption [ref]. |
this assumption [rfc2518-andalso-rfc3253]_. |
239 |
|
|
240 |
On the other hand, Arch [ref] places all repositories |
On the other hand, Arch [arch]_ places all repositories |
241 |
into a global namespace and allows independent developers |
into a global namespace and allows independent developers |
242 |
to branch and merge overlapping repositories without any central control |
to branch and merge overlapping repositories without any central control. |
|
[is there a specific ref for this?]. |
|
243 |
|
|
244 |
|
|
245 |
Peer-to-peer systems |
Peer-to-peer systems |
249 |
|
|
250 |
There are two major directions in peer-to-peer search systems, |
There are two major directions in peer-to-peer search systems, |
251 |
originally motivated by file-sharing systems like Napster. |
originally motivated by file-sharing systems like Napster. |
252 |
The more conventional one, flooding broadcasting networks [gnutella1, |
The more conventional one, flooding broadcasting networks |
253 |
kazaa, limewire, shareaza], forwards queries to all peers |
[gnutellaurl-andalso-ripeanu02mappinggnutella-andalso-kazaaurl]_, forwards queries to all peers |
254 |
reachable in a given number of hops (time-to-live). |
reachable in a given number of hops (time-to-live). |
255 |
The second, distributed hashtables (DHTs) [chord, can, tapestry, pastry, |
The second, distributed hashtables (DHTs) |
256 |
kademlia, symphony, viceroy], stores (key,value) pairs which can be found given |
[stoica01chord-andalso-ratnasamy01can-andalso-zhao01tapestry-andalso-rowston01pastry-andalso-maymounkov02kademlia-andalso-malkhi02viceroy]_ |
257 |
|
stores (key,value) pairs which can be found given |
258 |
the key; a DHT assigns each peer a subset of all possible keys, and |
the key; a DHT assigns each peer a subset of all possible keys, and |
259 |
routes queries for a given key to the peer responsible for it. |
routes queries for a given key to the peer responsible for it. |
260 |
Before a pair can be found, it must be *inserted* in the DHT |
Before a pair can be found, it must be *inserted* in the DHT |
281 |
the peer responsible for them. |
the peer responsible for them. |
282 |
|
|
283 |
A common API that can be supported by current and future DHTs |
A common API that can be supported by current and future DHTs |
284 |
is being proposed in [zhao02api]. |
is being proposed in [zhao03api]_. |
285 |
|
|
286 |
Recently, a few DHT-like systems have been developed which employ |
Recently, a few DHT-like systems have been developed which employ |
287 |
a key space similarly to a DHT, but in which queries are routed |
a key space similarly to a DHT, but in which queries are routed |
288 |
to (key,value) pairs [SWAN, skip graph]: A peer |
to (key,value) pairs [bonsma02swan-andalso-AspnesS2003]_: A peer |
289 |
occupies several positions in the key space, one for each |
occupies several positions in the key space, one for each |
290 |
(key,value) pair. In such a system, the indirection of placing |
(key,value) pair. In such a system, the indirection of placing |
291 |
close keys in the custody of a 'hashtable bucket' peer is removed |
close keys in the custody of a 'hashtable bucket' peer is removed |
481 |
representations of the block ids for file names). |
representations of the block ids for file names). |
482 |
|
|
483 |
Many existing peer-to-peer systems could be used to |
Many existing peer-to-peer systems could be used to |
484 |
find blocks on the network. |
find blocks on the network-- for example, |
485 |
For example, Freenet [freenet-ieee]_, recent Gnutella-based clients |
Freenet [freenet-ieee]_, or recent Gnutella-based programs |
486 |
(e.g. Shareaza [shareazaurl]_), and Overnet/eDonkey2000 [ref] |
like Shareaza [shareazaurl]_. |
|
also use SHA-1-based identifiers [e.g. ref: magnet uri]. |
|
487 |
Implementations on top of a DHT could use both the |
Implementations on top of a DHT could use both the |
488 |
directory and the home store approach as defined by [iyer02squirrel]_. |
directory and the home store approach as defined by [iyer02squirrel]_. |
489 |
|
|
496 |
Many practical problems have to be overcome before this |
Many practical problems have to be overcome before this |
497 |
implementation will be usable (for example seeding the |
implementation will be usable (for example seeding the |
498 |
table of known peers, and issues with UDP and network |
table of known peers, and issues with UDP and network |
499 |
address translation [ref]). |
address translation [rfc3253]_). |
500 |
|
|
501 |
Sometimes it is useful to think about *zones* blocks are in, |
Sometimes it is useful to think about *zones* blocks are in, |
502 |
related to distribution policy: for example, a *public* |
related to distribution policy: for example, a *public* |
594 |
the data is transcluded. |
the data is transcluded. |
595 |
|
|
596 |
Our current implementation shows only links between documents |
Our current implementation shows only links between documents |
597 |
that are in memory at the same time [screenshot of xupdf, perhaps ref too |
that are in memory at the same time. |
|
(submitted) antont: was thinking the same, it would illustrate this well]. |
|
598 |
In the future, we will implement a global distributed index at top of |
In the future, we will implement a global distributed index at top of |
599 |
a distributed hashtable, with the scroll blocks' ids as the keys. |
a distributed hashtable, with the scroll blocks' ids as the keys. |
600 |
To find the transclusions of a span, the system will retrieve |
To find the transclusions of a span, the system will retrieve |
612 |
One question raised by xanalogical storage is which links to show |
One question raised by xanalogical storage is which links to show |
613 |
for a popular document that has been linked to by many users. |
for a popular document that has been linked to by many users. |
614 |
We hope to address this problem by collaborative filtering |
We hope to address this problem by collaborative filtering |
615 |
of links [explain, ref (grouplens.org pubs?)]. There has been research on |
of links. There has been research on |
616 |
collaborative filtering in peer-to-peer systems |
collaborative filtering in peer-to-peer systems |
617 |
without compromising participants' privacy [ref John Canny]. |
without compromising participants' privacy [canny02collaborative-andalso-canny02factor]_. |
|
For some purposes simple rules based on e.g. belonging to a group may be |
|
|
applicable as well: e.g. when working on a project with a project group, it |
|
|
may be beneficial for the members of the group to see other members' |
|
|
comments of articles etc. |
|
618 |
|
|
619 |
|
|
620 |
Application-specific reverse indexing |
Application-specific reverse indexing |
780 |
be wikis, which are collaboratively edited by anyone |
be wikis, which are collaboratively edited by anyone |
781 |
interested [leuf01wiki]_). It is not yet clear how to do this. |
interested [leuf01wiki]_). It is not yet clear how to do this. |
782 |
Signing pointer blocks digitally may be sensible, but |
Signing pointer blocks digitally may be sensible, but |
783 |
digital signatures require a public key infrastructure |
digital signatures require a public key infrastructure, which |
|
and a trusted timestamping mechanism [#]_, which |
|
784 |
is hardly feasible for a system intended to be used |
is hardly feasible for a system intended to be used |
785 |
for off-line as well as on-line work. |
for off-line as well as on-line work. |
786 |
For long-term publishing, one-time signatures have been |
For long-term publishing, one-time signatures have been |
787 |
found useful [ref]. For the time being, the pointer mechanism |
found useful [anderson98erl]_. For the time being, the pointer mechanism |
788 |
works only in trusted Storm zones (Section 3), e.g. |
works only in trusted Storm zones (Section 3), e.g. |
789 |
in a workgroup collaborating on a set of documents. |
in a workgroup collaborating on a set of documents. |
790 |
|
|
|
.. [#] Without timestamps, digital signatures are only valid |
|
|
for a limited time [ref]. |
|
|
|
|
791 |
The ability to retain multiple 'current' versions of a document |
The ability to retain multiple 'current' versions of a document |
792 |
can be useful, for example when there is no time to consolidate |
can be useful, for example when there is no time to consolidate |
793 |
changes at the time of synchronization. |
changes at the time of synchronization. |