2 |
Storm: Supporting data mobility through location-independent identifiers |
Storm: Supporting data mobility through location-independent identifiers |
3 |
======================================================================== |
======================================================================== |
4 |
|
|
|
Abstract |
|
|
======== |
|
|
|
|
|
Dangling links and tracking of alternative versions are identified as |
|
|
problems of data mobility, which is defined as a collective term for the |
|
|
movement of documents between computers, locations on one computer and |
|
|
movement of content between documents. |
|
|
Motivations and potential solutions in existing hypermedia research are |
|
|
reviewed. The design for Storm (for storage module) is |
|
|
presented, addressing the problems of data mobility by utilising |
|
|
location-independent globally unique identifiers, immutable block storage |
|
|
and peer-to-peer networking for hypermedia. |
|
|
The design and a partial implementation of it is evaluated in |
|
|
light of use cases from scenarios, that illustrate the benefits of storm for |
|
|
e.g. mobile knowledge workers. [mention some of the additional benefits?] |
|
|
Open issues are discussed, and possibilities for future work in areas such as interoperability, |
|
|
scalability and security are identified. |
|
|
|
|
|
Keywords: hypermedia, P2P, peer-to-peer, location-independent identifiers, |
|
|
versioned hypermedia, xanalogical storage, distributed hashtables |
|
|
|
|
|
|
|
5 |
1. Introduction |
1. Introduction |
6 |
=============== |
=============== |
7 |
|
|
123 |
on top of the block system. In section 5, we discuss application-specific |
on top of the block system. In section 5, we discuss application-specific |
124 |
reverse indexing of blocks by their content, essential for many applications. |
reverse indexing of blocks by their content, essential for many applications. |
125 |
In section 6, we present techiques for efficient versioned storage |
In section 6, we present techiques for efficient versioned storage |
126 |
of mutable data on top of blocks. In section 7, |
of mutable data on top of blocks. Section 7 concludes the paper. |
|
we report on implementation experience and future directions. |
|
|
Section 8 concludes the paper. |
|
127 |
|
|
128 |
.. uml:: storm_layers |
.. uml:: storm_layers_thesis |
129 |
|
|
130 |
package Blocks |
package Blocks |
131 |
|
|
830 |
indexing (Section 5): Keeping an index of all pointer blocks |
indexing (Section 5): Keeping an index of all pointer blocks |
831 |
about pointer ``P``, we can quickly find out the current one. |
about pointer ``P``, we can quickly find out the current one. |
832 |
|
|
833 |
.. uml:: storm_pointers |
.. uml:: storm_pointers_thesis |
834 |
|
|
835 |
class Pointer |
class Pointer |
836 |
|
|
1020 |
provide a callback interface for reading and writing versions |
provide a callback interface for reading and writing versions |
1021 |
and computing and applying differences. |
and computing and applying differences. |
1022 |
|
|
1023 |
.. uml:: version_interfaces |
.. uml:: version_interfaces_thesis |
1024 |
|
|
1025 |
class Version "interface" |
class Version "interface" |
1026 |
methods |
methods |
1073 |
would have to be sent through the network. |
would have to be sent through the network. |
1074 |
|
|
1075 |
|
|
1076 |
7. Experience and future directions |
7. Conclusions |
|
=================================== |
|
|
|
|
|
Review of the use cases: what does storm in each? -- |
|
|
|
|
|
(where&how to put this, if use cases are used like this at all? |
|
|
agreed with benja on irc that we might try this approach - hinting about the |
|
|
use cases early on and taking a detailed look later, in light of the design.) |
|
|
|
|
|
1. live meeting: |
|
|
Documents have been assigned block-ids (and urns?) at creation. |
|
|
Pools that are shared / set visible for the other user show over the |
|
|
network. (...) |
|
|
|
|
|
2. syncing via a remote connection after working separately: |
|
|
Both new versions are actually diffs to the original. Merging is |
|
|
application-specific. |
|
|
|
|
|
3. publising on/from the laptop: |
|
|
B sets the document public (how that is done depends on UI implementation), |
|
|
i.e. putting in a published pool, which may reside locally or externally |
|
|
e.g. on a (dedicated) server that is "always-on". These public/private zones |
|
|
are an area where future research is needed, possibly related to rights and |
|
|
permissions etc. too. |
|
|
|
|
|
4. A and other people viewing, commenting, bookmarking, linking: |
|
|
When someone views the document, the system retrieves the datablock. |
|
|
Comments may be new entities(?) linking to it, editing would create a new |
|
|
version as a diff as a new datablock with a new guid. Bookmarks and links to |
|
|
the original are made to the ID of the original, with no trace of the |
|
|
location (originally B's laptop). (more info about pointers here, and in the |
|
|
next one?) |
|
|
|
|
|
5. When B takes her laptop off-line, the searches for the ID of her (and |
|
|
A's) Document simply result pointing at the other copies (e.g. on A's |
|
|
machine). |
|
|
|
|
|
6. Increasing traffic: The peer-to-peer network can automatically share the |
|
|
load, as popular items get replicated and further delivered from different |
|
|
hosts. (does storm actually do this / is it aimed at this?) |
|
|
|
|
|
7. When B reconnects, can check comments to the Document etc? How does that |
|
|
happen? Index? |
|
|
|
|
|
|
|
|
Open issue: Tree hashing, also possibly needed for (OceanStore-like?) |
|
|
distribution of shares |
|
|
|
|
|
Open issue: We want to be able |
|
|
to copy the blocks from the NYT publication pool to our local computer |
|
|
and verify offline that it really comes from NYT. How to do this will go |
|
|
into the article as an open research question ;-) |
|
|
|
|
|
Future directions: of course, we should implement a prototype |
|
|
|
|
|
Open issue/Future directions: implement multisource downloading |
|
|
|
|
|
Future directions: Implement home node model or directory model ? |
|
|
|
|
|
Open issue: Firewall Transparency/NATs ? |
|
|
|
|
|
Open issue: identification of online entities ? |
|
|
|
|
|
Open issue: how to address interoperability - what do we mean with it? |
|
|
One thing might be to look at the OHS OHProtocol and such. |
|
|
Another is to investigate Web publishing with using storm for zope? |
|
|
interestingly zope by default stores diffs in the zodb (similarly:cvs&storm?) |
|
|
Benja also noted about pointing to external data? (couldn't find the post |
|
|
from the archives) |
|
|
|
|
|
Open issue: notes in the text about block sizes with video |
|
|
|
|
|
Open issue: other related systems (within gzz?) for e.g. more synchronous |
|
|
collabaration (irc?)? |
|
|
|
|
|
.. moved the following here from intro, as suggested by (b&) Hermanni |
|
|
|
|
|
No work on integrating Storm with current programs (in the spirit of Open |
|
|
Hypermedia) has been done so far. It is not clear how far this is possible |
|
|
without changing applications substantially, if advantage of our |
|
|
implementation of Xanalogical storage is to be taken. (Vitali [ref] notes |
|
|
that Xanalogical storage necessiates strong discipline in version tracking, |
|
|
which current systems lack.) An idea for future work in this area is |
|
|
investigating potential combinations of Gzz/Storm and the Object-Oriented |
|
|
Web publishing environment Zope [ref zope.org/com, book, article?]. For |
|
|
example, Storm might be used as a storage module for Zope (instead or |
|
|
combined with the objectbase ZODB).(?) The benefits Storm would provide |
|
|
there, and how e.g. peer-to-peer based Zope would then work, are yet unknown |
|
|
for the authors. (feel free to know, though :) Similarly(?) to Storm, Zope |
|
|
also stores saves as diffs to the database. (haven't looked what zodb |
|
|
identifiers are like. what else should?). The fact that Zope is open source |
|
|
might allow for the (even substantial) changes that 'xanalogicalising' it |
|
|
would mean in practise. For Gzz, Zope might provide a window to the Web. |
|
|
|
|
|
.. Investigating the degrees of interoperability with Open Hypermedia |
|
|
(/Structural Computing?) and (more primitive?) other (Web) XXX is a possible |
|
|
direction for future work, and will be preliminarly discussed here too. |
|
|
[benja says: Really? Do we/can we discuss that really? antont: 'll try.] |
|
|
|
|
|
(Also about OHProtocol? will check) |
|
|
|
|
|
When Xanalogical storage is not applied, using Storm as a |
|
|
replacement/equivalent of a conventional file and versioning system is |
|
|
trivial? |
|
|
|
|
|
.. p2p -> cf half-life of peers (Mojo Nation): Is it desirable that 'weak' peers |
|
|
participate in a DHT? -- In Circle, peers must have been online |
|
|
for at least an hour... In which ways, then, can 'weak' peers contribute |
|
|
to the network in a p2p fashion? Caching is certainly one central |
|
|
way, esp. when combined with multisource downloading (this can |
|
|
potentially boost download speeds to the full available bandwidth). |
|
|
This is a performance/reliability issue rather than something |
|
|
changing the fundamental qualities of the network, but still important. |
|
|
|
|
|
The important point about p2p publishing is that no account and setup |
|
|
is necessary to start publishing. |
|
|
|
|
|
One possibility: Use IBP for limited-time publishing, referring to |
|
|
the location through the DHT? This might be related to p2p publishing. |
|
|
|
|
|
|
|
|
8. Conclusions |
|
1077 |
============== |
============== |
1078 |
|
|
1079 |
We have presented the Storm design, which makes use of recent advances |
We have presented the Storm design, which makes use of recent advances |
1103 |
location-dependent identifiers at all. We hope to raise awareness |
location-dependent identifiers at all. We hope to raise awareness |
1104 |
for the prospects of location-independent systems based |
for the prospects of location-independent systems based |
1105 |
on structured overlay networks like DHTs. |
on structured overlay networks like DHTs. |
|
|
|
|
|
|
|
9. Acknowledgements |
|
|
==================== |
|
|
|
|
|
We would like to thank Sarah Stehlig for discussions. |
|
|
|
|
|
.. (Add others here. Sarah pointed out the privacy issues with |
|
|
exposing one's cache-- duh, but I for one hadn't thought |
|
|
about it! And the Squirrel paper doesn't mention it either. |
|
|
Probably it's pointed out somewhere in the literature-- |
|
|
too basic a problem not to have been noticed-- but still... -b) |
|
|
|
|
|
Erno Kuusela for comments (will check those from SCRATCH and ask him to read |
|
|
a current version also) |
|
|
|
|
|
Harri Oinas-Kukkonen, observations (overall, about benefits, presentation) |
|