15 |
|
|
16 |
.. Affect-PEGs: |
.. Affect-PEGs: |
17 |
|
|
18 |
- this document describes the usage of Tapestry with Storm |
This document describes the use of Tapestry overlay with Storm. We start by |
19 |
- possibly some architectural changes are required to Storm if Tapestry is |
proviving a short introduction to Tapestry, to the SEDA model and to the Tapestry |
20 |
taken to in use. Thus, this document is needed |
API. Then, we propose new classes and a pseudo code to be included into Storm |
21 |
- first, this PEG discusses general aspects of Tapestry, it's API and the |
code base. Pseudo code describes a simple StormTapestryManager which supports |
22 |
usage of the API with other applications |
query and insert operations in a Tapestry overlay. |
23 |
- introduction to Tapestry, the SEDA model, Tapestry API, how to use Tapestry |
|
24 |
with other application |
This document is based on Tapestry 2.0 release. |
|
- then, a proposal of usage with Storm |
|
|
- pseudo code: search and insert operations |
|
|
- simple "implementation" and architectural design |
|
|
- possible changes are listed |
|
|
- this document is based on Tapestry release 2.0 |
|
25 |
|
|
26 |
Issues |
Issues |
27 |
====== |
====== |
32 |
Introduction to Tapestry |
Introduction to Tapestry |
33 |
======================== |
======================== |
34 |
|
|
35 |
- currently Tapestry_ implements the DOLR abstracion |
Tapestry_ is a application level Peer-to-Peer overlay. Currently Tapestry_ |
36 |
- other abstraction can be implemented also (e.g., DHT) |
supports the DOLR abstracion but other abstraction can be implemented |
37 |
- two primary functions: 1) efficient way for nodes to communicate across |
(e.g., DHT). Tapestry's DOLR interface has two primary functions: efficient |
38 |
an overlay and 2) effcient object location |
way for nodes to communicate across the overlay and effcient object location. |
39 |
- objects are *published* with object to location mappings (pointers) |
In Tapestry, objects are *published* with object to location mappings (a.k.a. |
40 |
- differs from DHTs in that always use the closest available object |
pointers). This operation differs from DHTs in that always use the closest |
41 |
- routing table consists of levels of neighbor links |
available object. |
42 |
- each level represents a matching prefix up to a digit position in the ID |
|
43 |
- for each level, neighbor links are selected based on "closest" in network |
Tapestry's routing table consists of levels of neighbor links. Each level |
44 |
latency |
represents a matching prefix up to a digit position in the ID. Additionally |
45 |
- along the publication process, the overlay deposits a pointer at every hop |
for each level, neighbor links are selected based on closest in network |
46 |
along the way |
*latency*. Along the publication process, the overlay deposits a pointer at |
47 |
- when an object is searched, a query is forwarded towards object's |
every hop along the way. When an object is searched, a query is forwarded |
48 |
publication route, and when a query run into a first pointer, overlay |
towards object's publication route, and when a query run into a first pointer, |
49 |
directly forwards to the object |
overlay directly forwards to the object. |
50 |
- written in Java (currently JDK 1.3) |
|
51 |
- uses SEDA (Staged Event-Driven Architecture) framwork |
Tapestry 2.0 release is written in Java and uses SEDA (Staged Event-Driven |
52 |
- approximately 57000 lines of code |
Architecture) framwork for thread and I/O-operations. Tapestry's code base |
53 |
- shares codebase with Oceanstore |
has approximately 57000 lines of code and shares some code with Oceanstore's |
54 |
|
code base. |
55 |
|
|
56 |
Overview of SEDA |
Overview of SEDA |
57 |
================ |
================ |
58 |
|
|
59 |
- asynchronous I/O library (similar to JDK 1.4's java.nio package) |
SEDA is a asynchronous I/O library (similar to JDK 1.4's java.nio package). In |
60 |
- stages communicate with each other by sending events |
SEDA there are stages and they communicate with each other by sending events. |
61 |
- each Tapestry component is implemented as a stage |
A stage runs as a thread which starts by executing initialization routines and |
62 |
- stage runs as a thread which starts by executing initialization routines |
then enters an event loop. Inside each JVM, a dispatcher monitors all messages |
63 |
and then enters an event loop |
and events, and delivers copies to each stage that subscribes to messages of |
64 |
- inside each JVM, a dispatcher monitors all messages and events, |
that type. |
65 |
and delivers copies to each stage that subscribes to messages of that type |
|
66 |
- a (common) Tapestry node uses StaticTClient stage, DynamicTClient stage, |
Each Tapestry component is implemented as a stage. A (common) Tapestry node |
67 |
Router stage and Tapestry application stage |
uses StaticTClient stage, DynamicTClient stage, Router stage and Tapestry |
68 |
- for each stage, a config file is required to specify stage's properties |
application stage. For each stage, a config file is required to specify |
69 |
(XML-like tag structure) |
stage's properties (XML-like tag structure). During the initialization phase, |
70 |
- during the initialization phase, a stage specifies which event and messages |
a stage specifies which event and messages it wants to "listen" to. For Storm, |
71 |
it wants to "listen" to |
the Tapestry application stage is the most important. |
|
- for Storm, the Tapestry application stage is the most important |
|
72 |
|
|
73 |
|
|
74 |
Overview of Tapestry API |
Overview of Tapestry API |
95 |
Using Tapestry API from other applications |
Using Tapestry API from other applications |
96 |
========================================== |
========================================== |
97 |
|
|
|
- confgig file for defining a Storm stage (Tapestry application stage) |
|
|
|
|
98 |
According to the `Tapestry programmer's guide`_, there three steps to write |
According to the `Tapestry programmer's guide`_, there three steps to write |
99 |
a Tapestry application: |
a Tapestry application: |
100 |
|
|
101 |
- Write the necessary messages to interface with Tapestry |
- Write the necessary messages to interface with Tapestry |
102 |
- Write an Eventhandler class that would serve as the application |
- Write an event handler class that would serve as the application |
103 |
- Write configuration file(s) to define stages and specify initialization |
- Write configuration file(s) to define stages and specify initialization |
104 |
arguments |
arguments |
105 |
|
|
106 |
Interface with Tapestry |
Custom messages for Tapestry |
107 |
----------------------- |
----------------------- |
108 |
|
|
109 |
|
Again, According to the `Tapestry programmer's guide`_: |
110 |
|
|
111 |
1. An application must extend extending the abstract API message types, e.g.: |
1. An application must extend extending the abstract API message types, e.g.: |
112 |
|
|
113 |
public class StormMsg extends TapestryPublishmsg { |
public class StormMsg extends TapestryPublishmsg { |
120 |
|
|
121 |
3. Do not write a working type_code method |
3. Do not write a working type_code method |
122 |
|
|
123 |
4. Register your message class |
|
124 |
|
An event handler class |
|
Eventhandler class |
|
125 |
------------------ |
------------------ |
126 |
|
|
127 |
1. Write a class which extends SEDA's existing stages, e.g., |
1. Write a eventhandler class which extends SEDA's existing stages, e.g., |
128 |
|
|
129 |
public class StormP2p extends StandardStage |
public class StormTapestryManager implements EventHandlerIF |
130 |
|
|
131 |
2. In Eventhandler class, implement a hanleEvent method which will |
2. In a eventhandler class, implement a hanleEvent method which will |
132 |
process registered messages |
process registered messages |
133 |
|
|
134 |
3. In Eventhandler class, implement a init method which will define |
3. In a eventhandler class, implement a init method which will define |
135 |
the messages that are "listened" for this class |
the events and messages that are "listened" for this class |
136 |
|
|
137 |
|
4. Register your custom message classes |
138 |
|
|
139 |
Using Tapestry with Storm |
Using Tapestry with Storm |
140 |
========================= |
========================= |
141 |
|
|
142 |
- in this section, we will outline the design which would allow Storm to |
In this section, we will outline the design which would allow Storm to |
143 |
Tapestry's routing services |
Tapestry's routing services. |
|
|
|
|
|
|
|
|
|
|
- For finding Storm blocks in a Tapestry overlay, we must create: |
|
144 |
|
|
145 |
|
For finding Storm blocks in a Tapestry overlay, we must create custom message |
146 |
|
classes: |
147 |
|
|
148 |
- BlockIDTag class for Storm blocks' IDs (implements |
- BlockIDTag class for Storm blocks' IDs (implements |
149 |
ostore.tapestry.api.TapestryTag) for storing query specific information: |
ostore.tapestry.api.TapestryTag) for storing query specific information: |
174 |
implements ostore.util.QuickSerializable) to carry a requested Storm |
implements ostore.util.QuickSerializable) to carry a requested Storm |
175 |
block being sent to a original requester |
block being sent to a original requester |
176 |
|
|
177 |
- For interacting Storm's storage model with Tapestry: |
For interacting Storm's storage model with Tapestry we must create and event |
178 |
|
handler class: |
179 |
|
|
180 |
- StormTapestryManager class which |
- StormTapestryManager class (implements sandStorm.api.EventHandlerIF) for |
181 |
- registers Storm to handle Storm-related Tapestry events and messages |
listening standard Tapestry events, registering custom Storm messages, |
182 |
- handles and dispatches Storm-related Tapestry events and messages |
dispatching custom Storm messages and interacting with a local Storm pool |
|
- interacts with a local Storm pool |
|
183 |
|
|
184 |
|
|
185 |
PSEUDOCODE for Storm |
Here, we propose a simple StormTapestryManager pseudo code. This is an event |
186 |
|
handler class for Tapestry. In addition, this class provides Storm block |
187 |
|
query and insert operations in a Tapestry overlay. |
188 |
|
|
189 |
class StormTapestryManager implements EventHandlerIF { |
class StormTapestryManager implements EventHandlerIF { |
190 |
|
|
191 |
// dispatcher which dispatch all items |
// dispatcher which dispatch all items |
192 |
// to target peer |
// to target peer (Mandatory: sandStorm requires this) |
193 |
method dispatch (QueueElementIF item) { |
method dispatch (QueueElementIF item) { |
194 |
//try do dispatch an item |
//try do dispatch an item |
195 |
try { |
try { |
202 |
// general initialization method |
// general initialization method |
203 |
// that handles subscribes this stage |
// that handles subscribes this stage |
204 |
// to listen certain events and messages |
// to listen certain events and messages |
205 |
|
// (Mandatory: sandStorm requires this) |
206 |
method init (ConfigDataIF config) { |
method init (ConfigDataIF config) { |
207 |
|
|
208 |
// find our NodeId |
// find our NodeId |
218 |
// create peer ID based on peer's public key |
// create peer ID based on peer's public key |
219 |
_peer = new SHA1Hash (pkey); |
_peer = new SHA1Hash (pkey); |
220 |
|
|
221 |
// the *events* which we want to listen |
// the mandatory *events* which we want to listen |
222 |
|
|
223 |
array event_types = { |
array event_types = { |
224 |
"seda.sandStorm.api.StagesInitializedSignal", |
"seda.sandStorm.api.StagesInitializedSignal", |
230 |
classifier.subscribe (event_type[i]) |
classifier.subscribe (event_type[i]) |
231 |
} |
} |
232 |
|
|
233 |
// the *messages* we want receive through Tapestry |
// the mandatory *messages* we want receive through Tapestry |
234 |
|
|
235 |
array message_types = { |
array message_types = { |
236 |
"org.nongnu.storm.p2p.tapestry.StormErrorMsg", |
"org.nongnu.storm.p2p.tapestry.StormErrorMsg", |
259 |
} |
} |
260 |
|
|
261 |
// handles *all* events which stage is listening |
// handles *all* events which stage is listening |
262 |
|
// (Mandatory: sandStorm requires this) |
263 |
method handleEvents(QueueElementIF array items) |
method handleEvents(QueueElementIF array items) |
264 |
throws EventHandlerException { |
throws EventHandlerException { |
265 |
|
|
269 |
|
|
270 |
// handles single event for this stage. handleEvents |
// handles single event for this stage. handleEvents |
271 |
// method gives an item this method |
// method gives an item this method |
272 |
|
// (Mandatory: sandStorm requires this) |
273 |
method handleEvent(QueueElementIF item) { |
method handleEvent(QueueElementIF item) { |
274 |
|
|
275 |
if (item instanceof StagesInitializedSignal) { |
if (item instanceof StagesInitializedSignal) { |
311 |
dispatch(response) |
dispatch(response) |
312 |
} |
} |
313 |
|
|
314 |
|
// handles a certain type of event(XXX) |
315 |
|
method handleXXXMsg(item){ |
316 |
|
// do something |
317 |
|
} |
318 |
|
|
319 |
// create a Storm query to a Tapestry network |
// create a Storm query to a Tapestry network |
320 |
// Query tag is Storm block's GUID |
// Query tag is Storm block's GUID |
321 |
method createQuery(){ |
method createQuery(){ |
347 |
dispatch(unpublishMsg) |
dispatch(unpublishMsg) |
348 |
} |
} |
349 |
|
|
350 |
|
// dispatch TapestryPublishMsg for a single block in the |
351 |
|
// local Storm pool |
352 |
|
method publishSingleStormBlock(blockGUID) { |
353 |
|
tag = new BlockIDTag(blockGUID) |
354 |
|
publishMsg = new |
355 |
|
TapestryPublishMsg(blockGUID, tag) |
356 |
|
dispatch(publishMsg) |
357 |
|
} |
358 |
|
|
359 |
|
// dispatch TapestryUnpublishMsg for a single block in the |
360 |
|
// local Storm pool |
361 |
|
method unpublishSingleStormBlock(blockGUID) { |
362 |
|
tag = new BlockIDTag(blockGUID) |
363 |
|
unpublishMsg = new |
364 |
|
TapestryUnpublishMsg(blockGUID, tag) |
365 |
|
dispatch(unpublishMsg) |
366 |
|
} |
367 |
} |
} |
368 |
|
|
369 |
.. more to come |
.. more to come |