/[storm]/storm/doc/pegboard/storm_with_tapestry--hemppah/peg.rst
ViewVC logotype

Diff of /storm/doc/pegboard/storm_with_tapestry--hemppah/peg.rst

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.10 by hemppah, Mon Jul 7 10:44:48 2003 UTC revision 1.11 by hemppah, Mon Jul 7 11:25:43 2003 UTC
# Line 15  PEG storm_with_tapestry--hemppah: Line 15  PEG storm_with_tapestry--hemppah:
15    
16  .. Affect-PEGs:  .. Affect-PEGs:
17    
18  - this document describes the usage of Tapestry with Storm  This document describes the use of Tapestry overlay with Storm. We start by
19  - possibly some architectural changes are required to Storm if Tapestry is  proviving a short introduction to Tapestry, to the SEDA model and to the Tapestry
20    taken to in use. Thus, this document is needed  API. Then, we propose new classes and a pseudo code to be included into Storm
21  - first, this PEG discusses general aspects of Tapestry, it's API and the  code base. Pseudo code describes a simple StormTapestryManager which supports
22    usage of the API with other applications  query and insert operations in a Tapestry overlay.
23  - introduction to Tapestry, the SEDA model, Tapestry API, how to use Tapestry  
24    with other application    This document is based on Tapestry 2.0 release.
 - then, a proposal of usage with Storm  
 - pseudo code: search and insert operations  
 - simple "implementation" and architectural design  
 - possible changes are listed  
 - this document is based on Tapestry release 2.0  
25    
26  Issues  Issues
27  ======  ======
# Line 37  ISSUE: Do we need any code from the Ocea Line 32  ISSUE: Do we need any code from the Ocea
32  Introduction to Tapestry  Introduction to Tapestry
33  ========================  ========================
34    
35  - currently Tapestry_ implements the DOLR abstracion  Tapestry_ is a application level Peer-to-Peer overlay. Currently Tapestry_
36  - other abstraction can be implemented also (e.g., DHT)  supports the DOLR abstracion but other abstraction can be implemented
37  - two primary functions: 1) efficient way for nodes to communicate across  (e.g., DHT). Tapestry's DOLR interface has two primary functions: efficient
38    an overlay and 2) effcient object location  way for nodes to communicate across the overlay and effcient object location.
39  - objects are *published* with object to location mappings (pointers)  In Tapestry, objects are *published* with object to location mappings (a.k.a.
40  - differs from DHTs in that always use the closest available object  pointers). This operation differs from DHTs in that always use the closest
41  - routing table consists of levels of neighbor links  available object.
42  - each level represents a matching prefix up to a digit position in the ID  
43  - for each level, neighbor links are selected based on "closest" in network  Tapestry's routing table consists of levels of neighbor links. Each level
44    latency  represents a matching prefix up to a digit position in the ID. Additionally
45  - along the publication process, the overlay deposits a pointer at every hop  for each level, neighbor links are selected based on closest in network
46    along the way  *latency*. Along the publication process, the overlay deposits a pointer at
47  - when an object is searched, a query is forwarded towards object's  every hop along the way. When an object is searched, a query is forwarded
48    publication route, and when a query run into a first pointer, overlay  towards object's publication route, and when a query run into a first pointer,
49    directly forwards to the object    overlay directly forwards to the object.
50  - written in Java (currently JDK 1.3)  
51  - uses SEDA (Staged Event-Driven Architecture) framwork  Tapestry 2.0 release is written in Java and uses SEDA (Staged Event-Driven
52  - approximately 57000 lines of code  Architecture) framwork for thread and I/O-operations. Tapestry's code base
53  - shares codebase with Oceanstore  has approximately 57000 lines of code and shares some code with Oceanstore's
54    code base.
55    
56  Overview of SEDA  Overview of SEDA
57  ================  ================
58    
59  - asynchronous I/O library (similar to JDK 1.4's java.nio package)  SEDA is a asynchronous I/O library (similar to JDK 1.4's java.nio package). In
60  - stages communicate with each other by sending events  SEDA there are stages and they communicate with each other by sending events.
61  - each Tapestry component is implemented as a stage  A stage runs as a thread which starts by executing initialization routines and
62  - stage runs as a thread which starts by executing initialization routines  then enters an event loop.  Inside each JVM, a dispatcher monitors all messages
63    and then enters an event loop  and events, and delivers copies to each stage that subscribes to messages of
64  - inside each JVM, a dispatcher monitors all messages and events,  that type.
65    and delivers copies to each stage that subscribes to messages of that type  
66  - a (common) Tapestry node uses StaticTClient stage, DynamicTClient stage,  Each Tapestry component is implemented as a stage. A (common) Tapestry node
67    Router stage and Tapestry application stage  uses StaticTClient stage, DynamicTClient stage, Router stage and Tapestry
68  - for each stage, a config file is required to specify stage's properties  application stage. For each stage, a config file is required to specify
69    (XML-like tag structure)  stage's properties (XML-like tag structure). During the initialization phase,
70  - during the initialization phase, a stage specifies which event and messages  a stage specifies which event and messages it wants to "listen" to. For Storm,
71    it wants to "listen" to  the Tapestry application stage is the most important.
 - for Storm, the Tapestry application stage is the most important  
72    
73    
74  Overview of Tapestry API  Overview of Tapestry API
# Line 100  Tapestry that are used by other applicat Line 95  Tapestry that are used by other applicat
95  Using Tapestry API from other applications  Using Tapestry API from other applications
96  ==========================================  ==========================================
97    
 - confgig file for defining a Storm stage (Tapestry application stage)  
   
98  According to the `Tapestry programmer's guide`_, there three steps to write  According to the `Tapestry programmer's guide`_, there three steps to write
99  a Tapestry application:  a Tapestry application:
100    
101     - Write the necessary messages to interface with Tapestry     - Write the necessary messages to interface with Tapestry
102     - Write an Eventhandler class that would serve as the application     - Write an event handler class that would serve as the application
103     - Write configuration file(s) to define stages and specify initialization     - Write configuration file(s) to define stages and specify initialization
104       arguments       arguments
105            
106  Interface with Tapestry  Custom messages for Tapestry
107  -----------------------  -----------------------
108    
109    Again, According to the `Tapestry programmer's guide`_:
110    
111  1. An application must extend extending the abstract API message types, e.g.:  1. An application must extend extending the abstract API message types, e.g.:
112    
113     public class StormMsg extends TapestryPublishmsg {     public class StormMsg extends TapestryPublishmsg {
# Line 125  network: Line 120  network:
120        
121  3. Do not write a working type_code method  3. Do not write a working type_code method
122    
123  4. Register your message class  
124    An event handler class
 Eventhandler class  
125  ------------------  ------------------
126    
127  1. Write a class which extends SEDA's existing stages, e.g.,  1. Write a eventhandler class which extends SEDA's existing stages, e.g.,
128    
129    public class StormP2p extends StandardStage    public class StormTapestryManager implements EventHandlerIF
130        
131  2. In Eventhandler class, implement a hanleEvent method which will  2. In a eventhandler class, implement a hanleEvent method which will
132     process registered messages     process registered messages
133        
134  3. In Eventhandler class, implement a init method which will define  3. In a eventhandler class, implement a init method which will define
135     the messages that are "listened" for this class     the events and messages that are "listened" for this class
136    
137    4. Register your custom message classes
138    
139  Using Tapestry with Storm  Using Tapestry with Storm
140  =========================  =========================
141    
142  - in this section, we will outline the design which would allow Storm to  In this section, we will outline the design which would allow Storm to
143    Tapestry's routing services  Tapestry's routing services.
     
     
     
 - For finding Storm blocks in a Tapestry overlay, we must create:  
144    
145    For finding Storm blocks in a Tapestry overlay, we must create custom message
146    classes:
147    
148     - BlockIDTag class for Storm blocks' IDs (implements     - BlockIDTag class for Storm blocks' IDs (implements
149       ostore.tapestry.api.TapestryTag) for storing query specific information:       ostore.tapestry.api.TapestryTag) for storing query specific information:
# Line 181  Using Tapestry with Storm Line 174  Using Tapestry with Storm
174       implements ostore.util.QuickSerializable) to carry a requested Storm       implements ostore.util.QuickSerializable) to carry a requested Storm
175       block being sent to a original requester       block being sent to a original requester
176        
177  - For interacting Storm's storage model with Tapestry:  For interacting Storm's storage model with Tapestry we must create and event
178    handler class:
179    
180     - StormTapestryManager class which     - StormTapestryManager class (implements sandStorm.api.EventHandlerIF) for
181       - registers Storm to handle Storm-related Tapestry events and messages       listening standard Tapestry events, registering custom Storm messages,
182       - handles and dispatches Storm-related Tapestry events and messages       dispatching custom Storm messages and interacting with a local Storm pool
      - interacts with a local Storm pool  
183            
184            
185   PSEUDOCODE for Storm  Here, we propose a simple StormTapestryManager pseudo code. This is an event
186    handler class for Tapestry. In addition, this class provides Storm block
187    query and insert operations in a Tapestry overlay.
188    
189          class StormTapestryManager implements EventHandlerIF {          class StormTapestryManager implements EventHandlerIF {
190                    
191                  // dispatcher which dispatch all items                  // dispatcher which dispatch all items
192                  // to target peer                  // to target peer (Mandatory: sandStorm requires this)
193                  method dispatch (QueueElementIF item) {                  method dispatch (QueueElementIF item) {
194                          //try do dispatch an item                          //try do dispatch an item
195                          try {                          try {
# Line 207  Using Tapestry with Storm Line 202  Using Tapestry with Storm
202                  // general initialization method                  // general initialization method
203                  // that handles subscribes this stage                  // that handles subscribes this stage
204                  // to listen certain events and messages                  // to listen certain events and messages
205                    // (Mandatory: sandStorm requires this)
206                  method init (ConfigDataIF config) {                  method init (ConfigDataIF config) {
207    
208                  // find our NodeId                  // find our NodeId
# Line 222  Using Tapestry with Storm Line 218  Using Tapestry with Storm
218                  // create peer ID based on peer's public key                  // create peer ID based on peer's public key
219                   _peer = new SHA1Hash (pkey);                   _peer = new SHA1Hash (pkey);
220                                    
221                   // the *events* which we want to listen                   // the mandatory *events* which we want to listen
222    
223                   array event_types = {                   array event_types = {
224                   "seda.sandStorm.api.StagesInitializedSignal",                   "seda.sandStorm.api.StagesInitializedSignal",
# Line 234  Using Tapestry with Storm Line 230  Using Tapestry with Storm
230                          classifier.subscribe (event_type[i])                          classifier.subscribe (event_type[i])
231                   }                   }
232                                    
233                  // the *messages* we want receive through Tapestry                  // the mandatory *messages* we want receive through Tapestry
234    
235                  array message_types = {                  array message_types = {
236                  "org.nongnu.storm.p2p.tapestry.StormErrorMsg",                  "org.nongnu.storm.p2p.tapestry.StormErrorMsg",
# Line 263  Using Tapestry with Storm Line 259  Using Tapestry with Storm
259                  }                  }
260                                    
261                  // handles *all* events which stage is listening                  // handles *all* events which stage is listening
262                    // (Mandatory: sandStorm requires this)
263                   method handleEvents(QueueElementIF array items)                   method handleEvents(QueueElementIF array items)
264                   throws EventHandlerException {                   throws EventHandlerException {
265                                    
# Line 272  Using Tapestry with Storm Line 269  Using Tapestry with Storm
269                                    
270                  // handles single event for this stage. handleEvents                  // handles single event for this stage. handleEvents
271                  // method gives an item this method                  // method gives an item this method
272                    // (Mandatory: sandStorm requires this)
273                  method handleEvent(QueueElementIF item) {                  method handleEvent(QueueElementIF item) {
274                                    
275                  if (item instanceof StagesInitializedSignal) {                  if (item instanceof StagesInitializedSignal) {
# Line 313  Using Tapestry with Storm Line 311  Using Tapestry with Storm
311                          dispatch(response)                                        dispatch(response)              
312                  }                  }
313                                    
314                    // handles a certain type of event(XXX)
315                    method handleXXXMsg(item){
316                            // do something
317                    }
318                            
319                  // create a Storm query to a Tapestry network                  // create a Storm query to a Tapestry network
320                  // Query tag is Storm block's GUID                  // Query tag is Storm block's GUID
321                  method createQuery(){                  method createQuery(){
# Line 344  Using Tapestry with Storm Line 347  Using Tapestry with Storm
347                                  dispatch(unpublishMsg)                                  dispatch(unpublishMsg)
348                  }                  }
349                                    
350                    // dispatch TapestryPublishMsg for a single block in the
351                    // local Storm pool            
352                    method publishSingleStormBlock(blockGUID) {
353                            tag = new BlockIDTag(blockGUID)                        
354                            publishMsg = new
355                            TapestryPublishMsg(blockGUID, tag)
356                            dispatch(publishMsg)
357                    }
358                    
359                    // dispatch TapestryUnpublishMsg for a single block in the
360                    // local Storm pool            
361                    method unpublishSingleStormBlock(blockGUID) {
362                            tag = new BlockIDTag(blockGUID)                        
363                            unpublishMsg = new
364                            TapestryUnpublishMsg(blockGUID, tag)
365                            dispatch(unpublishMsg)
366                    }
367          }          }
368            
369  .. more to come  .. more to come

Legend:
Removed from v.1.10  
changed lines
  Added in v.1.11

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26