/[gzz]/gzz/Documentation/misc/hemppah-progradu/research_problems
ViewVC logotype

Diff of /gzz/Documentation/misc/hemppah-progradu/research_problems

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.6 by hemppah, Tue Dec 17 13:57:40 2002 UTC revision 1.7 by hemppah, Wed Dec 18 11:08:14 2002 UTC
# Line 1  Line 1 
   
   
1  1. Approaches  1. Approaches
2    
3  -there are five approaches when performing searches in p2p networks.  -there are five approaches when performing searches in p2p networks.
# Line 40  impossible Line 38  impossible
38  -SWNs require O(log^2 n) hops to reach arbitrary destinations, assuming (*only and only if* !!!) that  -SWNs require O(log^2 n) hops to reach arbitrary destinations, assuming (*only and only if* !!!) that
39  links between nodes are constructed in the way that they are uniformly distributed over all distances  links between nodes are constructed in the way that they are uniformly distributed over all distances
40  in the network (Kleinberg)  in the network (Kleinberg)
41    
42  -Example systems: SWAN, Freenet  -Example systems: SWAN, Freenet
43    
44    Req. 1:
45    The short-range links must be such that for all nodes n and m, where n != m, n
46    has a short-range link to a node l, so that distance(l, m) < distance(n, m).
47    
48    Notice: The length of the link from node n to node m is distance(n, m)
49    
50    Req. 2:
51    The long-range links must be such that for all Long-range links are nearly
52    uniformly distributed over all 'distance scales'.
53    
54    Notice: There *is* a more formal specification about req. 2, but I won't give it
55    right here, because it quite mathematical (and therefore it makes no sense to
56    present it here with these characters).
57    
58    And, if these requirements are met, SWN network can locate any data in O(log^2
59    n) hops (Kleinberg and e.g. simulations in SWAN)
60    
61  1.3. Flooding Broadcast Networks (FBN)  1.3. Flooding Broadcast Networks (FBN)
62  +own resources are not mapped into the network  +own resources are not mapped into the network
63  +keyword/fuzzy search possible  +keyword/fuzzy search possible
# Line 89  Social:                N/A***          N/A***          N/A*** Line 105  Social:                N/A***          N/A***          N/A***
105    
106  ** = Please see http://www.darkridge.com/~jpr5/doc/gnutella.html for details  ** = Please see http://www.darkridge.com/~jpr5/doc/gnutella.html for details
107    
108  ** = From p2p-hackers mailinglist:  *** = From p2p-hackers mailinglist:
109  - - -  - - -
110    
111  > Btw, how well Alpine can scale (e.g. number of users) ? Do you have any "real-  > Btw, how well Alpine can scale (e.g. number of users) ? Do you have any "real-
# Line 205  c) FBN Line 221  c) FBN
221          -metadata ?          -metadata ?
222                    
223  2.1.5. Answers to research problem  2.1.5. Answers to research problem
224    -there are different kinds of uses of DHTs, e.g. DHT actually stores the data, or DHT only stores
225    the values, which are used for locating the data
226    -current systems mainly uses to latter method
227    -both have pros and cons:
228            -in the value use, several hostile nodes might say that they hosts the value, but refuse to serve any host
229            -in the storage use, hostile node might say that someone must save data block size of 1TB for her computer
230            -however, there are methods for preventing these kinds of actions
231    -DHTs and SWNs are very robust to failures: e.g. 20% of the nodes simultaneously fail, less than 0.1% of the data retrievals are failed
232    -the basic difference between DHTs/SWNs is that in DHT/SWN approach, systems "knows", where the desired data block resides
233            -based on data block ID's, nodes gives "hints" (routes more close to ID in the key space) constantly to a query lookup until
234            the specific node is found which hosts the block
235            -no extra network traffic
236    -so in a way, DHT/SWNs are structured entities
237    -instead, in other approaches, system doesn't "know" where the data block resides
238            -we have to ask from any participating node, if they have the data block
239            -very much extra network traffic
240    
241  -the most efficient algorithm: O(log n)  -the most efficient algorithm: O(log n)
242          -DHTs requires O(log n) hops, log n neighbors, except Viceroy (however, robustness suffers)          -DHTs requires O(log n) hops, log n neighbors, except Viceroy (however, robustness suffers)
243          -SWNs requires O(log^2 n) hops, much less neighbors (constant, is not depedent of n)          -SWNs requires O(log^2 n) hops, much less neighbors (constant, is not depedent of n)
# Line 214  c) FBN Line 247  c) FBN
247  -in this case, FBNs, Hybrids, Social Discovery require much bandwidth, since there is no benefit from the block's ID at all (they have to ask constantly)  -in this case, FBNs, Hybrids, Social Discovery require much bandwidth, since there is no benefit from the block's ID at all (they have to ask constantly)
248  -SWNs require less memory than DHTs, since there are less connections to other nodes  -SWNs require less memory than DHTs, since there are less connections to other nodes
249  -SWNs relies less to neighbor nodes than DHTs  -SWNs relies less to neighbor nodes than DHTs
250  -proposol: most efficient approaches for this research problem are DHTs and SWNs, since they are based on key-value pairs (Storm has a key as block ID)  -proposol: most efficient approaches for this research problem are DHTs and SWNs, since they are based on key-value pairs (Storm has a key as block ID)
251    
252    
253    
254    
# Line 243  return the most recent block, if the sig Line 277  return the most recent block, if the sig
277  -In this case (urn-5), there is no benefit from the block's ID at all (DHT, SWN)  -In this case (urn-5), there is no benefit from the block's ID at all (DHT, SWN)
278    
279  2.2.4. Open questions:  2.2.4. Open questions:
280  -in this case, is it sensible to maintain two different key-value mappings key-value based systems (DHT, SWN) ?  -in this case, is it sensible to maintain two different key-value mappings key-value based systems (DHT, SWN) ?l
281          -one fore block IDs          -one fore block IDs
282          -one for urn-5 names, which are associated with block IDs          -one for urn-5 names, which are associated with block IDs
283          -is this approach too difficult to maintain ?          -is this approach too difficult to maintain ?
# Line 253  tree/list based structure for all blocks Line 287  tree/list based structure for all blocks
287  -How CAs' data should be saved ?  -How CAs' data should be saved ?
288    
289  2.2.5. Answers to research problem  2.2.5. Answers to research problem
290  -compared to previous research problem, the "first seen" benefits of DHTs and SWNs in this research problem are smaller  -compared to previous research problem, the "obvious" benefits of DHTs and SWNs in this research problem are smaller
291          -we don't know which block is the most recent associated with a specfic urn-5 name  -we don't beforehand know which block is the most recent associated with a specfic urn-5 name
292  -in theory, though, the most efficient algorithm is O(log n), *assuming* that we already know the most recent block's ID (see previos research problem)  -in theory, though, the most efficient algorithm is O(log n), *assuming* that we already know the most recent block's ID (see previous research problem)
293  -currently, there is no "out-of-the-box" answer to this research problem  -the most important question is, how we can efficiently associate urn-5 names with the most recent block / to all blocks which are associated with it
294  -the question is, how we can efficiently associate urn-5 names with the most recent block / to all blocks which are associated with it  -for DHTs and SWNs:
295          For DHTs and SWNs:  
296          -adaption of Benja's 1st idea: each node maintains a local hash table (urn-5 name -> most recent local block ID) for every urn-5 names  Please notice: In this approach, DHT doesn't store the actual block, only the values for locating the data from the system
297          which node hosts --> we don't have to check all blocks and their urn-5 associations  
298          -adaption of Benja's 2nd idea: All urn-5 name mappings are stored as <key, value[ ]>, where the key is urn-5 name's hash and value is a record containing          Req. 1:
299            -each node maintains a local stack based data structure (urn-5 name -> most recent local block ID) for every urn-5 names
300            which node hosts. The most recent block is topmost --> we don't have to check all blocks and their urn-5 associations to get the most recent
301            Req. 2:
302            -all urn-5 name mappings are stored as <key, value[ ]>, where the key is urn-5 name's hash and value is a record containing
303          block ID and timestamp of that block. So, when we store a block in our system first time, we have to create a new key-value:          block ID and timestamp of that block. So, when we store a block in our system first time, we have to create a new key-value:
304          <hash_of_urn_5_name, [block id, block timestamp]> and route this mapping to node which is "closest" to a hash value. Now when we want to find the most          <hash_of_urn_5_name, [block id, block timestamp]> and route this mapping to node which is "closest" to a hash value. Now when we want to find the most
305          recent block associated with a specific urn-5 name, we do:                recent block associated with a specific urn-5 name, we do:
306            
307                  1. locally compute a hash for given urn-5 string                  1. locally compute a hash for given urn-5 string
308                  2. route the query to a node which hosts the given hash of urn-5 ("closest" node)                  2. route the query to a node which hosts the given hash of urn-5 ("closest" node)
309                  3. get most recent block's ID using the idea from previuos idea                  3. get most recent block's ID using the idea from previuos idea
310                    4. use ID to get the specific block using normal DHT/SWN operation
311                                    
312          In this approach, we don't have to perform additional searching and sorting of mappings. And of course, we know that for given urn-5, only one node          In this approach, we don't have to perform additional searching and sorting of mappings. And of course, we know that for given urn-5, only one node
313          hosts *all* the block information ("block history") for the urn-5, since mappings are mapped to a single node, closest to urn-5 hash value.          hosts *all* the block information ("block history") for the urn-5, since mappings are mapped to a single node, closest to urn-5 hash value.
314          Again, this should work fine under existing DHTs (and SWTs ?).            Again, this should work fine under existing DHTs (and SWTs ?).
315            
316            Some simple analysis:
317            -there are more key-value pairs in the system for additional urn-5 --> block associations
318            -however, I don't think this is an issue, since data's size is small
319            -efficiency: find node which hosts urn-5 names + find node which hosts blocks associated with urn-5 name: logn + logn = 2logn (logarithmical)
320            
321    -for FBS and others:
322            -there is no very efficient (simple) methods for finding urn-5 name associated with the most recent block
323            -one simple proposal is that when we visit to each node (first idea above), get only the most recent one and compare them (or greedy approach: dismiss currently
324            most recent block as we visit to nodes, if newer block have been found)
325    
326  2.3. "Searching for Storm blocks associated with specific urn-5 name, where  2.3. "Searching for Storm blocks associated with specific urn-5 name, where
327       specific date has been defined (or date range), and where Storm block       specific date has been defined (or date range), and where Storm block
# Line 301  return the most recent block, if the sig Line 351  return the most recent block, if the sig
351  tree based structure for all blocks for specific urn-5 name) ?  tree based structure for all blocks for specific urn-5 name) ?
352  -How CAs' data should be saved ?  -How CAs' data should be saved ?
353    
354    2.3.5. Answers to research problem
355    This is quite similar to previous research problem's answer. There are slight differences, though.
356    
357    -for DHTs and SWNs:
358    
359    Please notice: In this approach, DHT doesn't store the actual block, only the values for locating the data from the system
360    
361            Req. 1:
362            -each node maintains a local stack based data structure (urn-5 name -> most recent local block ID) for every urn-5 names
363            which node hosts. The most recent block is topmost --> we don't have to check all blocks and their urn-5 associations to get the most recent
364            Req. 2:
365            -all urn-5 name mappings are stored as <key, value[ ]>, where the key is urn-5 name's hash and value is a record containing
366            block ID and timestamp of that block. So, when we store a block in our system first time, we have to create a new key-value:
367            <hash_of_urn_5_name, [block id, block timestamp]> and route this mapping to node which is "closest" to a hash value. Now when we want to find the most
368            recent block associated with a specific urn-5 name, we do:      
369            
370                    1. locally compute a hash for given urn-5 string
371                    2. route the query to a node which hosts the given hash of urn-5 ("closest" node)
372                    3. get the specific block(s) data (block ID) from the stack which matches to given date&time properties (we compare to block header data)
373                    4. use IDs to get the specific blocks using normal DHT/SWN operation            
374            
375            Some simple analysis:
376            -there are more key-value pairs in the system for additional urn-5 --> block associations
377            -however, I don't think this is an issue, since data's size is small
378            -efficiency: find node which hosts urn-5 names + find node which hosts blocks associated with urn-5 name: logn + logn = 2logn (logarithmical)
379            
380    -for FBS and others:
381            -there is no very efficient (simple) methods for finding urn-5 name associated with the most recent block
382            -one simple proposal is that when we be that we visit to each node (first idea above), get all blocks which matches to given properties
383                
384    
385  2.4. "How does search engine should work?"  2.4. "How does search engine should work?"

Legend:
Removed from v.1.6  
changed lines
  Added in v.1.7

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26