/[gzz]/gzz/Documentation/misc/hemppah-progradu/masterthesis.tex
ViewVC logotype

Diff of /gzz/Documentation/misc/hemppah-progradu/masterthesis.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.185 by hemppah, Tue Mar 25 12:10:22 2003 UTC revision 1.186 by hemppah, Tue Mar 25 13:37:05 2003 UTC
# Line 1181  may not be fast when desired data item r Line 1181  may not be fast when desired data item r
1181  Directed BFS \cite{yang02improvingsearch} optimizes the original  Directed BFS \cite{yang02improvingsearch} optimizes the original
1182  BFS in a way that a peer selects the neighbors which have provided many quality results in the past,  BFS in a way that a peer selects the neighbors which have provided many quality results in the past,
1183  thereby maintaining the quality of costs and decreasing the amount  thereby maintaining the quality of costs and decreasing the amount
1184  of messages sent to network. Alpine \cite{alpineurl} and NeuroGrid \cite{joseph02neurogrid}  of messages sent to network.
 are Peer-to-Peer systems which use somewhat similar method when performing data lookups.  
1185    
1186  Local indices \cite{yang02improvingsearch} is a variation of active caching.  In the local indices techique \cite{yang02improvingsearch}, each peer maintains an index over the data of all peers within
 In this scheme, each peer maintains an index over the data of all peers within  
1187  $h$ hops of itself, where $h$ is a system-wide variable, called radius of the  $h$ hops of itself, where $h$ is a system-wide variable, called radius of the
1188  index\footnote{In the normal BFS case, the value of $h$ is 0, as a peer only has index  index\footnote{In the normal BFS case, the value of $h$ is 0, as a peer only has index
1189  over its local content.}. Mutual index caching architecture, as proposed in  over its local content.}. Thus, when a peer receives a data lookup request, it can
1190  \cite{osokine02distnetworks}, is a variation of local indices technique.  process the request on behalf of every node within $r$ hops. Compared to original BFS,
1191    the local indices technique keeps data lookup cost low while maintaining the same
1192    number of search results.
1193    
1194  In the random walk approach \cite{lv02searchreplication}, a peer forwards query to a  In the random walk approach \cite{lv02searchreplication}, a peer forwards query to a
1195  randomly selected neighbor. The basic random walk approach  randomly selected neighbor. The basic random walk approach
1196  has a poor response time but it doesn't generate as much network traffic as  has a poor response time but it doesn't generate as much network traffic as
1197  the original BFS. As suggested in \cite{lv02searchreplication}, the  the original BFS. As suggested in \cite{lv02searchreplication}, the
1198  random walk approach can be made more effective by introducing  random walk approach can be made more effective by introducing
1199  multiple ''walkers''.  multiple simultaneously working ''walkers''. Freenet \cite{clarke00freenet} uses
1200    random walk searches in data lookups. Freenet's data lookup model resembles
 Freenet \cite{clarke00freenet} uses random walk searches in data lookups. Freenet's data lookup model resembles  
1201  Depth-First-Search (DFS) and peers' routing tables are dynamically built  Depth-First-Search (DFS) and peers' routing tables are dynamically built
1202  using caching. This is an outcome of Freenet's main design principles, anonymity.  using caching. This is an outcome of Freenet's main design principles, anonymity.
1203  Another property of the Freenet's data lookup model is that  Another property of the Freenet's data lookup model is that
1204  it adapts well with varying usage patterns. Improvements to Freenet's data lookup using  it adapts well with varying usage patterns (e.g., searching for popular data items in the overlay).
1205  the ''small-world phenomenon'' have been proposed by Zhang et al. \cite{zhang02using}.  Improvements to Freenet's data lookup using
1206    the ''small-world'' techniques have been proposed by Zhang et al. \cite{zhang02using}.
1207    
1208  Since tightly structured systems have an efficient data lookup at the application level overlay,  Since tightly structured systems have an efficient data lookup at the application level overlay,
1209  current research efforts are focused on the proximity-based data lookup. In the proximity-based data lookup,  current research efforts are focused on the proximity-based data lookup. In the proximity-based data lookup,
1210  peers try to choose entries of routing-tables referring to other peers that are \emph{nearby} in the  peers try to choose entries of routing-tables referring to other peers that are \emph{nearby} in the
1211  underlying network. In this way, tightly structured systems are able to decrease actual  underlying network. In this way, tightly structured systems are able to decrease the actual
1212  lookup \emph{latency}. CAN \cite{ratnasamy01can}, Kademlia \cite{maymounkov02kademlia},  lookup \emph{latency}. CAN \cite{ratnasamy01can}, Kademlia \cite{maymounkov02kademlia},
1213  Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry} have advanced heuristics for  Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry} have advanced heuristics for
1214  the proximity-based routing. Additionally, most recent version of Chord uses proximity-based  the proximity-based routing. Additionally, most recent version of Chord uses proximity-based
# Line 1216  routing, inspired by Karger and Ruhl \ci Line 1216  routing, inspired by Karger and Ruhl \ci
1216  uses a combination of proximity and application level overlay routing when performing data  uses a combination of proximity and application level overlay routing when performing data
1217  lookups. Authors call this feature as a \emph{constrained load balancing}.  lookups. Authors call this feature as a \emph{constrained load balancing}.
1218    
 Additional research related to proximity-based routing include \cite{karger02findingnearest, hildrum02distributedobject,  
 brinkmann02compactplacement, rhea02probabilistic, castro02networkproximity, ng02predicting, pias03lighthouse}.  
   
1219  \subsection{Fast and usable search}  \subsection{Fast and usable search}
1220    
1221  To make Peer-to-Peer systems even more popular (and usable), these systems have to support flexible, efficient  To make Peer-to-Peer systems even more popular (and usable), these systems have to support flexible, efficient
1222  and easy search methods. For instance, Internet's perhaps the most important feature  and simple search methods \cite{li03feasibility}. Currently, loosely structured systems are able to carry out the flexibility and
1223  is the ability to perform keyword searches (e.g., Google \cite{googleurl}). Currently, only loosely  simplicity requirements and tightly structured systems are able to fulfill the efficiency requirement.
1224  structured systems are able to carry out this requirement. Unfortunately, as discussed in this text,  Research efforts have been focused to make security methods in tightly structured systems more usable since
1225  the data lookup model of the loosely structured approach doesn't scale. Thus, research efforts have  the data lookup model of the loosely structured approach doesn't scale. However, studies
1226  been focused towards tightly structured systems. The main problem with tightly structured systems is the  show that combining flexible search methods and tightly structured systems may not be trivial \cite{harren02complex,
1227  fact that tightly structured algorithms perform data lookups based on a globally unique identifier (key).  ansaryefficientbroadcast03}.  The main problem with tightly structured systems is the
1228    fact that tightly structured algorithms perform data lookups based on a globally unique identifier thereby
1229    making efficient keyword searches hard to implement.
1230    
1231    Some studies have been concentrated on SQL-like queries \cite{harren02complex}
1232    in tightly structured overlays. It is unknown, however, if this approach is realizable to implement, since
1233    initial analysis have shown that this approach is rather complex. Other approaches include adaption of the data lookup model of the loosely
1234    structured approach into tightly structured systems \cite{ansaryefficientbroadcast03, chord:om_p-meng}.
1235    Work in \cite{ansaryefficientbroadcast03} seems quite promising. Authors' work is based on insight that
1236    performing data lookup in the overlay resembles regular tree-like search, where trees' data structure
1237    is distributed throughout the overlay. Some studies suggest additional layer upon overlay network \cite{kronfol02fasdsearch,
1238    joseph02p2players}, which use metadata to implement search methods. The feasibility of implementing additional
1239    search layer on top of the network layer is questionable, especially if the search layer and the network
1240    layer have different assumptions about the participating peers (e.g., the network layer supports heterogeneity
1241    of peers, but the search layer doesn't). Andrzejak et al. propose range queries \cite{andrzejak02rangequeries}
1242    to be used with tightly structured overlays. In this technique, it is feasible to perform data lookups
1243    using ranges of keys thereby covering larger amount of possible data items. Currently their prototype
1244    is designed for the CAN system \cite{ratnasamy01can}.
1245    
1246  Recent study has been focused on the feasibility of Peer-to-Peer Web-like indexing and searching  Recent study has been focused on the feasibility of Peer-to-Peer Web-like indexing and searching
1247  on top of tightly structured overlays \cite{li03feasibility} . Authors argue, that it is possible to implement  on top of tightly structured overlays \cite{li03feasibility} . Authors argue, that it is possible to implement
1248  Peer-to-Peer Web-like search with certain compromises. First, Peer-to-Peer search engine may need to  Peer-to-Peer Web-like search with certain compromises. First, Peer-to-Peer search engine may need to
1249  decrease the result quality in order to make searching more efficient. Second, Peer-to-Peer systems must  decrease the result quality in order to make searching more efficient. Second, Peer-to-Peer systems must
1250  consult the properties of underlying network for better performance.  consult the properties of underlying network for better performance.
   
 Some studies have been concentrated on SQL-like queries \cite{harren02complex}  
 in tightly structured overlays. Other approaches include adaption of the data lookup model of the loosely  
 structured approach into tightly structured systems \cite{ansaryefficientbroadcast03, chord:om_p-meng}.  
 Some studies suggest additional layer upon overlay network \cite{kronfol02fasdsearch, joseph02p2players}  
 and range queries \cite{andrzejak02rangequeries}.  
1251    
1252  Many techniques have been developed in order to provide more efficient search indexing. As  Many techniques have been developed in order to provide more efficient search indexing. As
1253  several studies show, the popularity of queries in the Internet follow Zipf-like  several studies show, the popularity of queries in the Internet follow Zipf-like
1254  distributions\footnote{Zipf-distribution is a variant of power-law function.  distributions\footnote{Zipf-distribution is a variant of power-law function.
1255  Zipf-distribution can be used in observation of frequency of occurrence event $E$, as a function of the rank  Zipf-distribution can be used in observation of frequency of occurrence event $E$, as a function of the rank
1256  $i$ when the rank is determined by the frequency of occurrence, is a power-law function $E_i \sim \frac{1}{i^{a}}$,  $i$ when the rank is determined by the frequency of occurrence, is a power-law function $E_i \sim \frac{1}{i^{a}}$,
1257  where the exponent $a$ is close to unity.} (e.g., \cite{breslau98implications}). Therefore, caching and pre-computation  where the exponent $a$ is close to unity.} (e.g., \cite{breslau98implications}).
1258  can be done for optimizing search indices \cite{li03feasibility}. Regular compression algorithms,  Therefore, according to \cite{li03feasibility}, caching and pre-computation can be done for optimizing search indices.
1259  Bloom filters \cite{362692}, vector space models \cite{CuencaAcuna2002DSIWorkshop} and view  Authors in \cite{li03feasibility} use Gap compression \cite{wittengigabytes}, Adaptive Set Intersection \cite{338634}  
1260  trees \cite{Bhattacharjee03resultcache} can be used for even better optimizations. Authors  and clustering with their search optimizations. Regular compression algorithms, Bloom filters \cite{362692}, vector
1261  in \cite{li03feasibility} use Gap compression \cite{wittengigabytes}, Adaptive Set Intersection \cite{338634}    space models \cite{CuencaAcuna2002DSIWorkshop} and view trees \cite{Bhattacharjee03resultcache} can be used for even
1262  and clustering with their search optimizations.  better optimizations.
1263    
1264  While it is expected that web-like searches can be layered on a top of tightly structured overlay, much  While it is expected that web-like searches can be layered on a top of tightly structured overlay, much
1265  more research is required to make indexing and searching more efficient.  more research is required to make indexing and searching more efficient.
1266    
   
1267  \subsection{System management}  \subsection{System management}
1268    
1269  Adaptive system management and self-organization are essential properties  Adaptive system management and self-organization are essential properties
1270  of any Peer-to-Peer system, since centralized control over the system is missing. Loosely structured  of any Peer-to-Peer system, since centralized control over the system is missing. Loosely structured
1271  systems require less system management properties than tightly structured systems; in a loosely  systems require less system management properties than tightly structured systems: in a loosely
1272  structured system, peers join and leave the system constantly without any restrictions. On the  structured system, peers join and leave the system constantly without any restrictions \cite{saroiu02measurementstudyp2p}.
1273  other hand, however, peers in tightly structured system join and leave the system but have less freedom,  On the other hand, however, peers in tightly structured system join and leave the system but have less freedom,
1274  i.e., overlay chooses peer's neighbors on behalf of the peer itself and maps data items randomly  i.e., overlay chooses peer's neighbors on behalf of the peer itself and maps data items randomly
1275  throughout the overlay network.  throughout the overlay network \cite{balakrishanarticle03lookupp2p}. Almost all presented algorithms
1276    for the tightly structured systems have been analyzed under static simulation
1277  All presented algorithms of the tightly structured approach have been analyzed under static simulation  environments \cite{libennowell01observations}. Furthermore, proposed tightly structured overlays are configured statically to achieve
 environments. Furthermore, proposed tightly structured overlays are configured statically to achieve  
1278  the desired reliability even in a uncommon and adverse environment \cite{rowston03controlloingreliability}.  the desired reliability even in a uncommon and adverse environment \cite{rowston03controlloingreliability}.
1279  The most important factor for future research is to get real-life experiences from tightly structured  Thus, one of the most important factors for future research is to get real-life experiences from tightly structured
1280  systems, when there are frequent joins and leaves in the system.  systems, when there are frequent joins and leaves of peers in the system.
1281    
1282  The concept of ''half-life'' was introduced by Liben-Nowell \cite{libennowell01observations} since Peer-to-Peer  As mentioned before, an implicit assumption of almost every tightly structured system is that there is a random, uniform
1283  system is \emph{never} in the ''ideal'' state as Peer-to-Peer system is continiously evolving system. Half-life is defined  distribution of peer and key identifiers. Even if participating peers are extremely heterogeneous, e.g., in
1284  as follows: let there be $N$ live peers at time $t$. The doubling from time $t$ is the time that pass before  computing power or network bandwidth, all data items are distributed uniformly. Clearly, this is
1285  $N$ new additional peers arrive into the system. The halving time from time $t$ is the time  a serious problem of tightly structured overlays in face of performance and load balancing \cite{rao03loadbalancing}.
1286  required for half of the living peers at time $t$ to leave the system. The half-life from  Measurement study by Saroiu et al. show that there is a extreme heterogeneity among participating peers in already deployed Peer-to-Peer
1287  time $t$ is smaller of the properties stated above. The half-life of the entire system is the  systems \cite{saroiu02measurementstudyp2p}. Symphony \cite{gurmeet03symphony} seems to be the first tightly structured overlay system
1288  minimum half-life over all times $t$. Concept of half-time can be used as a basis for developing  which supports heterogeneity. Zhao et al. have proposed a secondary layer on top of a structured overlay
1289  more powerful analytical tools for modelling complex Peer-to-Peer systems.  to support heterogeneity better \cite{zhao02brocade}.
1290    
1291  Some research has been done with regard to load balancing properties of tightly structured  Some research has been done with regard to load balancing properties of tightly structured
1292  overlays. Byers et al. suggest an idea of ''power of two choices'' whereby data item is stored at the less loaded  overlays. Byers et al. suggest an idea of ''power of two choices'' whereby data item is stored at the less loaded
1293  of two (or more) random peer alternatives \cite{byers03dhtbalancing}. Rao et al. use virtual servers  of two (or more) random peer alternatives \cite{byers03dhtbalancing}. Rao et al. use virtual servers
1294  to control the load balance in a Peer-to-Peer system \cite{rao03loadbalancing}. Their work rests on the  to control the load balance in a Peer-to-Peer system \cite{rao03loadbalancing}. Their work rests on the
1295  idea which was originally introduced by Chord \cite{stoica01chord} system.  idea which was originally introduced by Chord \cite{stoica01chord} system.
1296    Ledlie et al. propose techniques for forming and maintaining groups in a highly dynamic environment
1297    \cite{ledlie02selfp2p}. Their work relies on the idea that
1298    participating peers would create multiple hierarchical groups. It is not clear whether this approach
1299    is scalable or fault tolerant and suitable for Peer-to-Peer environment. More promising work has been done by Rowston et al.
1300    in \cite{rowston03controlloingreliability}. Authors propose techniques for self-tuning, dealing with
1301    uncommon conditions (e.g., network partition and high failure rates). Moreover, authors argue that
1302    with these techniques, the concerns over tightly structured overlay maintenance costs are no more
1303    an open issue.
1304    
1305  Also, query and routing hot spots may be an issue in tightly structured overlays \cite{ratnasamy02routing}.  Also, query and routing hot spots may be an issue in tightly structured overlays \cite{ratnasamy02routing}.
1306  Hot spots happen, when a specific key is being requested extremely often in tightly structured overlays. Recent study  Hot spots happen, when a specific key is being requested extremely often in tightly structured overlays. Recent study
1307  by Freedman et al. tries to reduce hot spots in the system by performing \emph{sloppy} hashing  by Freedman et al. tries to reduce hot spots in the system by performing \emph{sloppy} hashing
1308  \cite{sloppy:iptps03}. Authors' technique is especially suitable for the DOLR abstraction of tightly structured overlays.  \cite{sloppy:iptps03}. Authors' technique is especially suitable for the DOLR abstraction of tightly structured overlays.
1309  With Sloppy hashing, we are able to reduce the generation of query hot spots. Sloppy hashing enables to  They arque that with Sloppy hashing, the generation of query hot spots can be reduces and peers are able
1310  locate nearby data without looking up data from distant peers. Moreover, authors'  locate nearby data without looking up data from distant peers. Moreover, authors'
1311  proposal for self-organizing clusters using network diameters may be useful,  proposal for self-organizing clusters using network diameters may be useful,
1312  especially within small groups of working people. Thus, with Sloppy hashing  especially within small groups of working people.
 we can provide locality properties the system.  
   
   
   
 As mentioned before, an implicit assumption of almost every tightly structured system is that there is a random, uniform  
 distribution of peer and key identifiers. Even if participating peers are extremely heterogeneous, e.g., in  
 computing power or network bandwidth, all data items are distributed uniformly. Clearly, this is  
 a serious problem of tightly structured overlays in face of performance and load balancing. Measurement study  
 by Saroiu et al. show that there is a extreme heterogeneity among participating peers in already deployed Peer-to-Peer  
 systems \cite{saroiu02measurementstudyp2p}. Symphony \cite{gurmeet03symphony} seems to be the first tightly structured overlay system  
 which supports heterogeneity. Zhao et al. have proposed a secondary layer on top of a structured overlay  
 to support heterogeneity better \cite{zhao02brocade}.  
1313    
1314  Research has been done on self-organization. Ledlie et al. propose techniques for forming and maintaining  The concept of ''half-life'' was introduced by Liben-Nowell \cite{libennowell01observations} since Peer-to-Peer
1315  groups in a highly dynamic environment \cite{ledlie02selfp2p}. Unfortunately their work relies on the idea that  system is \emph{never} in the ''ideal'' state as Peer-to-Peer system is continiously evolving system. Half-life is defined
1316  participating peers would create multiple hierarchical groups. It is not clear whether this approach  as follows: let there be $N$ live peers at time $t$. The doubling from time $t$ is the time that pass before
1317  is fault tolerant and suitable for Peer-to-Peer environment. More promising work has been done by Rowston et al.  $N$ new additional peers arrive into the system. The halving time from time $t$ is the time
1318  in \cite{rowston03controlloingreliability}. Authors propose techniques for self-tuning, dealing with  required for half of the living peers at time $t$ to leave the system. The half-life from
1319  uncommon conditions (e.g., network partition and high failure rates). Moreover, authors argue that  time $t$ is smaller of the properties stated above. The half-life of the entire system is the
1320  with these techniques, the concerns over tightly structured overlay maintenance costs are no more  minimum half-life over all times $t$. Concept of half-time can be used as a basis for developing
1321  an open issue.  more powerful analytical tools for modelling complex Peer-to-Peer systems.  
1322    
1323  Finally, little research has been done regarding self-monitoring and data availability. Zhang et al.  Finally, little research has been done regarding self-monitoring. Zhang et al.
1324  describe an arbitrary data structure on top of a tightly structured overlay \cite{zhang03somo}. Authors  describe an arbitrary data structure on top of a tightly structured overlay \cite{zhang03somo}. Authors
1325  call their technique as a \emph{data overlay}, since it supports several fundamental data structures.  call their technique as a \emph{data overlay}, since it supports several fundamental data structures.
1326  Authors have used this data overlay when building a Self-Organized Meta data Overlay (SOMO), which can be used  Authors have used this data overlay when building a Self-Organized Meta data Overlay (SOMO), which can be used

Legend:
Removed from v.1.185  
changed lines
  Added in v.1.186

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26