/[gzz]/gzz/Documentation/misc/hemppah-progradu/masterthesis.tex
ViewVC logotype

Diff of /gzz/Documentation/misc/hemppah-progradu/masterthesis.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.207 by hemppah, Mon Jun 2 12:14:08 2003 UTC revision 1.208 by hemppah, Wed Oct 8 10:07:18 2003 UTC
# Line 42  E-mail: hemppah@cc.jyu.fi} Line 42  E-mail: hemppah@cc.jyu.fi}
42  In this thesis, first we review existing Peer-to-Peer approaches, algorithms and their  In this thesis, first we review existing Peer-to-Peer approaches, algorithms and their
43  key properties. We summarize open problems in Peer-to-Peer systems and divide these  key properties. We summarize open problems in Peer-to-Peer systems and divide these
44  problems into three sub-categories. We realize that there are many problems and few  problems into three sub-categories. We realize that there are many problems and few
45  practical solutions, and some problems have no solution at all.  practical solutions, and recognize that some problems have no solution at all.
46    
47  Then, we provide an overview of the Fenfire system. The Fenfire system is a free  Then, we provide an overview of the Fenfire system. The Fenfire system is a free
48  software effort to build a location transparent, hyperstructured desktop environment.  software effort to build a location transparent, hyperstructured desktop environment.
49  We evaluate existing Peer-to-Peer approaches-- loosely and tightly structured overlays-- with regard  We evaluate existing Peer-to-Peer approaches--loosely and tightly structured overlays--with regard
50  to Fenfire's needs. Finally, we propose simple methods to efficiently locate Fenfire  to Fenfire's needs. Finally, we propose simple methods to efficiently locate Fenfire
51  data from Peer-to-Peer networks.  data within Peer-to-Peer networks.
52  }  }
53  \tiivistelma{  \tiivistelma{
54  Tässä opinnäytetyössä esittelemme olemassaolevia vertaisverkkoja, algoritmeja ja  Tässä opinnäytetyössä esittelemme olemassaolevia vertaisverkkoja, algoritmeja ja
# Line 83  Peer-to-Peer systems have recently recei Line 83  Peer-to-Peer systems have recently recei
83  academia \cite{projectirisurl} and industry \cite{p2pworkinggroup, jxtaurl} for a  academia \cite{projectirisurl} and industry \cite{p2pworkinggroup, jxtaurl} for a
84  number of reasons. The lack of centralization in Peer-to-Peer systems  number of reasons. The lack of centralization in Peer-to-Peer systems
85  means that the participants can form a distributed system \cite{couloris94distributedsystems}  means that the participants can form a distributed system \cite{couloris94distributedsystems}
86  without any investment to centralized hardware by sharing their services and connecting to each  without any investment in centralized hardware by sharing their services and by connecting to each
87  other directly. Peer-to-Peer systems can be characterized as distributed systems in which all  other directly. Peer-to-Peer systems can be characterized as distributed systems in which all
88  communication is symmetric and all participant entities have similar capabilities and responsibilities  communication is symmetrical and all participant entities have similar capabilities and responsibilities
89  \cite{oram01harnessingpower}. Schollmeier \cite{schollmeier01p2pdefinition} describes a Peer-to-Peer system as a system of  \cite{oram01harnessingpower}. Schollmeier \cite{schollmeier01p2pdefinition} describes a Peer-to-Peer system as a system of
90  distributed entities that share their own services.  distributed entities that share their own services.
91  Each entity, i.e., \emph{peer}, may contribute services to the overall system. The distributed  Each entity, i.e., \emph{peer}, may contribute services to the overall system. The distributed
92  and ad hoc nature of Peer-to-Peer improves scalability and avoids single points of failure.    and ad hoc nature of Peer-to-Peer improves scalability and avoids single points of failure.  
93    
94  The Fenfire project is an attempt to build a hyperstructured, seamlessly interoperating desktop  The Fenfire project is an attempt to build a hyperstructured, seamlessly interoperating desktop
95  environment. In the Fenfire system, all data is stored as blocks.    environment. In the Fenfire system, all data is stored as "blocks".  
96  Each block has a globally unique identifier and it can be referred, by pointer blocks.  Each block has a globally unique identifier and it can be referenced to, by pointer blocks.
97  Other features of Fenfire include innovative user  Other features of Fenfire include innovative user
98  interfaces for viewing data. The applicability of Peer-to-Peer networking with Fenfire for network  interfaces for viewing data. The applicability of Peer-to-Peer networking with Fenfire for network
99  transparency is currently under investigation.  transparency is currently under investigation.
100    
101  Three research problems are discussed in this thesis: First, finding the most efficient  Three research problems are discussed in this thesis: First, finding the most efficient
102  way to locate and fetch Fenfire data block from a Peer-to-Peer network when the block's  way to locate and fetch a Fenfire data block from a Peer-to-Peer network when the block's
103  identifier is given. Second, we want to find the most efficient way to locate and fetch the most  identifier is given. Second, finding the most efficient way to locate and fetch the most
104  recent Fenfire data block from a Peer-to-Peer network referred by a pointer. The third problem  recent Fenfire data block from a Peer-to-Peer network referenced to by a pointer. Finally, the
105  is similar to the second problem, except we want to locate and fetch the Fenfire  challenge of locating and fetching the Fenfire data block when a date and or time range is given.
 data block when a date and or time range is given.  
106    
107  In this thesis, we evaluate existing Peer-to-Peer approaches and  In this thesis, we evaluate existing Peer-to-Peer approaches and
108  evaluate them, based on Fenfire's needs. We start by reviewing existing Peer-to-Peer approaches,  evaluate them, based on Fenfire's needs. We start by reviewing existing Peer-to-Peer approaches,
109  algorithms and their key properties. Our insight is that, despite the great amount of proposed  algorithms and their key properties. Our insight is that, despite the great number of existing
110  Peer-to-Peer systems, we are able to classify \emph{all} systems either as loosely or  Peer-to-Peer systems, we are able to classify \emph{all} systems either as loosely or
111  tightly structured approaches. We also discuss open problems in  tightly structured approaches. We also discuss open problems in
112  Peer-to-Peer research and divide problems into three sub-categories: security, performance, and miscellaneous  Peer-to-Peer research and divide problems into three sub-categories: security, performance, and miscellaneous
113  problems. We attempt to comprehensively summarize existing algorithms and open problems in the  problems. We attempt to comprehensively summarize existing algorithms and open problems in the
114  Peer-to-Peer domain. This thesis does not provide detailed information about reviewed algorithms nor  Peer-to-Peer domain. This thesis does not provide detailed information about reviewed algorithms nor
115  open problems. More detailed information can be found from the references.  open problems. More detailed information can be found within references.
116    
117  Finally, we give an overview of the Fenfire project, and compare Peer-to-Peer approaches to Fenfire's  Finally, we give an overview of the Fenfire project, and compare Peer-to-Peer approaches to Fenfire's
118  needs. Finally, we propose simple yet efficient methods that could be used for data lookups in a Peer-to-Peer  needs. We propose simple yet efficient methods that could be used for data lookups in a Peer-to-Peer
119  environment.  environment.
120    
121  \chapter{Peer-to-Peer architectures}  \chapter{Peer-to-Peer architectures}
122  In this chapter we will give a brief history and overview of Peer-to-Peer networks,  In this chapter we provide a brief history and overview of Peer-to-Peer networks,
123  review the most important Peer-to-Peer algorithms and list key differences between the  review the most important Peer-to-Peer algorithms, and list key differences between the
124  two main approaches.  two main approaches.
125    
126  \section{Brief history and overview}  \section{Brief history and overview}
127    
128  The Internet was originally established in the late 1960s \cite{253741}. The objective  The Internet was originally established in the late 1960s \cite{253741}. The objective
129  of the ARPANET-project was to share information resources among military computers  of the ARPANET project was to share information resources among military computers
130  in the United States. The most challenging purpose of ARPANET was to integrate  in the United States. The most challenging purpose of ARPANET was to integrate
131  different kinds of existing network technologies with one common network architecture.  various kinds of existing network technologies with one common network architecture.
132  The ARPANET connected the first few hosts together not in client--server relationship,  The ARPANET connected the first few hosts together not in a client--server relationship,
133  but rather as equal networking \emph{peers}. This could be seen as the starting point  but rather as equal networking \emph{peers}. This could be seen as the starting point
134  of both the Peer-to-Peer concept and the Internet \cite{oram01harnessingpower}.  of both the Peer-to-Peer concept and the Internet \cite{oram01harnessingpower}.
135    
# Line 142  the network operates at the application Line 141  the network operates at the application
141  network with regard to the ISO-OSI reference model (e.g., \cite{800902}). Figure \ref{fig:application_level}  network with regard to the ISO-OSI reference model (e.g., \cite{800902}). Figure \ref{fig:application_level}
142  illustrates the Peer-to-Peer application level overlay network.  illustrates the Peer-to-Peer application level overlay network.
143  Compared to ARPANET's Peer-to-Peer functionality, modern Peer-to-Peer systems  Compared to ARPANET's Peer-to-Peer functionality, modern Peer-to-Peer systems
144  are \emph{ad hoc}, i.e., peers join and leave the system constantly. Thus, this property  are ad hoc, i.e., peers join and leave the system constantly. Thus, this property
145  poses challenges for efficient construction and maintenance  poses challenges for efficient construction and maintenance
146  of the overlay network, performing efficient data lookups and maintaining security in  of the overlay network, performing efficient data lookups and maintaining security in
147  a distributed environment.  a distributed environment.
# Line 157  a distributed environment. Line 156  a distributed environment.
156    
157    
158  In the development of modern Peer-to-Peer systems, many influences have come from  In the development of modern Peer-to-Peer systems, many influences have come from
159  outside of computer science. First, it is interesting to realize that chemical properties of biological cells, the Internet, ad hoc  outside of computer science. First, it is interesting to realize that the chemical properties of biological cells, the Internet, ad hoc
160  Peer-to-Peer systems, and social network self-organize based on the same  Peer-to-Peer systems, and social networks self-organize based on the same
161  principles \cite{albert-02-statistical, albert-00-tolerance, watts00dynamics}.  Second, the  principles \cite{albert-02-statistical, albert-00-tolerance, watts00dynamics}.  Second, the
162  association between social relationships among people and Peer-to-Peer overlay topology has been  association between social relationships among people and Peer-to-Peer overlay topology has been
163  recently studied \cite{watts00dynamics, kleinberg99small, nips02-Kleinberg}.  recently studied \cite{watts00dynamics, kleinberg99small, nips02-Kleinberg}.
164  This insight is motivated by Milgram \cite{milgram67smallworld}, who noticed that people very effectively  This insight is motivated by Milgram \cite{milgram67smallworld}, who noticed that people very effectively
165  locate other people on a wide geographic scale based on local knowledge. This phenomenon is called  locate other people on a wide geographic scale based on local knowledge. This phenomenon is called
166  ''small-world phenomenon''. As a consequence, many modern Peer-to-Peer systems  ''small-world phenomenon.'' As a consequence, many modern Peer-to-Peer systems
167  have applied similar techniques when constructing and maintaining the application level  have applied similar techniques when constructing and maintaining the application level
168  overlay network.  overlay network.
169    
# Line 175  and maintained. In the following section Line 174  and maintained. In the following section
174    
175  \section{Loosely structured}  \section{Loosely structured}
176    
177  In the loosely structured approach the construction and maintenance of the overlay is controlled  In the loosely structured approach, the construction and maintenance of the overlay is controlled
178  loosely. The placement of services and the topology of overlay is random. The data lookup model in loosely structured systems is  loosely. The placement of services and the topology of overlay is random. The data lookup model in loosely structured systems is
179  not very efficient because of unstructured properties of the overlay. The data lookup model is a combination of methods which  not very efficient because of unstructured properties of the overlay. The data lookup model is a combination of methods which
180  are used for locating data in the overlay.    are used for locating data in the overlay.  
181    
182  \subsection{Proposed definition}  \subsection{Proposed definition}
183    
184  In this subsection, we try to \emph{sketch out} a formal definition of the loosely structured overlay. This  In this subsection, we try to sketch a formal definition of the loosely structured overlay. This
185  model is based on the original Gnutella overlay network with power-law improvements. Please notice that the  model is based on the original Gnutella overlay network with power law improvements. Please notice that the
186  definition proposal is not used elsewhere in this thesis.  definition proposed is not used elsewhere in this thesis.
187    
188  Let $S$ be the aggregate of all services $s$ in the system. Let $P$ be the aggregate of  Let $S$ be the aggregate of all services $s$ in the system. Let $P$ be the aggregate of
189  all peers $p$ in the system. Then, $\forall s \in S$, there is a provider of the service,  all peers $p$ in the system. Then, $\forall s \in S$, there is a provider of the service,
# Line 209  directory, and had a single point of fai Line 208  directory, and had a single point of fai
208  Gnutella \cite{gnutellaurl} is a well-known example of loosely structured overlay system. Gnutella  Gnutella \cite{gnutellaurl} is a well-known example of loosely structured overlay system. Gnutella
209  is a pure Peer-to-Peer network as no peer is more important than any other peer in the network.  is a pure Peer-to-Peer network as no peer is more important than any other peer in the network.
210  The construction and maintenance of Gnutella network is extremely ad hoc, since participating  The construction and maintenance of Gnutella network is extremely ad hoc, since participating
211  peers can form the overlay network based on \emph{local} knowledge (i.e., a peer has no knowledge  peers can form the overlay network based on local knowledge (i.e., a peer has no knowledge
212  of global state of the system). Figure \ref{fig:gnutella_overlay}  of global state of the system). Figure \ref{fig:gnutella_overlay}
213  illustrates the overlay network of Gnutella network. The Gnutella network can be considered as a variation of power-law  illustrates the overlay network of Gnutella network. The Gnutella network can be considered as a variation of power-law
214  graph \cite{albert-02-statistical}. In power-law graphs only few peers have high  graph \cite{albert-02-statistical}. In power-law graphs only few peers have high
# Line 231  in the network can grow with $O(n^{2})$ Line 230  in the network can grow with $O(n^{2})$
230  Gnutella network. Figure \ref{fig:gnutella_query} illustrates why Gnutella's data lookup model has  Gnutella network. Figure \ref{fig:gnutella_query} illustrates why Gnutella's data lookup model has
231  $O(n^{2})$ properties.  $O(n^{2})$ properties.
232    
233  To limit the amount of network traffic, Gnutella uses Time-To-Live-limited  To limit the amount of network traffic, Gnutella uses Time-To-Live limited
234  (TTL) flooding to distribute queries. Therefore, Gnutella's data lookup algorithm is a Breadth-First-Search (BFS)  (TTL) flooding to distribute queries. Therefore, Gnutella's data lookup algorithm is a Breadth-First-Search (BFS)
235  with depth limit $T$ (e.g., 7), where $T$ is the system-wide maximum TTL of a message in hops. Thus,  with depth limit $T$ (e.g., 7), where $T$ is the system-wide maximum TTL of a message in hops. Thus,
236  only peers that are TTL hops away from the query originator will forward the query or respond to the query.  only peers that are TTL hops away from the query originator will forward the query or respond to the query.
# Line 246  every possible neighbor. Line 245  every possible neighbor.
245  \label{fig:gnutella_query}  \label{fig:gnutella_query}
246  \end{figure}  \end{figure}
247    
248  According to \cite{lv02searchreplication}, Gnutella's way to perform data lookups, \emph{flooding}, has the  According to Lv et al. \cite{lv02searchreplication}, Gnutella's way to perform data lookups, known as \emph{flooding}, has the
249  following limitations. First, choosing the appropriate TTL is not easy. If the  following limitations. First, choosing the appropriate TTL is not easy. If the
250  TTL is too high, the query originator may unnecessarily strain the network. If the TTL is too  TTL is too high, the query originator may unnecessarily strain the network. If the TTL is too
251  low, the query originator might not find the desired data even if it is available somewhere  low, the query originator might not find the desired data even if it is available somewhere
252  in the network. Second, there are many duplicate messages generated by flooding, especially  in the network. Second, there are many duplicate messages generated by flooding, especially
253  in high connectivity graphs. It is obvious that with these limitations, flooding creates  in high connectivity graphs. It is obvious that, with these limitations, flooding creates
254  significant message processing overhead for each data lookup. Even worse, flooding may increase  significant message processing overhead for each data lookup. Even worse, flooding may increase
255  the load on participating peer to the point where it has to leave the network.  the load on a participating peer to the point where it has to leave the network.
256    
257  Adamic et al. \cite{adamic99small, adamic02localsearch, adamic01powerlawsearch}  Adamic et al. \cite{adamic99small, adamic02localsearch, adamic01powerlawsearch}
258  have studied different data lookup methods in power-law networks and have found that by  have studied different data lookup methods in power-law networks and have found that by
# Line 288  these peers bear the load of the entire Line 287  these peers bear the load of the entire
287  \end{figure}      \end{figure}    
288    
289  The improvements presented above are only partial solutions. More advanced techniques  The improvements presented above are only partial solutions. More advanced techniques
290  to improve data lookup of loosely structured systems are discussed in chapter 3. Yet, however,  to improve data lookup of loosely structured systems are discussed in Chapter 3. However,
291  techniques presented in chapter 3 are not adopted in any loosely structured system.  techniques presented in Chapter 3 have not been adopted in any loosely structured system.
292    
293  \section{Tightly structured}  \section{Tightly structured}
294    
295  Partly due to scalability problems of loosely structured systems, several tightly  Due partly to scalability problems of loosely structured systems, several tightly
296  structured overlays have been proposed. In the tightly structured  structured overlays have been proposed. In the tightly structured
297  approach the overlay is constructed deterministically, which all participating peers have to follow; the topology of the  approach, the overlay is constructed deterministically, which all participating peers must follow; the topology of the
298  overlay and the placement of services is controlled tightly.  overlay and the placement of services is controlled tightly.
299    
300  \subsection{Proposed definition}  \subsection{Proposed definition}
301    
302  In this subsection, we try to \emph{sketch out} a formal definition of the tightly structured overlay, such as  In this subsection, we try to sketch a formal definition of the tightly structured overlay, such as
303  identifiers, identifier space and the mapping function. Please notice that the  identifiers, identifier space and the mapping function. Please notice that the
304  definition proposal is not used elsewhere in this thesis.  definition proposed is not used elsewhere in this thesis.
305    
306  Let $S$ be the aggregate of all services $s$ in the system. Let $P$ be the aggregate of  Let $S$ be the aggregate of all services $s$ in the system. Let $P$ be the aggregate of
307  all peers $p$ in the system. Let $I$ be the aggregate of all identifiers $i$ in the system.  all peers $p$ in the system. Let $I$ be the aggregate of all identifiers $i$ in the system.
# Line 318  where $\theta(p,p_n)$ = ''close'', and ' Line 317  where $\theta(p,p_n)$ = ''close'', and '
317    
318  \subsection{Systems}  \subsection{Systems}
319    
320  With tightly structured systems, it is feasible to efficiently perform \emph{global} data lookups in the overlay. By global lookup, we mean  With tightly structured systems, it is feasible to efficiently perform global data lookups in the overlay. By global lookup, we mean
321  that the system is able to find a service from the overlay, if it exists.  that the system is able to find a information from the overlay, if the information exists.
322  While there are significant differences among proposed tightly structured systems, they all have a common property,  While there are significant differences among proposed tightly structured systems, they all share common property:
323  in that \emph{peer identifiers} are assigned to participating peers from  peer identifiers are assigned to participating peers from
324  a large \emph{identifier space} by the overlay. Globally unique identifiers, \emph{keys},  a large identifier space by the overlay. Globally unique identifiers, known as \emph{keys},
325  are also assigned to application-specific data items  are also assigned to application-specific data items
326  which are selected from the same identifier space. For instance, globally unique keys can be created  that are selected from the same identifier space. For instance, globally unique keys can be created
327  using a cryptographic content hash function (e.g., SHA-1 \cite{fips-sha-1}) over the contents of a data item.  using a cryptographic content hash function (e.g., SHA-1 \cite{fips-sha-1}) over the contents of a data item.
328  The form of identifier space differs between proposed systems. A geometrical circular form of identifier space (and variants)  The form of identifier space differs between proposed systems. A geometrical circular form of identifier space (and variants)
329  is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde},  is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde},
330  Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry}  Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry}
331  and Viceroy \cite{malkhi02viceroy} use a circular form of identifier space of $n$-bit integers modulo $2^{n}$. The  and Viceroy \cite{malkhi02viceroy} use a circular form of identifier space of $n$-bit integers modulo $2^{n}$. The
332  value of $n$ varies among systems. Again, CAN \cite{ratnasamy01can} uses a $d$-dimensional geometrical torus  value of $n$ varies among systems. On the other hand, CAN \cite{ratnasamy01can} uses a $d$-dimensional geometrical torus
333  model to implement the form of identifier space.  model to implement the form of identifier space.
334    
335  To store data in a tightly structured overlay, each application-specific  To store data in a tightly structured overlay, each application-specific
336  unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent  unique key (e.g., SHA-1 \cite{fips-sha-1}) is mapped uniformly (e.g., using consistent
337  hashing \cite{258660}) to an existing peer in the overlay. Thus, a tightly  hashing \cite{258660}) to an existing peer in the overlay. Thus, a tightly
338  structured overlay assigns a subset of all possible keys to every participating peer.  structured overlay assigns a subset of all possible keys to every participating peer.
339  We say that a peer is \emph{responsible} for the keys which are assigned by the overlay.  We say that a peer is responsible for the keys that are assigned by the overlay.
340  Figure \ref{fig:structured_hashing} illustrates this  Figure \ref{fig:structured_hashing} illustrates this
341  process. Also, each peer in the tightly structured overlay maintains a \emph{routing table}, which  process. Also, each peer in the tightly structured overlay maintains a routing table, which
342  consists of identifiers and IP addresses of other peers in the overlay. Entries of the routing  consists of identifiers and IP addresses of other peers in the overlay. Entries of the routing
343  table represent peer's neighbors in the overlay network.  table represent peer's neighbors in the overlay network.
344    
# Line 350  table represent peer's neighbors in the Line 349  table represent peer's neighbors in the
349  \label{fig:structured_hashing}  \label{fig:structured_hashing}
350  \end{figure}  \end{figure}
351    
352  Currently, all proposed tightly structured overlays provide at least  Currently, all tightly structured overlays provide at least
353  poly--loga-rithmical data lookup operations. However, there are some key  polylogarithmical data lookup operations. However, there are some key
354  differences between the data structures representing the identifier space.  differences between the data structures representing the identifier space.
355  For example, Chord \cite{stoica01chord}, Skip graphs \cite{AspnesS2003} and SkipNet \cite{harvey03skipnet2} maintain a  For example, Chord \cite{stoica01chord}, Skip graphs \cite{AspnesS2003} and SkipNet \cite{harvey03skipnet2} maintain a
356  distributed data structure which resembles skip lists \cite{78977}.  distributed data structure that resembles skip lists \cite{78977}.
357  In figure \ref{fig:structured_query}, we present an overview of Chord's data lookup process.  In figure \ref{fig:structured_query}, we present an overview of Chord's data lookup process.
358  On the right side of Chord's lookup process, the same data lookup process  On the right side of Chord's lookup process, the same data lookup process
359  is shown as a binary-tree abstraction.  It can be seen, that in each step, the distance  is shown as a binary-tree abstraction.  It can be seen that, in each step, the distance
360  decreases with a logarithmic efficiency.  decreases with a logarithmic efficiency.
361    
362  \begin{figure}  \begin{figure}
# Line 368  decreases with a logarithmic efficiency. Line 367  decreases with a logarithmic efficiency.
367  \end{figure}  \end{figure}
368    
369  Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry  Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry
370  \cite{zhao01tapestry} use balanced $k$-trees to implement the data structure of the identifier space. Figure  \cite{zhao01tapestry} use balanced k-trees to implement the data structure of the identifier space. Figure
371  \ref{fig:kademlia_lookup} shows the process of Kademlia's  \ref{fig:kademlia_lookup} shows the process of Kademlia's
372  data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}),  data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}),
373  which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup  which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup
# Line 384  each peer to have only about two links t Line 383  each peer to have only about two links t
383  \label{fig:kademlia_lookup}  \label{fig:kademlia_lookup}
384  \end{figure}  \end{figure}
385    
386  Currently, there are only three higher level abstractions which are provided by the tightly structured overlays  Currently, only three higher level abstractions are provided by the tightly structured overlays
387  \cite{zhao03api}. Each of these abstractions represent a storage layer in the overlay, but  \cite{zhao03api}. Each of these abstractions represent a storage layer in the overlay, but
388  have semantical differences in the \emph{usage} of the overlay.  have semantical differences in the usage of the overlay.
389    
390  First, Distributed Hash Table (DHT) (see e.g., \cite{dabek01widearea}, \cite{rowstron01storage})    First, Distributed Hash Table (DHT) (see e.g., \cite{dabek01widearea}, \cite{rowstron01storage})  
391  implements the same functionality as a regular hash table by storing the mapping between a key and a value:  implements the same functionality as a regular hash table by storing the mapping between a key and a value:
# Line 397  implements the same functionality as a r Line 396  implements the same functionality as a r
396  \item \texttt{remove(key)}: remove a data item with a given key.  \item \texttt{remove(key)}: remove a data item with a given key.
397  \end{itemize}  \end{itemize}
398    
399  DHT's \emph{interface} is generic; values can be any size and type (e.g., content hash over a file). In the  DHT's interface is generic; values can be any size and type (e.g., content hash over a file). In the
400  DHT abstraction the overlay itself stores the data items. Figure \ref{fig:Structured_lookup_using_DHT_model} shows the DHT abstraction  DHT abstraction the overlay itself stores the data items. Figure \ref{fig:Structured_lookup_using_DHT_model} shows the DHT abstraction
401  of the tightly structured overlay.  of the tightly structured overlay.
402    
403  Second, Decentralized Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is a distributed  Second, Decentralized Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is a distributed
404  directory service. DOLR stores \emph{pointers} to data items throughout the overlay. DOLR's main  directory service. DOLR stores pointers to data items throughout the overlay. DOLR's main
405  operations are:  operations are:
406    
407  \begin{itemize}  \begin{itemize}
# Line 412  operations are: Line 411  operations are:
411  \end{itemize}  \end{itemize}
412    
413    
414  The key difference between the DHT and the DOLR abstraction is that, in the DOLR abstraction, the overlay maintains only \emph{pointers} to the data.  The key difference between the DHT and the DOLR abstraction is that, in the DOLR abstraction, the overlay maintains only pointers to the data.
415  Also, the DOLR abstraction routes overlay messages to the nearest available peer, hosting a specific data item. This form of locality  Also, the DOLR abstraction routes overlay messages to the nearest available peer hosting a specific data item. This form of locality
416  is not supported by DHT. DOLR's interface is similar to the DHT's interface, i.e., values can be any size and type.  is not supported by DHT. DOLR's interface is similar to the DHT's interface, i.e., values can be any size and type.
417    
418  Third, tightly structured overlays can be used for scalable group multicast or anycast operations (CAST) (see e.g., \cite{zhuang01bayeux}).  Third, tightly structured overlays can be used for scalable group multicast or anycast operations (CAST) (see e.g., \cite{zhuang01bayeux}).
# Line 447  In the DOLR abstraction, a data item is Line 446  In the DOLR abstraction, a data item is
446  \label{fig:Strucutred_lookup_using_DOLR_model}  \label{fig:Strucutred_lookup_using_DOLR_model}
447  \end{figure}  \end{figure}
448    
449  In tightly structured systems, messages are routed across the overlay towards peers, whose  In tightly structured systems, messages are routed across the overlay toward peers, whose
450  peer identifier is gradually ''closer'' to the key's identifier  peer identifier is gradually ''closer'' to the key's identifier
451  in the identifier space. The distance can be measured by numerical  in the identifier space. The distance can be measured by numerical
452  difference between identifiers (e.g., Chord \cite{stoica01chord}), number of  difference between identifiers (e.g., Chord \cite{stoica01chord}), number of
453  same prefix bits between identifiers (e.g., Pastry \cite{rowston01pastry} and Tapestry  same prefix bits between identifiers (e.g., Pastry \cite{rowston01pastry} and Tapestry
454  \cite{zhao01tapestry}) or bit-wise exclusive or (XOR) (e.g., Kademlia \cite{maymounkov02kademlia}).  \cite{zhao01tapestry}) or Bit-Wise Exclusive Or (XOR) (e.g., Kademlia \cite{maymounkov02kademlia}).
455  Chord's \cite{stoica01chord} distance function does have the property of unidirection  Chord's \cite{stoica01chord} distance function does have the property of unidirection
456  (for a given point $p_i$ in the identifier space and distance $d$ > 0, there  (for a given point $p_i$ in the identifier space and distance $d$ > 0, there
457  is exactly one point $p_j$ in a way that the distance between $p_i$ and $p_j$  is exactly one point $p_j$ in a way that the distance between $p_i$ and $p_j$
458  is $d$), but does not have symmetry (the distance from $p_i$ to $p_j$ is same as the  is $d$), but does not have symmetry (the distance from $p_i$ to $p_j$ is same as the
459  distance from $p_j$ to $p_i$). Pastry's \cite{rowston01pastry} distance function supports  distance from $p_j$ to $p_i$). Pastry's \cite{rowston01pastry} distance function supports
460  symmetry, but does not support unidirection. According to \cite{balakrishanarticle03lookupp2p}, because  symmetry, but does not support unidirection. According to Balakrishnan et al. \cite{balakrishanarticle03lookupp2p},
461  of XOR-metric, Kademlia's distance function is both unidirectional and symmetric. Moreover, Kademlia's \cite{maymounkov02kademlia}  Kademlia's distance function is both unidirectional and symmetric because of the XOR-metric.
462    Moreover, Kademlia's \cite{maymounkov02kademlia}
463  XOR-based metric does not need stabilization (like in Chord \cite{stoica01chord}) and backup links  XOR-based metric does not need stabilization (like in Chord \cite{stoica01chord}) and backup links
464  (like in Pastry \cite{rowston01pastry}).  (like in Pastry \cite{rowston01pastry}).
465  However, in all of the above schemes, each hop in the overlay shortens the distance between  However, in all of the above schemes, each hop in the overlay shortens the distance between
466  current peer working with the data lookup and the key that was looked up in the identifier space.  current peer working with the data lookup and the key that was looked up in the identifier space.
467    
468  Skip Graphs \cite{AspnesS2003} and SWAN \cite{bonsma02swan} employ a identifier space    Skip Graphs \cite{AspnesS2003} and SWAN \cite{bonsma02swan} employ a identifier space  
469  in which queries are routed to \emph{keys}. In these systems  in which queries are routed to keys. In these systems
470  a peer occupies several positions in the identifier space, one for each  a peer occupies several positions in the identifier space, one for each
471  application-specific key. The indirection of placing close keys in the  application-specific key. The opposite action of placing close keys in the
472  custody of a provider peer is removed at the cost of each peer maintaining one  custody of a provider peer is removed at the cost of each peer maintaining one
473  ''resource peer'' in the overlay network for each data item it publishes. The provider peer is a peer  ''resource peer'' in the overlay network for each data item it publishes. The provider peer is a peer
474  which has initially published services into the overlay.  that has initially published services into the overlay.
475    
476  PeerNet \cite{eriksson03peernet} differs from other tightly structured overlays in that it operates  PeerNet \cite{eriksson03peernet} differs from other tightly structured overlays in that it operates
477  at the \emph{network} layer instead of application layer (see the ISO-OSI reference model, e.g., \cite{800902}).  at the \emph{network} layer instead of application layer (see the ISO-OSI reference model \cite{800902}).
478  This property would provide a common interface  This property would provide a common interface
479  to all Peer-to-Peer systems using PeerNet. PeerNet makes an explicit distinction  to all Peer-to-Peer systems using PeerNet. PeerNet makes an explicit distinction
480  between peer identity and address, which is not supported by standard  between peer identity and address, which is not supported by standard
# Line 483  overlays, i.e., $O(\log{n})$ space requi Line 483  overlays, i.e., $O(\log{n})$ space requi
483  the system and $O(\log{n})$ data lookup efficiency.  the system and $O(\log{n})$ data lookup efficiency.
484    
485  Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} have listed four requirements  Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} have listed four requirements
486  for tightly structured overlays\footnote{Authors use the term 'DHT' in their text, but in this context  for tightly structured overlays\footnote{The authors use the term 'DHT' in their text, but in this paper
487  it does not matter as they list \emph{general} properties of tightly structured overlays.} that have to be addressed in order  it is not meant to be specific as the authors list \emph{general} properties of tightly structured overlays.} that have to be addressed in order
488  to perform efficient data lookups in tightly structured overlays.  to perform efficient data lookups in tightly structured overlays.
489  First, mapping of keys to peers must be done in a load-balanced  First, the mapping of keys to peers must be done in a load-balanced
490  way. Second, the overlay must be able to forward a data lookup for a  way. Second, the overlay must be able to forward a data lookup for a
491  specific key to an appropriate peer. Third, overlay must  specific key to an appropriate peer. Third, the overlay must
492  support efficient distance function. Finally,  routing tables for each peer  support efficient distance function. Finally, the routing tables for each peer
493  must be constructed and maintained adaptively.  must be constructed and maintained adaptively.
494    
495  Additionally, authors argue in \cite{balakrishnan03semanticfree} that tightly structured systems  Additionally, Balakrishnan et al. argue \cite{balakrishnan03semanticfree} that tightly structured systems
496  are suitable for next generation Reference Resolutions Services (RRS)\footnote{  are suitable for next generation Reference Resolutions Services (RRS)\footnote{
497  Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system in the Internet.}. They present  Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system on the Internet.}. They present
498  two requirements about the nature of reference resolution. First, there should be a general-purpose  two requirements about the nature of reference resolution. First, there should be a general-purpose
499  and application-indepedent substrate for reference resolution. Second, the references themselves  and application-indepedent substratum for reference resolution. Second, the references themselves
500  should be unstructured and semantic-free. In this text, we define unstructured reference  should be unstructured and semantic-free. In this paper, we define a unstructured reference
501  as a reference that does not expose the target in any way and semantic-free reference as a reference  as one that does not expose the target in any way and a semantic-free reference as a reference
502  that there are no directives in the reference itself which would expose how the reference should be processed.  that has no directives in the reference itself that would expose how the reference should be processed.
503    
504    
505  \section{Differences}  \section{Differences}
506    
507  Even though the loosely structured and the tightly structured approach are both Peer-to-Peer schemes, they  Even though the loosely structured and the tightly structured approaches are both Peer-to-Peer schemes, they
508  have very little in common. Indeed, the only thing they share is the fact that no other peer is more  have very little in common. Indeed, the only similarity they share is the fact that no one peer is more
509  important than any other in the Peer-to-Peer network. Fault tolerance \emph{may}  important than another within the Peer-to-Peer network. Fault tolerance \emph{may}
510  be an area in which approaches have similar properties (e.g., no single point of failure) \cite{milojicic02peertopeer}.  be an other area in which these approaches have similar properties (e.g., no single point of failure) \cite{milojicic02peertopeer}.
511  Fault tolerance properties of both approaches are currently only initial calculations, or  The fault tolerance properties of both approaches are currently only initial calculations, or
512  experimented in simulation environments. In real life, however, measuring fault tolerance is a much more  experimented in simulation environments. In practical applications, however, measuring fault tolerance is a much more
513  challenging task and requires more research to get reliable answers.  challenging task and requires additional research to obtain reliable answers.
514    
515  The most important differences between approaches are the performance and scalability properties.  The most important differences between the approaches are in the performance and scalability properties.
516  Generally tightly structured systems can perform all internal operations in poly--logarithmic time  Generally, tightly structured systems can perform all internal operations in polylogarithmic time,
517  while the performance of loosely structured systems is not always even linear \cite{balakrishanarticle03lookupp2p}.  while the performance of loosely structured systems is not always even linear \cite{balakrishanarticle03lookupp2p}.
518  Moreover, loosely structured systems scale to millions of peers,  Moreover, loosely structured systems scale to millions of peers,
519  whereas tightly structured systems are able to cope with billions of concurrent  whereas tightly structured systems are able to cope with billions of concurrent
520  peers \cite{osokine02distnetworks}, \cite{kubiatowicz00oceanstore}. However, it is unknown  peers \cite{osokine02distnetworks}, \cite{kubiatowicz00oceanstore}. However, it is unknown
521  whether all proposed algorithms can preserve logarithmic efficiency and scalability properties  whether or not all proposed algorithms can preserve logarithmic efficiency and scalability properties
522  in real-life applications or not; several tightly structured systems  in real-life applications; several tightly structured systems
523  assume that participating peers are homogeneous, and the rate of join or leave operation is low \cite{gurmeet03symphony,  assume that participating peers are homogeneous, and the rate of join or leave operation is low \cite{gurmeet03symphony,
524  libennowell01observations, rowston03controlloingreliability}.    libennowell01observations, rowston03controlloingreliability}.  
525    
526  To the end user, the biggest difference between these systems is how data lookups are performed. Loosely  For the end user, the biggest difference between these systems is how data lookups are performed. Loosely
527  structured systems provide a more rich and user friendly way of searching data than tightly structured systems  structured systems provide a more rich and user friendly way of searching for data than do tightly structured systems,
528  as they have a support for keyword searches \cite{yang02efficientsearch, lv02searchreplication}. Tightly structured  as the former have a support for keyword searches \cite{yang02efficientsearch, lv02searchreplication}. Tightly structured
529  systems support only exact key lookups since each data item is identified by globally unique keys \cite{balakrishanarticle03lookupp2p,  systems support only exact key lookups since each data item is identified by globally unique keys \cite{balakrishanarticle03lookupp2p,
530  harren02complex, ansaryefficientbroadcast03}.  harren02complex, ansaryefficientbroadcast03}.
531    
532  Table \ref{table_comparison_approach} lists the key differences between the loosely structured  Table \ref{table_comparison_approach} lists the primary differences between the loosely structured
533  approach and the tightly structured approach.  approach and the tightly structured approach.
534    
535    
# Line 613  approach and the tightly structured appr Line 613  approach and the tightly structured appr
613  Table \ref{table_Peer-to-Peer_algorithms} lists proposed Peer-to-Peer algorithms  Table \ref{table_Peer-to-Peer_algorithms} lists proposed Peer-to-Peer algorithms
614  and their key properties with regard to performance and scalability. The list  and their key properties with regard to performance and scalability. The list
615  includes algorithms from both loosely and tightly structured approaches. The list does not  includes algorithms from both loosely and tightly structured approaches. The list does not
616  include \emph{all} proposed Peer-to-Peer algorithms but rather includes the ones which already have  include \emph{all} proposed Peer-to-Peer algorithms but rather includes the ones that already have
617  been widely deployed, or the ones which may be promising in the future  been widely deployed or which show promise for future Peer-to-Peer systems.
 Peer-to-Peer systems.  
618    
619  We decided to follow the guidelines from \cite{kaashoek03koorde} in measuring  We decided to follow the guidelines provided by Kaashoek et al. \cite{kaashoek03koorde} in measuring
620  the properties of different Peer-to-Peer systems. However, we dropped  the properties of various Peer-to-Peer systems. We eliminated
621  out fault tolerance and load balancing properties, since they are hard to measure  out fault tolerance and load balancing properties from consideration since they are difficult to measure
622  in real life requirements. Additionally, however, we decided to include  in real life applications. However, we decided to include
623  the number of \emph{real} network connections for each peer in the overlay. Next,  a number of real network connections for each peer in the overlay. Here,
624  we describe the listed properties of Peer-to-Peer algorithms:  we describe the listed properties of Peer-to-Peer algorithms:
625    
626  \begin{itemize}  \begin{itemize}

Legend:
Removed from v.1.207  
changed lines
  Added in v.1.208

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26