42 |
In this thesis, first we review existing Peer-to-Peer approaches, algorithms and their |
In this thesis, first we review existing Peer-to-Peer approaches, algorithms and their |
43 |
key properties. We summarize open problems in Peer-to-Peer systems and divide these |
key properties. We summarize open problems in Peer-to-Peer systems and divide these |
44 |
problems into three sub-categories. We realize that there are many problems and few |
problems into three sub-categories. We realize that there are many problems and few |
45 |
practical solutions, and some problems have no solution at all. |
practical solutions, and recognize that some problems have no solution at all. |
46 |
|
|
47 |
Then, we provide an overview of the Fenfire system. The Fenfire system is a free |
Then, we provide an overview of the Fenfire system. The Fenfire system is a free |
48 |
software effort to build a location transparent, hyperstructured desktop environment. |
software effort to build a location transparent, hyperstructured desktop environment. |
49 |
We evaluate existing Peer-to-Peer approaches-- loosely and tightly structured overlays-- with regard |
We evaluate existing Peer-to-Peer approaches--loosely and tightly structured overlays--with regard |
50 |
to Fenfire's needs. Finally, we propose simple methods to efficiently locate Fenfire |
to Fenfire's needs. Finally, we propose simple methods to efficiently locate Fenfire |
51 |
data from Peer-to-Peer networks. |
data within Peer-to-Peer networks. |
52 |
} |
} |
53 |
\tiivistelma{ |
\tiivistelma{ |
54 |
Tässä opinnäytetyössä esittelemme olemassaolevia vertaisverkkoja, algoritmeja ja |
Tässä opinnäytetyössä esittelemme olemassaolevia vertaisverkkoja, algoritmeja ja |
83 |
academia \cite{projectirisurl} and industry \cite{p2pworkinggroup, jxtaurl} for a |
academia \cite{projectirisurl} and industry \cite{p2pworkinggroup, jxtaurl} for a |
84 |
number of reasons. The lack of centralization in Peer-to-Peer systems |
number of reasons. The lack of centralization in Peer-to-Peer systems |
85 |
means that the participants can form a distributed system \cite{couloris94distributedsystems} |
means that the participants can form a distributed system \cite{couloris94distributedsystems} |
86 |
without any investment to centralized hardware by sharing their services and connecting to each |
without any investment in centralized hardware by sharing their services and by connecting to each |
87 |
other directly. Peer-to-Peer systems can be characterized as distributed systems in which all |
other directly. Peer-to-Peer systems can be characterized as distributed systems in which all |
88 |
communication is symmetric and all participant entities have similar capabilities and responsibilities |
communication is symmetrical and all participant entities have similar capabilities and responsibilities |
89 |
\cite{oram01harnessingpower}. Schollmeier \cite{schollmeier01p2pdefinition} describes a Peer-to-Peer system as a system of |
\cite{oram01harnessingpower}. Schollmeier \cite{schollmeier01p2pdefinition} describes a Peer-to-Peer system as a system of |
90 |
distributed entities that share their own services. |
distributed entities that share their own services. |
91 |
Each entity, i.e., \emph{peer}, may contribute services to the overall system. The distributed |
Each entity, i.e., \emph{peer}, may contribute services to the overall system. The distributed |
92 |
and ad hoc nature of Peer-to-Peer improves scalability and avoids single points of failure. |
and ad hoc nature of Peer-to-Peer improves scalability and avoids single points of failure. |
93 |
|
|
94 |
The Fenfire project is an attempt to build a hyperstructured, seamlessly interoperating desktop |
The Fenfire project is an attempt to build a hyperstructured, seamlessly interoperating desktop |
95 |
environment. In the Fenfire system, all data is stored as blocks. |
environment. In the Fenfire system, all data is stored as "blocks". |
96 |
Each block has a globally unique identifier and it can be referred, by pointer blocks. |
Each block has a globally unique identifier and it can be referenced to, by pointer blocks. |
97 |
Other features of Fenfire include innovative user |
Other features of Fenfire include innovative user |
98 |
interfaces for viewing data. The applicability of Peer-to-Peer networking with Fenfire for network |
interfaces for viewing data. The applicability of Peer-to-Peer networking with Fenfire for network |
99 |
transparency is currently under investigation. |
transparency is currently under investigation. |
100 |
|
|
101 |
Three research problems are discussed in this thesis: First, finding the most efficient |
Three research problems are discussed in this thesis: First, finding the most efficient |
102 |
way to locate and fetch Fenfire data block from a Peer-to-Peer network when the block's |
way to locate and fetch a Fenfire data block from a Peer-to-Peer network when the block's |
103 |
identifier is given. Second, we want to find the most efficient way to locate and fetch the most |
identifier is given. Second, finding the most efficient way to locate and fetch the most |
104 |
recent Fenfire data block from a Peer-to-Peer network referred by a pointer. The third problem |
recent Fenfire data block from a Peer-to-Peer network referenced to by a pointer. Finally, the |
105 |
is similar to the second problem, except we want to locate and fetch the Fenfire |
challenge of locating and fetching the Fenfire data block when a date and or time range is given. |
|
data block when a date and or time range is given. |
|
106 |
|
|
107 |
In this thesis, we evaluate existing Peer-to-Peer approaches and |
In this thesis, we evaluate existing Peer-to-Peer approaches and |
108 |
evaluate them, based on Fenfire's needs. We start by reviewing existing Peer-to-Peer approaches, |
evaluate them, based on Fenfire's needs. We start by reviewing existing Peer-to-Peer approaches, |
109 |
algorithms and their key properties. Our insight is that, despite the great amount of proposed |
algorithms and their key properties. Our insight is that, despite the great number of existing |
110 |
Peer-to-Peer systems, we are able to classify \emph{all} systems either as loosely or |
Peer-to-Peer systems, we are able to classify \emph{all} systems either as loosely or |
111 |
tightly structured approaches. We also discuss open problems in |
tightly structured approaches. We also discuss open problems in |
112 |
Peer-to-Peer research and divide problems into three sub-categories: security, performance, and miscellaneous |
Peer-to-Peer research and divide problems into three sub-categories: security, performance, and miscellaneous |
113 |
problems. We attempt to comprehensively summarize existing algorithms and open problems in the |
problems. We attempt to comprehensively summarize existing algorithms and open problems in the |
114 |
Peer-to-Peer domain. This thesis does not provide detailed information about reviewed algorithms nor |
Peer-to-Peer domain. This thesis does not provide detailed information about reviewed algorithms nor |
115 |
open problems. More detailed information can be found from the references. |
open problems. More detailed information can be found within references. |
116 |
|
|
117 |
Finally, we give an overview of the Fenfire project, and compare Peer-to-Peer approaches to Fenfire's |
Finally, we give an overview of the Fenfire project, and compare Peer-to-Peer approaches to Fenfire's |
118 |
needs. Finally, we propose simple yet efficient methods that could be used for data lookups in a Peer-to-Peer |
needs. We propose simple yet efficient methods that could be used for data lookups in a Peer-to-Peer |
119 |
environment. |
environment. |
120 |
|
|
121 |
\chapter{Peer-to-Peer architectures} |
\chapter{Peer-to-Peer architectures} |
122 |
In this chapter we will give a brief history and overview of Peer-to-Peer networks, |
In this chapter we provide a brief history and overview of Peer-to-Peer networks, |
123 |
review the most important Peer-to-Peer algorithms and list key differences between the |
review the most important Peer-to-Peer algorithms, and list key differences between the |
124 |
two main approaches. |
two main approaches. |
125 |
|
|
126 |
\section{Brief history and overview} |
\section{Brief history and overview} |
127 |
|
|
128 |
The Internet was originally established in the late 1960s \cite{253741}. The objective |
The Internet was originally established in the late 1960s \cite{253741}. The objective |
129 |
of the ARPANET-project was to share information resources among military computers |
of the ARPANET project was to share information resources among military computers |
130 |
in the United States. The most challenging purpose of ARPANET was to integrate |
in the United States. The most challenging purpose of ARPANET was to integrate |
131 |
different kinds of existing network technologies with one common network architecture. |
various kinds of existing network technologies with one common network architecture. |
132 |
The ARPANET connected the first few hosts together not in client--server relationship, |
The ARPANET connected the first few hosts together not in a client--server relationship, |
133 |
but rather as equal networking \emph{peers}. This could be seen as the starting point |
but rather as equal networking \emph{peers}. This could be seen as the starting point |
134 |
of both the Peer-to-Peer concept and the Internet \cite{oram01harnessingpower}. |
of both the Peer-to-Peer concept and the Internet \cite{oram01harnessingpower}. |
135 |
|
|
141 |
network with regard to the ISO-OSI reference model (e.g., \cite{800902}). Figure \ref{fig:application_level} |
network with regard to the ISO-OSI reference model (e.g., \cite{800902}). Figure \ref{fig:application_level} |
142 |
illustrates the Peer-to-Peer application level overlay network. |
illustrates the Peer-to-Peer application level overlay network. |
143 |
Compared to ARPANET's Peer-to-Peer functionality, modern Peer-to-Peer systems |
Compared to ARPANET's Peer-to-Peer functionality, modern Peer-to-Peer systems |
144 |
are \emph{ad hoc}, i.e., peers join and leave the system constantly. Thus, this property |
are ad hoc, i.e., peers join and leave the system constantly. Thus, this property |
145 |
poses challenges for efficient construction and maintenance |
poses challenges for efficient construction and maintenance |
146 |
of the overlay network, performing efficient data lookups and maintaining security in |
of the overlay network, performing efficient data lookups and maintaining security in |
147 |
a distributed environment. |
a distributed environment. |
156 |
|
|
157 |
|
|
158 |
In the development of modern Peer-to-Peer systems, many influences have come from |
In the development of modern Peer-to-Peer systems, many influences have come from |
159 |
outside of computer science. First, it is interesting to realize that chemical properties of biological cells, the Internet, ad hoc |
outside of computer science. First, it is interesting to realize that the chemical properties of biological cells, the Internet, ad hoc |
160 |
Peer-to-Peer systems, and social network self-organize based on the same |
Peer-to-Peer systems, and social networks self-organize based on the same |
161 |
principles \cite{albert-02-statistical, albert-00-tolerance, watts00dynamics}. Second, the |
principles \cite{albert-02-statistical, albert-00-tolerance, watts00dynamics}. Second, the |
162 |
association between social relationships among people and Peer-to-Peer overlay topology has been |
association between social relationships among people and Peer-to-Peer overlay topology has been |
163 |
recently studied \cite{watts00dynamics, kleinberg99small, nips02-Kleinberg}. |
recently studied \cite{watts00dynamics, kleinberg99small, nips02-Kleinberg}. |
164 |
This insight is motivated by Milgram \cite{milgram67smallworld}, who noticed that people very effectively |
This insight is motivated by Milgram \cite{milgram67smallworld}, who noticed that people very effectively |
165 |
locate other people on a wide geographic scale based on local knowledge. This phenomenon is called |
locate other people on a wide geographic scale based on local knowledge. This phenomenon is called |
166 |
''small-world phenomenon''. As a consequence, many modern Peer-to-Peer systems |
''small-world phenomenon.'' As a consequence, many modern Peer-to-Peer systems |
167 |
have applied similar techniques when constructing and maintaining the application level |
have applied similar techniques when constructing and maintaining the application level |
168 |
overlay network. |
overlay network. |
169 |
|
|
174 |
|
|
175 |
\section{Loosely structured} |
\section{Loosely structured} |
176 |
|
|
177 |
In the loosely structured approach the construction and maintenance of the overlay is controlled |
In the loosely structured approach, the construction and maintenance of the overlay is controlled |
178 |
loosely. The placement of services and the topology of overlay is random. The data lookup model in loosely structured systems is |
loosely. The placement of services and the topology of overlay is random. The data lookup model in loosely structured systems is |
179 |
not very efficient because of unstructured properties of the overlay. The data lookup model is a combination of methods which |
not very efficient because of unstructured properties of the overlay. The data lookup model is a combination of methods which |
180 |
are used for locating data in the overlay. |
are used for locating data in the overlay. |
181 |
|
|
182 |
\subsection{Proposed definition} |
\subsection{Proposed definition} |
183 |
|
|
184 |
In this subsection, we try to \emph{sketch out} a formal definition of the loosely structured overlay. This |
In this subsection, we try to sketch a formal definition of the loosely structured overlay. This |
185 |
model is based on the original Gnutella overlay network with power-law improvements. Please notice that the |
model is based on the original Gnutella overlay network with power law improvements. Please notice that the |
186 |
definition proposal is not used elsewhere in this thesis. |
definition proposed is not used elsewhere in this thesis. |
187 |
|
|
188 |
Let $S$ be the aggregate of all services $s$ in the system. Let $P$ be the aggregate of |
Let $S$ be the aggregate of all services $s$ in the system. Let $P$ be the aggregate of |
189 |
all peers $p$ in the system. Then, $\forall s \in S$, there is a provider of the service, |
all peers $p$ in the system. Then, $\forall s \in S$, there is a provider of the service, |
208 |
Gnutella \cite{gnutellaurl} is a well-known example of loosely structured overlay system. Gnutella |
Gnutella \cite{gnutellaurl} is a well-known example of loosely structured overlay system. Gnutella |
209 |
is a pure Peer-to-Peer network as no peer is more important than any other peer in the network. |
is a pure Peer-to-Peer network as no peer is more important than any other peer in the network. |
210 |
The construction and maintenance of Gnutella network is extremely ad hoc, since participating |
The construction and maintenance of Gnutella network is extremely ad hoc, since participating |
211 |
peers can form the overlay network based on \emph{local} knowledge (i.e., a peer has no knowledge |
peers can form the overlay network based on local knowledge (i.e., a peer has no knowledge |
212 |
of global state of the system). Figure \ref{fig:gnutella_overlay} |
of global state of the system). Figure \ref{fig:gnutella_overlay} |
213 |
illustrates the overlay network of Gnutella network. The Gnutella network can be considered as a variation of power-law |
illustrates the overlay network of Gnutella network. The Gnutella network can be considered as a variation of power-law |
214 |
graph \cite{albert-02-statistical}. In power-law graphs only few peers have high |
graph \cite{albert-02-statistical}. In power-law graphs only few peers have high |
230 |
Gnutella network. Figure \ref{fig:gnutella_query} illustrates why Gnutella's data lookup model has |
Gnutella network. Figure \ref{fig:gnutella_query} illustrates why Gnutella's data lookup model has |
231 |
$O(n^{2})$ properties. |
$O(n^{2})$ properties. |
232 |
|
|
233 |
To limit the amount of network traffic, Gnutella uses Time-To-Live-limited |
To limit the amount of network traffic, Gnutella uses Time-To-Live limited |
234 |
(TTL) flooding to distribute queries. Therefore, Gnutella's data lookup algorithm is a Breadth-First-Search (BFS) |
(TTL) flooding to distribute queries. Therefore, Gnutella's data lookup algorithm is a Breadth-First-Search (BFS) |
235 |
with depth limit $T$ (e.g., 7), where $T$ is the system-wide maximum TTL of a message in hops. Thus, |
with depth limit $T$ (e.g., 7), where $T$ is the system-wide maximum TTL of a message in hops. Thus, |
236 |
only peers that are TTL hops away from the query originator will forward the query or respond to the query. |
only peers that are TTL hops away from the query originator will forward the query or respond to the query. |
245 |
\label{fig:gnutella_query} |
\label{fig:gnutella_query} |
246 |
\end{figure} |
\end{figure} |
247 |
|
|
248 |
According to \cite{lv02searchreplication}, Gnutella's way to perform data lookups, \emph{flooding}, has the |
According to Lv et al. \cite{lv02searchreplication}, Gnutella's way to perform data lookups, known as \emph{flooding}, has the |
249 |
following limitations. First, choosing the appropriate TTL is not easy. If the |
following limitations. First, choosing the appropriate TTL is not easy. If the |
250 |
TTL is too high, the query originator may unnecessarily strain the network. If the TTL is too |
TTL is too high, the query originator may unnecessarily strain the network. If the TTL is too |
251 |
low, the query originator might not find the desired data even if it is available somewhere |
low, the query originator might not find the desired data even if it is available somewhere |
252 |
in the network. Second, there are many duplicate messages generated by flooding, especially |
in the network. Second, there are many duplicate messages generated by flooding, especially |
253 |
in high connectivity graphs. It is obvious that with these limitations, flooding creates |
in high connectivity graphs. It is obvious that, with these limitations, flooding creates |
254 |
significant message processing overhead for each data lookup. Even worse, flooding may increase |
significant message processing overhead for each data lookup. Even worse, flooding may increase |
255 |
the load on participating peer to the point where it has to leave the network. |
the load on a participating peer to the point where it has to leave the network. |
256 |
|
|
257 |
Adamic et al. \cite{adamic99small, adamic02localsearch, adamic01powerlawsearch} |
Adamic et al. \cite{adamic99small, adamic02localsearch, adamic01powerlawsearch} |
258 |
have studied different data lookup methods in power-law networks and have found that by |
have studied different data lookup methods in power-law networks and have found that by |
287 |
\end{figure} |
\end{figure} |
288 |
|
|
289 |
The improvements presented above are only partial solutions. More advanced techniques |
The improvements presented above are only partial solutions. More advanced techniques |
290 |
to improve data lookup of loosely structured systems are discussed in chapter 3. Yet, however, |
to improve data lookup of loosely structured systems are discussed in Chapter 3. However, |
291 |
techniques presented in chapter 3 are not adopted in any loosely structured system. |
techniques presented in Chapter 3 have not been adopted in any loosely structured system. |
292 |
|
|
293 |
\section{Tightly structured} |
\section{Tightly structured} |
294 |
|
|
295 |
Partly due to scalability problems of loosely structured systems, several tightly |
Due partly to scalability problems of loosely structured systems, several tightly |
296 |
structured overlays have been proposed. In the tightly structured |
structured overlays have been proposed. In the tightly structured |
297 |
approach the overlay is constructed deterministically, which all participating peers have to follow; the topology of the |
approach, the overlay is constructed deterministically, which all participating peers must follow; the topology of the |
298 |
overlay and the placement of services is controlled tightly. |
overlay and the placement of services is controlled tightly. |
299 |
|
|
300 |
\subsection{Proposed definition} |
\subsection{Proposed definition} |
301 |
|
|
302 |
In this subsection, we try to \emph{sketch out} a formal definition of the tightly structured overlay, such as |
In this subsection, we try to sketch a formal definition of the tightly structured overlay, such as |
303 |
identifiers, identifier space and the mapping function. Please notice that the |
identifiers, identifier space and the mapping function. Please notice that the |
304 |
definition proposal is not used elsewhere in this thesis. |
definition proposed is not used elsewhere in this thesis. |
305 |
|
|
306 |
Let $S$ be the aggregate of all services $s$ in the system. Let $P$ be the aggregate of |
Let $S$ be the aggregate of all services $s$ in the system. Let $P$ be the aggregate of |
307 |
all peers $p$ in the system. Let $I$ be the aggregate of all identifiers $i$ in the system. |
all peers $p$ in the system. Let $I$ be the aggregate of all identifiers $i$ in the system. |
317 |
|
|
318 |
\subsection{Systems} |
\subsection{Systems} |
319 |
|
|
320 |
With tightly structured systems, it is feasible to efficiently perform \emph{global} data lookups in the overlay. By global lookup, we mean |
With tightly structured systems, it is feasible to efficiently perform global data lookups in the overlay. By global lookup, we mean |
321 |
that the system is able to find a service from the overlay, if it exists. |
that the system is able to find a information from the overlay, if the information exists. |
322 |
While there are significant differences among proposed tightly structured systems, they all have a common property, |
While there are significant differences among proposed tightly structured systems, they all share common property: |
323 |
in that \emph{peer identifiers} are assigned to participating peers from |
peer identifiers are assigned to participating peers from |
324 |
a large \emph{identifier space} by the overlay. Globally unique identifiers, \emph{keys}, |
a large identifier space by the overlay. Globally unique identifiers, known as \emph{keys}, |
325 |
are also assigned to application-specific data items |
are also assigned to application-specific data items |
326 |
which are selected from the same identifier space. For instance, globally unique keys can be created |
that are selected from the same identifier space. For instance, globally unique keys can be created |
327 |
using a cryptographic content hash function (e.g., SHA-1 \cite{fips-sha-1}) over the contents of a data item. |
using a cryptographic content hash function (e.g., SHA-1 \cite{fips-sha-1}) over the contents of a data item. |
328 |
The form of identifier space differs between proposed systems. A geometrical circular form of identifier space (and variants) |
The form of identifier space differs between proposed systems. A geometrical circular form of identifier space (and variants) |
329 |
is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde}, |
is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde}, |
330 |
Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry} |
Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry} |
331 |
and Viceroy \cite{malkhi02viceroy} use a circular form of identifier space of $n$-bit integers modulo $2^{n}$. The |
and Viceroy \cite{malkhi02viceroy} use a circular form of identifier space of $n$-bit integers modulo $2^{n}$. The |
332 |
value of $n$ varies among systems. Again, CAN \cite{ratnasamy01can} uses a $d$-dimensional geometrical torus |
value of $n$ varies among systems. On the other hand, CAN \cite{ratnasamy01can} uses a $d$-dimensional geometrical torus |
333 |
model to implement the form of identifier space. |
model to implement the form of identifier space. |
334 |
|
|
335 |
To store data in a tightly structured overlay, each application-specific |
To store data in a tightly structured overlay, each application-specific |
336 |
unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent |
unique key (e.g., SHA-1 \cite{fips-sha-1}) is mapped uniformly (e.g., using consistent |
337 |
hashing \cite{258660}) to an existing peer in the overlay. Thus, a tightly |
hashing \cite{258660}) to an existing peer in the overlay. Thus, a tightly |
338 |
structured overlay assigns a subset of all possible keys to every participating peer. |
structured overlay assigns a subset of all possible keys to every participating peer. |
339 |
We say that a peer is \emph{responsible} for the keys which are assigned by the overlay. |
We say that a peer is responsible for the keys that are assigned by the overlay. |
340 |
Figure \ref{fig:structured_hashing} illustrates this |
Figure \ref{fig:structured_hashing} illustrates this |
341 |
process. Also, each peer in the tightly structured overlay maintains a \emph{routing table}, which |
process. Also, each peer in the tightly structured overlay maintains a routing table, which |
342 |
consists of identifiers and IP addresses of other peers in the overlay. Entries of the routing |
consists of identifiers and IP addresses of other peers in the overlay. Entries of the routing |
343 |
table represent peer's neighbors in the overlay network. |
table represent peer's neighbors in the overlay network. |
344 |
|
|
349 |
\label{fig:structured_hashing} |
\label{fig:structured_hashing} |
350 |
\end{figure} |
\end{figure} |
351 |
|
|
352 |
Currently, all proposed tightly structured overlays provide at least |
Currently, all tightly structured overlays provide at least |
353 |
poly--loga-rithmical data lookup operations. However, there are some key |
polylogarithmical data lookup operations. However, there are some key |
354 |
differences between the data structures representing the identifier space. |
differences between the data structures representing the identifier space. |
355 |
For example, Chord \cite{stoica01chord}, Skip graphs \cite{AspnesS2003} and SkipNet \cite{harvey03skipnet2} maintain a |
For example, Chord \cite{stoica01chord}, Skip graphs \cite{AspnesS2003} and SkipNet \cite{harvey03skipnet2} maintain a |
356 |
distributed data structure which resembles skip lists \cite{78977}. |
distributed data structure that resembles skip lists \cite{78977}. |
357 |
In figure \ref{fig:structured_query}, we present an overview of Chord's data lookup process. |
In figure \ref{fig:structured_query}, we present an overview of Chord's data lookup process. |
358 |
On the right side of Chord's lookup process, the same data lookup process |
On the right side of Chord's lookup process, the same data lookup process |
359 |
is shown as a binary-tree abstraction. It can be seen, that in each step, the distance |
is shown as a binary-tree abstraction. It can be seen that, in each step, the distance |
360 |
decreases with a logarithmic efficiency. |
decreases with a logarithmic efficiency. |
361 |
|
|
362 |
\begin{figure} |
\begin{figure} |
367 |
\end{figure} |
\end{figure} |
368 |
|
|
369 |
Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry |
Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry |
370 |
\cite{zhao01tapestry} use balanced $k$-trees to implement the data structure of the identifier space. Figure |
\cite{zhao01tapestry} use balanced k-trees to implement the data structure of the identifier space. Figure |
371 |
\ref{fig:kademlia_lookup} shows the process of Kademlia's |
\ref{fig:kademlia_lookup} shows the process of Kademlia's |
372 |
data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}), |
data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}), |
373 |
which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup |
which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup |
383 |
\label{fig:kademlia_lookup} |
\label{fig:kademlia_lookup} |
384 |
\end{figure} |
\end{figure} |
385 |
|
|
386 |
Currently, there are only three higher level abstractions which are provided by the tightly structured overlays |
Currently, only three higher level abstractions are provided by the tightly structured overlays |
387 |
\cite{zhao03api}. Each of these abstractions represent a storage layer in the overlay, but |
\cite{zhao03api}. Each of these abstractions represent a storage layer in the overlay, but |
388 |
have semantical differences in the \emph{usage} of the overlay. |
have semantical differences in the usage of the overlay. |
389 |
|
|
390 |
First, Distributed Hash Table (DHT) (see e.g., \cite{dabek01widearea}, \cite{rowstron01storage}) |
First, Distributed Hash Table (DHT) (see e.g., \cite{dabek01widearea}, \cite{rowstron01storage}) |
391 |
implements the same functionality as a regular hash table by storing the mapping between a key and a value: |
implements the same functionality as a regular hash table by storing the mapping between a key and a value: |
396 |
\item \texttt{remove(key)}: remove a data item with a given key. |
\item \texttt{remove(key)}: remove a data item with a given key. |
397 |
\end{itemize} |
\end{itemize} |
398 |
|
|
399 |
DHT's \emph{interface} is generic; values can be any size and type (e.g., content hash over a file). In the |
DHT's interface is generic; values can be any size and type (e.g., content hash over a file). In the |
400 |
DHT abstraction the overlay itself stores the data items. Figure \ref{fig:Structured_lookup_using_DHT_model} shows the DHT abstraction |
DHT abstraction the overlay itself stores the data items. Figure \ref{fig:Structured_lookup_using_DHT_model} shows the DHT abstraction |
401 |
of the tightly structured overlay. |
of the tightly structured overlay. |
402 |
|
|
403 |
Second, Decentralized Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is a distributed |
Second, Decentralized Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is a distributed |
404 |
directory service. DOLR stores \emph{pointers} to data items throughout the overlay. DOLR's main |
directory service. DOLR stores pointers to data items throughout the overlay. DOLR's main |
405 |
operations are: |
operations are: |
406 |
|
|
407 |
\begin{itemize} |
\begin{itemize} |
411 |
\end{itemize} |
\end{itemize} |
412 |
|
|
413 |
|
|
414 |
The key difference between the DHT and the DOLR abstraction is that, in the DOLR abstraction, the overlay maintains only \emph{pointers} to the data. |
The key difference between the DHT and the DOLR abstraction is that, in the DOLR abstraction, the overlay maintains only pointers to the data. |
415 |
Also, the DOLR abstraction routes overlay messages to the nearest available peer, hosting a specific data item. This form of locality |
Also, the DOLR abstraction routes overlay messages to the nearest available peer hosting a specific data item. This form of locality |
416 |
is not supported by DHT. DOLR's interface is similar to the DHT's interface, i.e., values can be any size and type. |
is not supported by DHT. DOLR's interface is similar to the DHT's interface, i.e., values can be any size and type. |
417 |
|
|
418 |
Third, tightly structured overlays can be used for scalable group multicast or anycast operations (CAST) (see e.g., \cite{zhuang01bayeux}). |
Third, tightly structured overlays can be used for scalable group multicast or anycast operations (CAST) (see e.g., \cite{zhuang01bayeux}). |
446 |
\label{fig:Strucutred_lookup_using_DOLR_model} |
\label{fig:Strucutred_lookup_using_DOLR_model} |
447 |
\end{figure} |
\end{figure} |
448 |
|
|
449 |
In tightly structured systems, messages are routed across the overlay towards peers, whose |
In tightly structured systems, messages are routed across the overlay toward peers, whose |
450 |
peer identifier is gradually ''closer'' to the key's identifier |
peer identifier is gradually ''closer'' to the key's identifier |
451 |
in the identifier space. The distance can be measured by numerical |
in the identifier space. The distance can be measured by numerical |
452 |
difference between identifiers (e.g., Chord \cite{stoica01chord}), number of |
difference between identifiers (e.g., Chord \cite{stoica01chord}), number of |
453 |
same prefix bits between identifiers (e.g., Pastry \cite{rowston01pastry} and Tapestry |
same prefix bits between identifiers (e.g., Pastry \cite{rowston01pastry} and Tapestry |
454 |
\cite{zhao01tapestry}) or bit-wise exclusive or (XOR) (e.g., Kademlia \cite{maymounkov02kademlia}). |
\cite{zhao01tapestry}) or Bit-Wise Exclusive Or (XOR) (e.g., Kademlia \cite{maymounkov02kademlia}). |
455 |
Chord's \cite{stoica01chord} distance function does have the property of unidirection |
Chord's \cite{stoica01chord} distance function does have the property of unidirection |
456 |
(for a given point $p_i$ in the identifier space and distance $d$ > 0, there |
(for a given point $p_i$ in the identifier space and distance $d$ > 0, there |
457 |
is exactly one point $p_j$ in a way that the distance between $p_i$ and $p_j$ |
is exactly one point $p_j$ in a way that the distance between $p_i$ and $p_j$ |
458 |
is $d$), but does not have symmetry (the distance from $p_i$ to $p_j$ is same as the |
is $d$), but does not have symmetry (the distance from $p_i$ to $p_j$ is same as the |
459 |
distance from $p_j$ to $p_i$). Pastry's \cite{rowston01pastry} distance function supports |
distance from $p_j$ to $p_i$). Pastry's \cite{rowston01pastry} distance function supports |
460 |
symmetry, but does not support unidirection. According to \cite{balakrishanarticle03lookupp2p}, because |
symmetry, but does not support unidirection. According to Balakrishnan et al. \cite{balakrishanarticle03lookupp2p}, |
461 |
of XOR-metric, Kademlia's distance function is both unidirectional and symmetric. Moreover, Kademlia's \cite{maymounkov02kademlia} |
Kademlia's distance function is both unidirectional and symmetric because of the XOR-metric. |
462 |
|
Moreover, Kademlia's \cite{maymounkov02kademlia} |
463 |
XOR-based metric does not need stabilization (like in Chord \cite{stoica01chord}) and backup links |
XOR-based metric does not need stabilization (like in Chord \cite{stoica01chord}) and backup links |
464 |
(like in Pastry \cite{rowston01pastry}). |
(like in Pastry \cite{rowston01pastry}). |
465 |
However, in all of the above schemes, each hop in the overlay shortens the distance between |
However, in all of the above schemes, each hop in the overlay shortens the distance between |
466 |
current peer working with the data lookup and the key that was looked up in the identifier space. |
current peer working with the data lookup and the key that was looked up in the identifier space. |
467 |
|
|
468 |
Skip Graphs \cite{AspnesS2003} and SWAN \cite{bonsma02swan} employ a identifier space |
Skip Graphs \cite{AspnesS2003} and SWAN \cite{bonsma02swan} employ a identifier space |
469 |
in which queries are routed to \emph{keys}. In these systems |
in which queries are routed to keys. In these systems |
470 |
a peer occupies several positions in the identifier space, one for each |
a peer occupies several positions in the identifier space, one for each |
471 |
application-specific key. The indirection of placing close keys in the |
application-specific key. The opposite action of placing close keys in the |
472 |
custody of a provider peer is removed at the cost of each peer maintaining one |
custody of a provider peer is removed at the cost of each peer maintaining one |
473 |
''resource peer'' in the overlay network for each data item it publishes. The provider peer is a peer |
''resource peer'' in the overlay network for each data item it publishes. The provider peer is a peer |
474 |
which has initially published services into the overlay. |
that has initially published services into the overlay. |
475 |
|
|
476 |
PeerNet \cite{eriksson03peernet} differs from other tightly structured overlays in that it operates |
PeerNet \cite{eriksson03peernet} differs from other tightly structured overlays in that it operates |
477 |
at the \emph{network} layer instead of application layer (see the ISO-OSI reference model, e.g., \cite{800902}). |
at the \emph{network} layer instead of application layer (see the ISO-OSI reference model \cite{800902}). |
478 |
This property would provide a common interface |
This property would provide a common interface |
479 |
to all Peer-to-Peer systems using PeerNet. PeerNet makes an explicit distinction |
to all Peer-to-Peer systems using PeerNet. PeerNet makes an explicit distinction |
480 |
between peer identity and address, which is not supported by standard |
between peer identity and address, which is not supported by standard |
483 |
the system and $O(\log{n})$ data lookup efficiency. |
the system and $O(\log{n})$ data lookup efficiency. |
484 |
|
|
485 |
Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} have listed four requirements |
Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} have listed four requirements |
486 |
for tightly structured overlays\footnote{Authors use the term 'DHT' in their text, but in this context |
for tightly structured overlays\footnote{The authors use the term 'DHT' in their text, but in this paper |
487 |
it does not matter as they list \emph{general} properties of tightly structured overlays.} that have to be addressed in order |
it is not meant to be specific as the authors list \emph{general} properties of tightly structured overlays.} that have to be addressed in order |
488 |
to perform efficient data lookups in tightly structured overlays. |
to perform efficient data lookups in tightly structured overlays. |
489 |
First, mapping of keys to peers must be done in a load-balanced |
First, the mapping of keys to peers must be done in a load-balanced |
490 |
way. Second, the overlay must be able to forward a data lookup for a |
way. Second, the overlay must be able to forward a data lookup for a |
491 |
specific key to an appropriate peer. Third, overlay must |
specific key to an appropriate peer. Third, the overlay must |
492 |
support efficient distance function. Finally, routing tables for each peer |
support efficient distance function. Finally, the routing tables for each peer |
493 |
must be constructed and maintained adaptively. |
must be constructed and maintained adaptively. |
494 |
|
|
495 |
Additionally, authors argue in \cite{balakrishnan03semanticfree} that tightly structured systems |
Additionally, Balakrishnan et al. argue \cite{balakrishnan03semanticfree} that tightly structured systems |
496 |
are suitable for next generation Reference Resolutions Services (RRS)\footnote{ |
are suitable for next generation Reference Resolutions Services (RRS)\footnote{ |
497 |
Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system in the Internet.}. They present |
Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system on the Internet.}. They present |
498 |
two requirements about the nature of reference resolution. First, there should be a general-purpose |
two requirements about the nature of reference resolution. First, there should be a general-purpose |
499 |
and application-indepedent substrate for reference resolution. Second, the references themselves |
and application-indepedent substratum for reference resolution. Second, the references themselves |
500 |
should be unstructured and semantic-free. In this text, we define unstructured reference |
should be unstructured and semantic-free. In this paper, we define a unstructured reference |
501 |
as a reference that does not expose the target in any way and semantic-free reference as a reference |
as one that does not expose the target in any way and a semantic-free reference as a reference |
502 |
that there are no directives in the reference itself which would expose how the reference should be processed. |
that has no directives in the reference itself that would expose how the reference should be processed. |
503 |
|
|
504 |
|
|
505 |
\section{Differences} |
\section{Differences} |
506 |
|
|
507 |
Even though the loosely structured and the tightly structured approach are both Peer-to-Peer schemes, they |
Even though the loosely structured and the tightly structured approaches are both Peer-to-Peer schemes, they |
508 |
have very little in common. Indeed, the only thing they share is the fact that no other peer is more |
have very little in common. Indeed, the only similarity they share is the fact that no one peer is more |
509 |
important than any other in the Peer-to-Peer network. Fault tolerance \emph{may} |
important than another within the Peer-to-Peer network. Fault tolerance \emph{may} |
510 |
be an area in which approaches have similar properties (e.g., no single point of failure) \cite{milojicic02peertopeer}. |
be an other area in which these approaches have similar properties (e.g., no single point of failure) \cite{milojicic02peertopeer}. |
511 |
Fault tolerance properties of both approaches are currently only initial calculations, or |
The fault tolerance properties of both approaches are currently only initial calculations, or |
512 |
experimented in simulation environments. In real life, however, measuring fault tolerance is a much more |
experimented in simulation environments. In practical applications, however, measuring fault tolerance is a much more |
513 |
challenging task and requires more research to get reliable answers. |
challenging task and requires additional research to obtain reliable answers. |
514 |
|
|
515 |
The most important differences between approaches are the performance and scalability properties. |
The most important differences between the approaches are in the performance and scalability properties. |
516 |
Generally tightly structured systems can perform all internal operations in poly--logarithmic time |
Generally, tightly structured systems can perform all internal operations in polylogarithmic time, |
517 |
while the performance of loosely structured systems is not always even linear \cite{balakrishanarticle03lookupp2p}. |
while the performance of loosely structured systems is not always even linear \cite{balakrishanarticle03lookupp2p}. |
518 |
Moreover, loosely structured systems scale to millions of peers, |
Moreover, loosely structured systems scale to millions of peers, |
519 |
whereas tightly structured systems are able to cope with billions of concurrent |
whereas tightly structured systems are able to cope with billions of concurrent |
520 |
peers \cite{osokine02distnetworks}, \cite{kubiatowicz00oceanstore}. However, it is unknown |
peers \cite{osokine02distnetworks}, \cite{kubiatowicz00oceanstore}. However, it is unknown |
521 |
whether all proposed algorithms can preserve logarithmic efficiency and scalability properties |
whether or not all proposed algorithms can preserve logarithmic efficiency and scalability properties |
522 |
in real-life applications or not; several tightly structured systems |
in real-life applications; several tightly structured systems |
523 |
assume that participating peers are homogeneous, and the rate of join or leave operation is low \cite{gurmeet03symphony, |
assume that participating peers are homogeneous, and the rate of join or leave operation is low \cite{gurmeet03symphony, |
524 |
libennowell01observations, rowston03controlloingreliability}. |
libennowell01observations, rowston03controlloingreliability}. |
525 |
|
|
526 |
To the end user, the biggest difference between these systems is how data lookups are performed. Loosely |
For the end user, the biggest difference between these systems is how data lookups are performed. Loosely |
527 |
structured systems provide a more rich and user friendly way of searching data than tightly structured systems |
structured systems provide a more rich and user friendly way of searching for data than do tightly structured systems, |
528 |
as they have a support for keyword searches \cite{yang02efficientsearch, lv02searchreplication}. Tightly structured |
as the former have a support for keyword searches \cite{yang02efficientsearch, lv02searchreplication}. Tightly structured |
529 |
systems support only exact key lookups since each data item is identified by globally unique keys \cite{balakrishanarticle03lookupp2p, |
systems support only exact key lookups since each data item is identified by globally unique keys \cite{balakrishanarticle03lookupp2p, |
530 |
harren02complex, ansaryefficientbroadcast03}. |
harren02complex, ansaryefficientbroadcast03}. |
531 |
|
|
532 |
Table \ref{table_comparison_approach} lists the key differences between the loosely structured |
Table \ref{table_comparison_approach} lists the primary differences between the loosely structured |
533 |
approach and the tightly structured approach. |
approach and the tightly structured approach. |
534 |
|
|
535 |
|
|
613 |
Table \ref{table_Peer-to-Peer_algorithms} lists proposed Peer-to-Peer algorithms |
Table \ref{table_Peer-to-Peer_algorithms} lists proposed Peer-to-Peer algorithms |
614 |
and their key properties with regard to performance and scalability. The list |
and their key properties with regard to performance and scalability. The list |
615 |
includes algorithms from both loosely and tightly structured approaches. The list does not |
includes algorithms from both loosely and tightly structured approaches. The list does not |
616 |
include \emph{all} proposed Peer-to-Peer algorithms but rather includes the ones which already have |
include \emph{all} proposed Peer-to-Peer algorithms but rather includes the ones that already have |
617 |
been widely deployed, or the ones which may be promising in the future |
been widely deployed or which show promise for future Peer-to-Peer systems. |
|
Peer-to-Peer systems. |
|
618 |
|
|
619 |
We decided to follow the guidelines from \cite{kaashoek03koorde} in measuring |
We decided to follow the guidelines provided by Kaashoek et al. \cite{kaashoek03koorde} in measuring |
620 |
the properties of different Peer-to-Peer systems. However, we dropped |
the properties of various Peer-to-Peer systems. We eliminated |
621 |
out fault tolerance and load balancing properties, since they are hard to measure |
out fault tolerance and load balancing properties from consideration since they are difficult to measure |
622 |
in real life requirements. Additionally, however, we decided to include |
in real life applications. However, we decided to include |
623 |
the number of \emph{real} network connections for each peer in the overlay. Next, |
a number of real network connections for each peer in the overlay. Here, |
624 |
we describe the listed properties of Peer-to-Peer algorithms: |
we describe the listed properties of Peer-to-Peer algorithms: |
625 |
|
|
626 |
\begin{itemize} |
\begin{itemize} |