264 |
and KaZaa \cite{kazaaurl} use the FastTrack-based flooding protocol \cite{fasttrackurl}. However, it is not clear |
and KaZaa \cite{kazaaurl} use the FastTrack-based flooding protocol \cite{fasttrackurl}. However, it is not clear |
265 |
whether the power-law method is scalable or not, |
whether the power-law method is scalable or not, |
266 |
as the majority of the query requests are sent only to the high degree peers while making |
as the majority of the query requests are sent only to the high degree peers while making |
267 |
these peers to bear the load of the entire system. |
these peers bear the load of the entire system. |
268 |
|
|
269 |
%\begin{figure} |
%\begin{figure} |
270 |
%\centering |
%\centering |
287 |
\label{fig:gnutella_powerlaw} |
\label{fig:gnutella_powerlaw} |
288 |
\end{figure} |
\end{figure} |
289 |
|
|
290 |
Above presented improvements are only partial solutions. More advanced techniques |
The improvements presented above are only partial solutions. More advanced techniques |
291 |
to improve data lookup of loosely structured systems are discussed in chapter 3. Yet, however, |
to improve data lookup of loosely structured systems are discussed in chapter 3. Yet, however, |
292 |
techniques presented in chapter 3 are not adopted in any loosely structured system. |
techniques presented in chapter 3 are not adopted in any loosely structured system. |
293 |
|
|
318 |
|
|
319 |
\subsection{Systems} |
\subsection{Systems} |
320 |
|
|
321 |
With tightly structured systems, it is feasible to perform \emph{global} data lookups in the overlay efficiently. By global lookup, we mean |
With tightly structured systems, it is feasible to efficiently perform \emph{global} data lookups in the overlay. By global lookup, we mean |
322 |
that the system is able to find a service from the overlay, if it exists. |
that the system is able to find a service from the overlay, if it exists. |
323 |
While there are significant differences among proposed tightly structured systems, they all have in common |
While there are significant differences among proposed tightly structured systems, they all have a common property, |
324 |
that \emph{peer identifiers} are assigned to participating peers from |
in that \emph{peer identifiers} are assigned to participating peers from |
325 |
a large \emph{identifier space} by the overlay. Globally unique identifiers |
a large \emph{identifier space} by the overlay. Globally unique identifiers, \emph{keys}, |
326 |
are also assigned to application-specific data items, \emph{keys}, |
are also assigned to application-specific data items |
327 |
which are selected from the same identifier space. For instance, globally unique keys can be created |
which are selected from the same identifier space. For instance, globally unique keys can be created |
328 |
using a cryptographic content hash function (e.g., SHA-1 \cite{fips-sha-1}) over the contents of a data item. |
using a cryptographic content hash function (e.g., SHA-1 \cite{fips-sha-1}) over the contents of a data item. |
329 |
The form of identifier space differs between proposed systems. Geometrical circular form of identifier space (and variants) |
The form of identifier space differs between proposed systems. A geometrical circular form of identifier space (and variants) |
330 |
is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde}, |
is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde}, |
331 |
Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry} |
Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry} |
332 |
and Viceroy \cite{malkhi02viceroy} use a circular form of identifier space of $n$-bit integers modulo $2^{n}$. The |
and Viceroy \cite{malkhi02viceroy} use a circular form of identifier space of $n$-bit integers modulo $2^{n}$. The |
333 |
value of $n$ varies among systems. Again, CAN \cite{ratnasamy01can} uses a $d$-dimensional geometrical torus |
value of $n$ varies among systems. Again, CAN \cite{ratnasamy01can} uses a $d$-dimensional geometrical torus |
334 |
model to implement the form of identifier space. |
model to implement the form of identifier space. |
335 |
|
|
336 |
To store data into a tightly structured overlay, each application-specific |
To store data in a tightly structured overlay, each application-specific |
337 |
unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent |
unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent |
338 |
hashing \cite{258660}) to an existing peer in the overlay. Thus, tightly |
hashing \cite{258660}) to an existing peer in the overlay. Thus, a tightly |
339 |
structured overlay assigns a subset of all possible keys to every participating peer. |
structured overlay assigns a subset of all possible keys to every participating peer. |
340 |
We say that a peer is \emph{responsible} for the keys which are assigned by the overlay. |
We say that a peer is \emph{responsible} for the keys which are assigned by the overlay. |
341 |
Figure \ref{fig:structured_hashing} illustrates this |
Figure \ref{fig:structured_hashing} illustrates this |
357 |
distributed data structure which resembles skip lists \cite{78977}. |
distributed data structure which resembles skip lists \cite{78977}. |
358 |
In figure \ref{fig:structured_query}, we present an overview of Chord's data lookup process. |
In figure \ref{fig:structured_query}, we present an overview of Chord's data lookup process. |
359 |
On the right side of Chord's lookup process, the same data lookup process |
On the right side of Chord's lookup process, the same data lookup process |
360 |
is shown as a binary-tree abstraction. It can be noticed, that in each step, the distance |
is shown as a binary-tree abstraction. It can be seen, that in each step, the distance |
361 |
decreases with a logarithmic efficiency. |
decreases with a logarithmic efficiency. |
362 |
|
|
363 |
\begin{figure} |
\begin{figure} |
372 |
\ref{fig:kademlia_lookup} shows the process of Kademlia's |
\ref{fig:kademlia_lookup} shows the process of Kademlia's |
373 |
data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}), |
data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}), |
374 |
which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup |
which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup |
375 |
efficiency, where $n$ is the number of peers in the system. Koorde \cite{kaashoek03koorde}, a recent modification of Chord, uses de Bruijn graphs |
efficiency where $n$ is the number of peers in the system. Koorde \cite{kaashoek03koorde}, a recent modification of Chord, uses de Bruijn graphs |
376 |
\cite{debruijn46graph} to maintain local routing tables. It requires |
\cite{debruijn46graph} to maintain local routing tables. It requires |
377 |
each peer to have only about two links to other peers to provide $O(\log{n})$ performance. |
each peer to have only about two links to other peers to provide $O(\log{n})$ performance. |
378 |
|
|
388 |
\cite{zhao03api}. Each of these abstractions represent a storage layer in the overlay, but |
\cite{zhao03api}. Each of these abstractions represent a storage layer in the overlay, but |
389 |
have semantical differences in the \emph{usage} of the overlay. |
have semantical differences in the \emph{usage} of the overlay. |
390 |
|
|
391 |
First, Distributed Hash Table (DHT) (see e.g., \cite{dabek01widearea}, \cite{rowstron01storage}), |
First, Distributed Hash Table (DHT) (see e.g., \cite{dabek01widearea}, \cite{rowstron01storage}) |
392 |
implements the same functionality as a regular hash table by storing the mapping between a key and a value: |
implements the same functionality as a regular hash table by storing the mapping between a key and a value: |
393 |
|
|
394 |
\begin{itemize} |
\begin{itemize} |
398 |
\end{itemize} |
\end{itemize} |
399 |
|
|
400 |
DHT's \emph{interface} is generic; values can be any size and type (e.g., content hash over a file). In the |
DHT's \emph{interface} is generic; values can be any size and type (e.g., content hash over a file). In the |
401 |
DHT abstraction, the overlay itself stores the data items. Figure \ref{fig:Structured_lookup_using_DHT_model} shows the DHT abstraction |
DHT abstraction the overlay itself stores the data items. Figure \ref{fig:Structured_lookup_using_DHT_model} shows the DHT abstraction |
402 |
of the tightly structured overlay. |
of the tightly structured overlay. |
403 |
|
|
404 |
Second, Decentralized Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is a distributed |
Second, Decentralized Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is a distributed |
412 |
\end{itemize} |
\end{itemize} |
413 |
|
|
414 |
|
|
415 |
The key difference between the DHT and the DOLR abstraction is that in the DOLR abstraction the overlay maintains only \emph{pointers} to the data. |
The key difference between the DHT and the DOLR abstraction is that, in the DOLR abstraction, the overlay maintains only \emph{pointers} to the data. |
416 |
Also, the DOLR abstraction routes overlay's messages to a nearest available peer, hosting a specific data item. This form of locality |
Also, the DOLR abstraction routes overlay messages to the nearest available peer, hosting a specific data item. This form of locality |
417 |
is not supported by DHT. DOLR's interface is similar to the DHT's interface, i.e., values can be any size and type. |
is not supported by DHT. DOLR's interface is similar to the DHT's interface, i.e., values can be any size and type. |
418 |
|
|
419 |
Third, tightly structured overlay can be used for scalable group multicast or anycast operations (CAST) (see e.g., \cite{zhuang01bayeux}). |
Third, tightly structured overlays can be used for scalable group multicast or anycast operations (CAST) (see e.g., \cite{zhuang01bayeux}). |
420 |
The basic operations include: |
The basic operations include: |
421 |
|
|
422 |
\begin{itemize} |
\begin{itemize} |
426 |
\item \texttt{anycast(message, groupIdentifier)}: anycast a message to a group with a given group identifier. |
\item \texttt{anycast(message, groupIdentifier)}: anycast a message to a group with a given group identifier. |
427 |
\end{itemize} |
\end{itemize} |
428 |
|
|
429 |
The DOLR and the CAST abstractions have in common that they both use network proximity techniques |
The DOLR and CAST abstractions both use network proximity techniques |
430 |
to optimize their operations in the overlay. Figure \ref{fig:Strucutred_lookup_using_DOLR_model} |
to optimize their operations in the overlay. Figure \ref{fig:Strucutred_lookup_using_DOLR_model} |
431 |
presents the DOLR abstraction. |
presents the DOLR abstraction. |
432 |
|
|
456 |
Chord's \cite{stoica01chord} distance function does have the property of unidirection |
Chord's \cite{stoica01chord} distance function does have the property of unidirection |
457 |
(for a given point $p_i$ in the identifier space and distance $d$ > 0, there |
(for a given point $p_i$ in the identifier space and distance $d$ > 0, there |
458 |
is exactly one point $p_j$ in a way that the distance between $p_i$ and $p_j$ |
is exactly one point $p_j$ in a way that the distance between $p_i$ and $p_j$ |
459 |
is $d$), but doesn't have symmetry (the distance from $p_i$ to $p_j$ is same as the |
is $d$), but does not have symmetry (the distance from $p_i$ to $p_j$ is same as the |
460 |
distance from $p_j$ to $p_i$). Pastry's \cite{rowston01pastry} distance function supports |
distance from $p_j$ to $p_i$). Pastry's \cite{rowston01pastry} distance function supports |
461 |
symmetry, but doesn't support unidirection. According to \cite{balakrishanarticle03lookupp2p}, because |
symmetry, but does not support unidirection. According to \cite{balakrishanarticle03lookupp2p}, because |
462 |
of XOR-metric, Kademlia's distance function is both unidirectional and symmetric. Moreover, Kademlia's \cite{maymounkov02kademlia} |
of XOR-metric, Kademlia's distance function is both unidirectional and symmetric. Moreover, Kademlia's \cite{maymounkov02kademlia} |
463 |
XOR-based metric doesn't need stabilization (like in Chord \cite{stoica01chord}) and backup links |
XOR-based metric doesn't need stabilization (like in Chord \cite{stoica01chord}) and backup links |
464 |
(like in Pastry \cite{rowston01pastry}). |
(like in Pastry \cite{rowston01pastry}). |
465 |
However, in all above schemes each hop in the overlay shortens the distance between |
However, in all of the above schemes, each hop in the overlay shortens the distance between |
466 |
current peer working with the data lookup and the key which was looked up in the identifier space. |
current peer working with the data lookup and the key that was looked up in the identifier space. |
467 |
|
|
468 |
Skip Graphs \cite{AspnesS2003} and SWAN \cite{bonsma02swan} employ a identifier space |
Skip Graphs \cite{AspnesS2003} and SWAN \cite{bonsma02swan} employ a identifier space |
469 |
in which queries are routed to \emph{keys}. In these systems |
in which queries are routed to \emph{keys}. In these systems |
484 |
|
|
485 |
Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} have listed four requirements |
Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} have listed four requirements |
486 |
for tightly structured overlays\footnote{Authors use the term 'DHT' in their text, but in this context |
for tightly structured overlays\footnote{Authors use the term 'DHT' in their text, but in this context |
487 |
it doesn't matter as they list \emph{general} properties of tightly structured overlays.} which have to be addressed in order |
it doesn't matter as they list \emph{general} properties of tightly structured overlays.} that have to be addressed in order |
488 |
to perform efficient data lookups in tightly structured overlays. |
to perform efficient data lookups in tightly structured overlays. |
489 |
First, mapping of keys to peers must be done in a load-balanced |
First, mapping of keys to peers must be done in a load-balanced |
490 |
way. Second, the overlay must be able to forward a data lookup for a |
way. Second, the overlay must be able to forward a data lookup for a |
507 |
Even though the loosely structured and the tightly structured approach are both Peer-to-Peer schemes, they |
Even though the loosely structured and the tightly structured approach are both Peer-to-Peer schemes, they |
508 |
have very little in common. Indeed, the only thing they share is the fact that no other peer is more |
have very little in common. Indeed, the only thing they share is the fact that no other peer is more |
509 |
important than any other in the Peer-to-Peer network. Fault tolerance \emph{may} |
important than any other in the Peer-to-Peer network. Fault tolerance \emph{may} |
510 |
be an area, in which approaches have similar properties (e.g., no single point of failure) \cite{milojicic02peertopeer}. |
be an area in which approaches have similar properties (e.g., no single point of failure) \cite{milojicic02peertopeer}. |
511 |
Fault tolerance properties of both approaches are currently only initial calculations, or |
Fault tolerance properties of both approaches are currently only initial calculations, or |
512 |
experimented in simulation environments. In real-life, however, measuring fault tolerance is much more |
experimented in simulation environments. In real life, however, measuring fault tolerance is a much more |
513 |
challenging task and requires more research to get reliable answers. |
challenging task and requires more research to get reliable answers. |
514 |
|
|
515 |
The most important differences between approaches are the performance and scalability properties. |
The most important differences between approaches are the performance and scalability properties. |
523 |
assume that participating peers are homogeneous, and the rate of join or leave operation is low \cite{gurmeet03symphony, |
assume that participating peers are homogeneous, and the rate of join or leave operation is low \cite{gurmeet03symphony, |
524 |
libennowell01observations, rowston03controlloingreliability}. |
libennowell01observations, rowston03controlloingreliability}. |
525 |
|
|
526 |
To end user, the biggest difference between these systems is how data lookups are performed. Loosely |
To the end user, the biggest difference between these systems is how data lookups are performed. Loosely |
527 |
structured systems provide more rich and user friendly way of searching data than tightly structured systems |
structured systems provide a more rich and user friendly way of searching data than tightly structured systems |
528 |
as they have a support for keyword searches \cite{yang02efficientsearch, lv02searchreplication}. Tightly structured |
as they have a support for keyword searches \cite{yang02efficientsearch, lv02searchreplication}. Tightly structured |
529 |
systems support only exact key lookups since each data item is identified by globally unique keys \cite{balakrishanarticle03lookupp2p, |
systems support only exact key lookups since each data item is identified by globally unique keys \cite{balakrishanarticle03lookupp2p, |
530 |
harren02complex, ansaryefficientbroadcast03}. |
harren02complex, ansaryefficientbroadcast03}. |
612 |
|
|
613 |
Table \ref{table_Peer-to-Peer_algorithms} lists proposed Peer-to-Peer algorithms |
Table \ref{table_Peer-to-Peer_algorithms} lists proposed Peer-to-Peer algorithms |
614 |
and their key properties with regard to performance and scalability. The list |
and their key properties with regard to performance and scalability. The list |
615 |
includes algorithms from both loosely and tightly structured approaches. The list doesn't |
includes algorithms from both loosely and tightly structured approaches. The list does not |
616 |
include \emph{all} proposed Peer-to-Peer algorithms; only the ones which already have |
include \emph{all} proposed Peer-to-Peer algorithms but rather includes the ones which already have |
617 |
been widely deployed, or the ones which may be promising in the future |
been widely deployed, or the ones which may be promising in the future |
618 |
Peer-to-Peer systems are included. |
Peer-to-Peer systems. |
619 |
|
|
620 |
We decided to follow the guidelines from \cite{kaashoek03koorde} in measuring |
We decided to follow the guidelines from \cite{kaashoek03koorde} in measuring |
621 |
the properties of different Peer-to-Peer systems. However, we dropped |
the properties of different Peer-to-Peer systems. However, we dropped |
622 |
out fault tolerance and load balancing properties, since they are hard to measure |
out fault tolerance and load balancing properties, since they are hard to measure |
623 |
in face of real life requirements. Additionally, however, we decided to include |
in real life requirements. Additionally, however, we decided to include |
624 |
the number of \emph{real} network connections for each peer in the overlay. Next, |
the number of \emph{real} network connections for each peer in the overlay. Next, |
625 |
we describe the listed properties of Peer-to-Peer algorithms: |
we describe the listed properties of Peer-to-Peer algorithms: |
626 |
|
|
627 |
\begin{itemize} |
\begin{itemize} |
628 |
\item \textbf{Lookup}: the number of messages required when a data lookup is performed. |
\item \textbf{Lookup}: the number of messages required when a data lookup is performed. |
629 |
\item \textbf{Space}: the number of other peers which peer knows about (neighbors). |
\item \textbf{Space}: the number of other peers a peer is aware of (neighbors). |
630 |
\item \textbf{Insert/delete}: the number of network messages required when a peer joins or leaves the system. |
\item \textbf{Insert/delete}: the number of network messages required when a peer joins or leaves the system. |
631 |
\item \textbf{Number of network connections}: the number of concurrent network connections required to maintain correct neighbor information. |
\item \textbf{Number of network connections}: the number of concurrent network connections required to maintain correct neighbor information. |
632 |
\end{itemize} |
\end{itemize} |