106 |
|
|
107 |
In this thesis, we evaluate existing Peer-to-Peer approaches and |
In this thesis, we evaluate existing Peer-to-Peer approaches and |
108 |
evaluate them to Fenfire's needs. We start by reviewing existing Peer-to-Peer approaches, |
evaluate them to Fenfire's needs. We start by reviewing existing Peer-to-Peer approaches, |
109 |
algorithms and their key properties. We emphasize that despite the great amount of proposed |
algorithms and their key properties. Our insight is that despite the great amount of proposed |
110 |
Peer-to-Peer systems, we are able to classify \emph{all} systems either to loosely or |
Peer-to-Peer systems, we are able to classify \emph{all} systems either to loosely or |
111 |
tightly structured approach. We also discuss open problems in |
tightly structured approach. We also discuss open problems in |
112 |
Peer-to-Peer systems and divide problems into three sub-categories: security, performance, and miscellaneous |
Peer-to-Peer systems and divide problems into three sub-categories: security, performance, and miscellaneous |
170 |
systems fall: the loosely structured approach and the tightly structured approach. In the loosely |
systems fall: the loosely structured approach and the tightly structured approach. In the loosely |
171 |
structured approach the construction and the maintenance of the overlay is controlled |
structured approach the construction and the maintenance of the overlay is controlled |
172 |
loosely. This approach gives freedom for participating peers |
loosely. This approach gives freedom for participating peers |
173 |
to perform certain tasks in a Peer-to-Peer network. On the other hand, the tightly structured |
to perform certain tasks in a Peer-to-Peer network. On the other hand in the tightly structured |
174 |
approach the overlay is constructed determistically, which all participating peers have to follow. |
approach, the overlay is constructed determistically, which all participating peers have to follow. |
175 |
|
|
176 |
|
|
177 |
\section{Centralized} |
\section{Centralized} |
311 |
Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry} |
Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry} |
312 |
and Viceroy \cite{malkhi02viceroy} use a circular identifier space of $n$-bit integers modulo $2^{n}$. The |
and Viceroy \cite{malkhi02viceroy} use a circular identifier space of $n$-bit integers modulo $2^{n}$. The |
313 |
value of $n$ varies among systems. Again, CAN \cite{ratnasamy01can} uses a $d$-dimensional Cartesian |
value of $n$ varies among systems. Again, CAN \cite{ratnasamy01can} uses a $d$-dimensional Cartesian |
314 |
model to implement identifier space. |
model to implement the identifier space. |
315 |
|
|
316 |
There are three higher level abstractions which tightly structured overlays provide |
There are three higher level abstractions which tightly structured overlays provide |
317 |
\cite{zhao03api}. Each of these abstractions fulfill a storage layer in an overlay, but |
\cite{zhao03api}. Each of these abstractions fulfill a storage layer in the overlay, but |
318 |
they have semantical differences in the \emph{usage} of overlay. First, Distributed Hash |
have semantical differences in the \emph{usage} of the overlay. First, Distributed Hash |
319 |
Table (DHT) (see e.g., \cite{dabek01widearea}, \cite{rowstron01storage}), |
Table (DHT) (see e.g., \cite{dabek01widearea}, \cite{rowstron01storage}), |
320 |
implements three operations: \texttt{lookup(key)}, \texttt{remove(key)} and |
implements three operations: \texttt{lookup(key)}, \texttt{remove(key)} and |
321 |
\texttt{insert(key)}. As the name suggests, DHT implements the same functionality |
\texttt{insert(key)}. As the name suggests, DHT implements the same functionality |
325 |
Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is a distributed |
Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is a distributed |
326 |
directory service. DOLR stores \emph{pointers} to data items throughout the overlay. DOLR's main |
directory service. DOLR stores \emph{pointers} to data items throughout the overlay. DOLR's main |
327 |
operations are \texttt{publish(key)}, \texttt{removePublished(key)} and \texttt{sendToObject(key)}. The key |
operations are \texttt{publish(key)}, \texttt{removePublished(key)} and \texttt{sendToObject(key)}. The key |
328 |
difference between DHT and DOLR abstraction is that DOLR routes overlay's messages |
difference between the DHT and the DOLR abstraction is that the DOLR abstraction routes overlay's messages |
329 |
to nearest available peer, hosting a specific data item. This form of locality |
to a nearest available peer, hosting a specific data item. This form of locality |
330 |
is not supported by DHT. Finally, tightly structured overlay can be used for |
is not supported by DHT. Finally, tightly structured overlay can be used for |
331 |
scalable group multicast/any cast operations (CAST) (see e.g., \cite{zhuang01bayeux}). |
scalable group multicast or anycast operations (CAST) (see e.g., \cite{zhuang01bayeux}). |
332 |
The basic operations are \texttt{join(groupIdentifier)}, \texttt{leave(groupIdentifier)}, |
The basic operations are \texttt{join(groupIdentifier)}, \texttt{leave(groupIdentifier)}, |
333 |
\texttt{multicast(message, groupIdentifier)}, \texttt{anycast(message, groupIdentifier)}. |
\texttt{multicast(message, groupIdentifier)}, \texttt{anycast(message, groupIdentifier)}. |
334 |
Participating peers may join and leave the group and send multicast messages to |
Participating peers may join and leave the group and send multicast messages to |
335 |
the group, or anycast message to a specific member of the group. DOLR and CAST abstractions |
the group, or anycast message to a specific member of the group. The DOLR and the CAST abstractions |
336 |
have in common that they both use network proximity techniques |
have in common that they both use network proximity techniques |
337 |
to optimize their operations in the overlay. Figure \ref{fig:Strucutred_lookup_using_DOLR_model} |
to optimize their operations in the overlay. Figure \ref{fig:Strucutred_lookup_using_DOLR_model} |
338 |
presents the DOLR abstraction. |
presents the DOLR abstraction. |
356 |
for tightly structured overlays which have to be addressed in order |
for tightly structured overlays which have to be addressed in order |
357 |
to perform efficient data lookups in tightly structured overlays. |
to perform efficient data lookups in tightly structured overlays. |
358 |
First, mapping of keys to peers must be done in a load-balanced |
First, mapping of keys to peers must be done in a load-balanced |
359 |
way. Second, the overlay must be able to forward a lookup for a |
way. Second, the overlay must be able to forward a data lookup for a |
360 |
specific key to an appropriate peer. Third, overlay must have |
specific key to an appropriate peer. Third, overlay must |
361 |
support for a efficient distance function. Finally, routing tables for each peer |
support efficient distance function. Finally, routing tables for each peer |
362 |
must be constructed and maintained adaptively. |
must be constructed and maintained adaptively. |
363 |
|
|
364 |
To store data into a tightly structured overlay, each application-specific |
To store data into a tightly structured overlay, each application-specific |
365 |
unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent |
unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent |
366 |
hashing \cite{258660}) by the overlay to an existing peer in the overlay. Thus, tightly |
hashing \cite{258660}) by the overlay to an existing peer in the overlay. Thus, tightly |
367 |
structured overlay assigns a subset of all possible keys to every participating peer. |
structured overlay assigns a subset of all possible keys to every participating peer. |
368 |
We say that a peer is \emph{responsible} for the keys which are assigned by the overlay. |
We say that a peer is \emph{responsible} for the keys which are assigned by the overlay. |
369 |
Also, each peer in tightly structured overlay maintains a \emph{routing table}, which |
Figure \ref{fig:structured_hashing} illustrates the |
370 |
|
process of data to key mapping in a tightly structured overlay. |
371 |
|
Also, each peer in the tightly structured overlay maintains a \emph{routing table}, which |
372 |
consists of identifiers and IP addresses of other peers in the overlay. Entries of the routing |
consists of identifiers and IP addresses of other peers in the overlay. Entries of the routing |
373 |
table represents peer's neighbors in the overlay network. Figure \ref{fig:structured_hashing} illustrates the |
table represent peer's neighbors in the overlay network. |
|
process of data to key mapping in a tightly structured overlay. |
|
374 |
|
|
375 |
\begin{figure} |
\begin{figure} |
376 |
\centering |
\centering |
382 |
Currently, all proposed tightly structured overlays provide at least |
Currently, all proposed tightly structured overlays provide at least |
383 |
poly--logarithmical data lookup operations. However, there are some key |
poly--logarithmical data lookup operations. However, there are some key |
384 |
differences in the data structure that they use as a routing table. For example, Chord |
differences in the data structure that they use as a routing table. For example, Chord |
385 |
\cite{stoica01chord}, Skip graphs \cite{AspnesS2003} and SkipNet \cite{harvey03skipnet2} maintain a local |
\cite{stoica01chord}, Skip graphs \cite{AspnesS2003} and SkipNet \cite{harvey03skipnet2} maintain a |
386 |
data structure which resembles Skip lists \cite{78977}. |
distributed data structure which resembles Skip lists \cite{78977}. |
387 |
In figure \ref{fig:structured_query}, we present an overview of Chord's lookup process. |
In figure \ref{fig:structured_query}, we present an overview of Chord's lookup process. |
388 |
On the right side of Chord's lookup process, the same data lookup process |
On the right side of Chord's lookup process, the same data lookup process |
389 |
is shown as a binary-tree abstraction. It can be noticed, that in each step, the distance |
is shown as a binary-tree abstraction. It can be noticed, that in each step, the distance |
397 |
\end{figure} |
\end{figure} |
398 |
|
|
399 |
Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry |
Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry |
400 |
\cite{zhao01tapestry} uses balanced $k$-trees as routing table's data structure. Figure |
\cite{zhao01tapestry} uses balanced $k$-trees to implement the overlay. Figure |
401 |
\ref{fig:kademlia_lookup} shows the process of Kademlia's |
\ref{fig:kademlia_lookup} shows the process of Kademlia's |
402 |
data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}), |
data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}), |
403 |
which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup |
which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup |
429 |
function is both unidirectional and symmetric. Moreover, Kademlia's \cite{maymounkov02kademlia} |
function is both unidirectional and symmetric. Moreover, Kademlia's \cite{maymounkov02kademlia} |
430 |
XOR-based metric doesn't need stabilization (like in Chord \cite{stoica01chord}) and backup links |
XOR-based metric doesn't need stabilization (like in Chord \cite{stoica01chord}) and backup links |
431 |
(like in Pastry \cite{rowston01pastry}) \cite{balakrishanarticle03lookupp2p}. |
(like in Pastry \cite{rowston01pastry}) \cite{balakrishanarticle03lookupp2p}. |
432 |
However, in all previous schemes each hop in the overlay shortens the distance between |
However, in all above schemes each hop in the overlay shortens the distance between |
433 |
current peer working with the data lookup and the key which was looked up in the identifier space. |
current peer working with the data lookup and the key which was looked up in the identifier space. |
434 |
|
|
435 |
Skip Graphs \cite{AspnesS2003} and SWAN \cite{bonsma02swan} employ a key space very similar to a tightly structured |
Skip Graphs \cite{AspnesS2003} and SWAN \cite{bonsma02swan} employ a key space very similar to a tightly structured |
436 |
overlay, but in which queries are routed to \emph{keys}. In these systems |
overlay, but in which queries are routed to \emph{keys}. In these systems |
437 |
a peer occupies several positions in the identifier space, one for each |
a peer occupies several positions in the identifier space, one for each |
438 |
application-specific key. The indirection of placing close keys in the |
application-specific key. The indirection of placing close keys in the |
439 |
custody of a storing peer is removed at the cost of each peer maintaining one |
custody of a provider peer is removed at the cost of each peer maintaining one |
440 |
''resource peer'' in the overlay network for each data item it publishes. Provider peer is the peer |
''resource peer'' in the overlay network for each data item it publishes. The provider peer is a peer |
441 |
in the overlay which is responsible for the assigned keys |
which has initially published services into the overlay. |
442 |
|
|
443 |
PeerNet \cite{eriksson03peernet} differs from other tightly structured overlays in that it operates |
PeerNet \cite{eriksson03peernet} differs from other tightly structured overlays in that it operates |
444 |
at the \emph{network} layer. PeerNet makes an explicit distinction |
at the \emph{network} layer. PeerNet makes an explicit distinction |
452 |
\subsection{Sketch of a formal definition} |
\subsection{Sketch of a formal definition} |
453 |
|
|
454 |
In this subsection, we formalize the main features of tightly structured overlay, i.e., |
In this subsection, we formalize the main features of tightly structured overlay, i.e., |
455 |
identifiers, identifier space and mapping function. |
identifiers, identifier space and the mapping function. |
456 |
|
|
457 |
Let $S$ be the aggregate of all services $s$ in the system. Let $P$ be the aggregate of |
Let $S$ be the aggregate of all services $s$ in the system. Let $P$ be the aggregate of |
458 |
all peers $p$ in system. Let $I$ be the aggregate of all identifiers $i$ in system. |
all peers $p$ in system. Let $I$ be the aggregate of all identifiers $i$ in system. |
484 |
challenging task and requires more research to get reliable answers. |
challenging task and requires more research to get reliable answers. |
485 |
|
|
486 |
The most important difference between approaches is performance and scalability properties. Generally |
The most important difference between approaches is performance and scalability properties. Generally |
487 |
tightly structured systems can perform all internal operations in poly-logarithmic time\footnote{However, it is unknown |
tightly structured systems can perform all internal operations in a poly-logarithmic time\footnote{However, it is unknown |
488 |
whether all proposed algorithms can preserve logarithmic properties in real-life applications or not.} |
whether all proposed algorithms can preserve logarithmic properties in real-life applications or not.} |
489 |
while the performance of loosely structured systems is not always even linear, . |
while the performance of loosely structured systems is not always even linear. |
490 |
Moreover, loosely structured systems scale to millions of peers, whereas tightly structured systems are able |
Moreover, loosely structured systems scale to millions of peers, whereas tightly structured systems are able |
491 |
to cope with billions of concurrent peers \cite{osokine02distnetworks}, \cite{kubiatowicz00oceanstore}. |
to cope with billions of concurrent peers \cite{osokine02distnetworks}, \cite{kubiatowicz00oceanstore}. |
492 |
|
|
493 |
To end user, biggest difference between these systems is how data lookups are performed. Loosely |
To end user, the biggest difference between these systems is how data lookups are performed. Loosely |
494 |
structured systems provide a more rich and user friendly way of searching data as they |
structured systems provide a more rich and user friendly way of searching data than tightly structured systems |
495 |
have support for keyword search than tightly structured systems. On the other hand, tightly structured |
as they have a support for keyword searches. On the other hand, tightly structured |
496 |
systems support only exact key lookups as each data item is identified by globally unique keys. |
systems support only exact key lookups as each data item is identified by globally unique keys. |
497 |
|
|
498 |
In the end, both systems have open problems and issues. We will discuss these aspects in more detail in |
In the end, both systems have open problems and issues. We will discuss these aspects more detail in |
499 |
chapter 3. Table \ref{table_comparison_approach} lists the key differences between the loosely structured |
chapter 3. Table \ref{table_comparison_approach} lists the key differences between the loosely structured |
500 |
approach and the tightly structured approach. |
approach and the tightly structured approach. |
501 |
|
|