363 |
To store data into tightly structured overlay, each application-specific |
To store data into tightly structured overlay, each application-specific |
364 |
unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent |
unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent |
365 |
hashing \cite{258660}) by the overlay to an existing peer in the overlay. Thus, tightly |
hashing \cite{258660}) by the overlay to an existing peer in the overlay. Thus, tightly |
366 |
structured overlay assigns a subset of all possible keys to every participating peer. |
structured overlay assigns a subset of all possible keys to every participating peer |
367 |
|
\footnote{We say that a peer is \emph{responsible} for the keys which are assigned by the overlay.}. |
368 |
Also, each peer in tightly structured overlay maintains a \emph{routing table}, which |
Also, each peer in tightly structured overlay maintains a \emph{routing table}, which |
369 |
consists of identifiers and IP addresses of other peers in the overlay. Entries of routing |
consists of identifiers and IP addresses of other peers in the overlay. Entries of routing |
370 |
table are peer's neighbors in the overlay network. Figure \ref{fig:structured_hashing} illustrates the |
table are peer's neighbors in the overlay network. Figure \ref{fig:structured_hashing} illustrates the |
414 |
for maintaining information about other peers in the system and |
for maintaining information about other peers in the system and |
415 |
$O(\log{n})$ data lookup efficiency. |
$O(\log{n})$ data lookup efficiency. |
416 |
|
|
417 |
Stoica et al. \cite{balakrishanarticle03lookupp2p} have listed |
Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} have listed |
418 |
four requirements for tightly structured overlays, which have to be |
four requirements for tightly structured overlays, which have to be |
419 |
addressed in order to perform efficient data lookups in tightly structured overlays. |
addressed in order to perform efficient data lookups in tightly structured overlays. |
420 |
First, mapping of keys to peers must be done in a load-balanced |
First, mapping of keys to peers must be done in a load-balanced |
535 |
experimented in simulation environments. In real-life, measuring fault tolerance is much more |
experimented in simulation environments. In real-life, measuring fault tolerance is much more |
536 |
challenging task and requires more research to get reliable answers. |
challenging task and requires more research to get reliable answers. |
537 |
|
|
|
There are significant differences between loosely structured and tightly structured approaches. |
|
538 |
The most important difference between approaches is performance and scalability properties. While |
The most important difference between approaches is performance and scalability properties. While |
539 |
performance of loosely structured approach is not always even linear, generally tightly structured |
performance of loosely structured approach is not always even linear, generally tightly structured |
540 |
approach can perform all internal operations in poly-logarithmic time\footnote{However, it is unknown |
approach can perform all internal operations in poly-logarithmic time\footnote{However, it is unknown |
541 |
whether all proposed algorithms can preserve logarithmic properties in real-life applications or not.}. |
whether all proposed algorithms can preserve logarithmic properties in real-life applications or not.}. |
542 |
|
Loosely structured systems scale to millions of peers, whereas tightly structured systems are able |
543 |
|
to cope with billions of concurrent peers. |
544 |
|
|
545 |
Another key point is the philosophy how overlay network is constructed and maintained. While loosely |
Another key point is the philosophy how overlay network is constructed and maintained. While loosely |
546 |
structured approach gives much freedom to individual peers to join and leave the overlay network, tightly |
structured approach gives much freedom to individual peers to join and leave the overlay network, tightly |
547 |
structured approach has certain features, in which participating peers have no control at all |
structured approach has certain features, in which participating peers have no control at all |
548 |
(such as mapping of data items). |
(such as mapping of data items). With DHT abstraction of tightly structured approach, for instance, |
549 |
|
peer has no power to decide where the data items are mapped in the overlay. |
550 |
|
|
551 |
To end user, biggest difference between these systems is how data lookups are performed. Loosely |
To end user, biggest difference between these systems is how data lookups are performed. Loosely |
552 |
structured systems provide more richer and user friendly way of searching data as they |
structured systems provide more richer and user friendly way of searching data as they |
582 |
\endfoot |
\endfoot |
583 |
|
|
584 |
|
|
|
|
|
|
\parbox{90pt}{Construction of overlay} & |
|
|
\parbox{100pt}{Uncontrolled} & |
|
|
\parbox{100pt}{Controlled} |
|
|
|
|
585 |
\\ \hline |
\\ \hline |
586 |
|
|
587 |
\parbox{90pt}{Queries} & |
\parbox{90pt}{Queries} & |
595 |
\\ \hline |
\\ \hline |
596 |
|
|
597 |
\parbox{90pt}{Query traffic} & |
\parbox{90pt}{Query traffic} & |
598 |
\parbox{100pt}{$O(n)/O(n^{2})$} & |
\parbox{100pt}{$O(n), O(n^{2})$} & |
599 |
\parbox{100pt}{$O(1)/O(\log{n})$} |
\parbox{100pt}{$O(1), O(\log{n})$} |
600 |
\\ \hline |
\\ \hline |
601 |
|
|
602 |
\parbox{90pt}{Guaranteed data lookup} & |
\parbox{90pt}{Guaranteed data lookup} & |
604 |
\parbox{100pt}{Yes} |
\parbox{100pt}{Yes} |
605 |
\\ \hline |
\\ \hline |
606 |
|
|
607 |
\parbox{90pt}{Overlay's structure} & |
\parbox{90pt}{Construction and maintenance of overlay} & |
608 |
\parbox{100pt}{Uncontrolled and ad hoc} & |
\parbox{100pt}{uncontrolled and ad hoc} & |
609 |
\parbox{100pt}{Controlled and structured} |
\parbox{100pt}{Controlled and structured} |
610 |
\\ \hline |
\\ \hline |
611 |
|
|
612 |
\parbox{90pt}{Max. number of peers} & |
\parbox{90pt}{Maximum number of peers} & |
613 |
\parbox{100pt}{Millions} & |
\parbox{100pt}{Millions} & |
614 |
\parbox{100pt}{Billions} |
\parbox{100pt}{Billions} |
615 |
\\ \hline |
\\ \hline |
618 |
\parbox{100pt}{Local} & |
\parbox{100pt}{Local} & |
619 |
\parbox{100pt}{Not local} |
\parbox{100pt}{Not local} |
620 |
\\ \hline |
\\ \hline |
|
|
|
|
\parbox{90pt}{Support for heterogeneity} & |
|
|
\parbox{100pt}{Yes} & |
|
|
\parbox{100pt}{No} |
|
|
\\ \hline |
|
621 |
|
|
622 |
\parbox{90pt}{Support for locality} & |
\parbox{90pt}{Support for locality} & |
623 |
\parbox{100pt}{Yes} & |
\parbox{100pt}{Yes} & |
628 |
\parbox{100pt}{No} & |
\parbox{100pt}{No} & |
629 |
\parbox{100pt}{Yes} |
\parbox{100pt}{Yes} |
630 |
\\ \hline |
\\ \hline |
|
|
|
|
\parbox{90pt}{Design/Implementation complexity} & |
|
|
\parbox{100pt}{Low} & |
|
|
\parbox{100pt}{High} |
|
|
\\ \hline |
|
631 |
|
|
632 |
\parbox{90pt}{Fault-tolerant} & |
\parbox{90pt}{Fault-tolerant} & |
633 |
\parbox{100pt}{High} & |
\parbox{100pt}{High} & |
1284 |
|
|
1285 |
Somewhat surprisingly little research has been done in this area, especially when considering |
Somewhat surprisingly little research has been done in this area, especially when considering |
1286 |
the possible impact of \emph{unwanted social behavior} to performance of Peer-to-Peer |
the possible impact of \emph{unwanted social behavior} to performance of Peer-to-Peer |
1287 |
system. Problem is addressed by Golle et al. \cite{golle01incentivesp2p}. Some |
system. The problem is addressed by Golle et al. \cite{golle01incentivesp2p}, Ngan et al. |
1288 |
|
\cite{ngan03enforcefile} and Shneidman et al. \cite{shneidman03rationality}. Some |
1289 |
research has been focused on semantic properties of the overlay in order to increase |
research has been focused on semantic properties of the overlay in order to increase |
1290 |
cooperation among participating peers \cite{crespo02semanticoverlay}. Ramanathan et al. |
cooperation among participating peers \cite{crespo02semanticoverlay}. Ramanathan et al. |
1291 |
\cite{ramanathan02goodpeers} and Bernstein et al. \cite{bernstein03selection} use |
\cite{ramanathan02goodpeers} and Bernstein et al. \cite{bernstein03selection} use |
1296 |
|
|
1297 |
\subsection{Simulating Peer-to-Peer systems} |
\subsection{Simulating Peer-to-Peer systems} |
1298 |
|
|
1299 |
Very little research has been done on simulating a \emph{global} Peer-to-Peer system. Presumably, this |
Very little research has been done on simulating a Peer-to-Peer system. Presumably, this |
1300 |
is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very |
is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very |
1301 |
difficult. Floyd et al. has been studying the simulation of the Internet in \cite{504642}. Authors |
difficult. Floyd et al. has been studying the simulation of the Internet in \cite{504642}. Authors |
1302 |
state that simulating the Internet is very challenging task, because of Internet's heterogeneity |
state that simulating the Internet is very challenging task, because of Internet's heterogeneity |
1303 |
and rapid change. Obviously, these factors exist also in Peer-to-Peer systems even with higher |
and rapid change. Obviously, these factors exist also in Peer-to-Peer systems even with higher |
1304 |
rates. |
rates. |
1305 |
|
|
1306 |
As long as global simulations of Peer-to-Peer systems are lacking, we cannot make any detailed |
As long as comprehensive simulations of Peer-to-Peer systems are lacking, we cannot make any detailed |
1307 |
analysis on usage patterns in Peer-to-Peer systems. However, we can assume that, e.g., |
analysis on general properties of Peer-to-Peer system such as usage patterns. However, we can assume |
1308 |
query keywords follow the Zipf-like distributions \cite{breslau98implications} both in the |
that, e.g., query keywords follow the Zipf-like distributions \cite{breslau98implications} both in the |
1309 |
Internet and in Peer-to-Peer systems. |
Internet and in Peer-to-Peer systems. |
1310 |
|
|
1311 |
\section{Summary} |
\section{Summary} |
1915 |
|
|
1916 |
\subsection{Algorithms} |
\subsection{Algorithms} |
1917 |
|
|
1918 |
We use DOLR abstraction of tightly structured approach, i.e., each participating peer hosts |
We use the DOLR abstraction of tightly structured approach, i.e., each participating peer hosts |
1919 |
the data and overlay maintains only the \emph{pointers} to the data. We decided to use DOLR in our |
the data and overlay maintains only the \emph{pointers} to the data. We decided to use the DOLR |
1920 |
model, since DOLR systems locate data without specifying a storage policy explicitly \cite{rhea03benchmarks}. |
abstraction in our model, since DOLR systems locate data without specifying a storage policy explicitly \cite{rhea03benchmarks}. |
1921 |
DHT based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have |
DHT based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have |
1922 |
critical problems with load balancing in highly heterogeneous environment. This problem is caused by peers |
critical problems with load balancing in highly heterogeneous environment. This problem is caused by peers |
1923 |
which may not be able to store relatively large amount of data with key/value pair, assigned randomly by |
which may not be able to store relatively large amount of data with key/value pair, assigned randomly by |
1924 |
mapping function of the overlay. Additionally, these systems wastes both storage and bandwidth, and |
mapping function of the overlay. These systems wastes both storage and bandwidth, and |
1925 |
are sensitive to certain attacks (e.g., DDoS attack). |
are sensitive to certain attacks (e.g., DDoS attack). Additionally, we prefer \emph{abstraction} |
1926 |
|
level analysis as very recently better and better tightly structured algorihtms have been proposed. |
1927 |
|
Thus, we don't want to bind our system proposal to a specific algorithm definitively as we expect |
1928 |
|
that this development continues. |
1929 |
|
|
1930 |
In the following subsections we assume that we know the structure of |
In the following subsections we assume that we know the structure of |
1931 |
''virtual file'' before hand, i.e., when assembling a ''virtual file'', we know all Storm |
''virtual file'' before hand, i.e., when assembling a ''virtual file'', we know all Storm |