256 |
have studied different data lookup methods in power-law networks and have found that by |
have studied different data lookup methods in power-law networks and have found that by |
257 |
instructing the peers that forward data lookups to select high degree peers, the performance of data lookup |
instructing the peers that forward data lookups to select high degree peers, the performance of data lookup |
258 |
increases significantly. Figure \ref{fig:gnutella_powerlaw} presents an example topology of power-law network with three high |
increases significantly. Figure \ref{fig:gnutella_powerlaw} presents an example topology of power-law network with three high |
259 |
degree peers. Some of the most recent loosely structured Peer-to-Peer systems have adopted this method to improve the data lookup model of loosely structured |
degree peers. Some of the most recent loosely structured Peer-to-Peer protocols have adopted this method to improve the data lookup model of loosely structured |
260 |
systems \cite{gnutella2url, fasttrackurl}. Both protocols use high degree peers to optimize the data lookup model of the |
systems \cite{gnutella2url, fasttrackurl}. Both protocols use high degree peers to optimize the data lookup model of the |
261 |
system. Shareaza \cite{shareazaurl} uses the Gnutella2 protocol \cite{gnutella2url} in data lookups, Morpheus \cite{morpheusurl} |
system. Shareaza \cite{shareazaurl} uses the Gnutella2-based flooding protocol \cite{gnutella2url} in data lookups, Morpheus \cite{morpheusurl} |
262 |
and KaZaa \cite{kazaaurl} use the FastTrack protocol \cite{fasttrackurl}. |
and KaZaa \cite{kazaaurl} use the FastTrack-based flooding protocol \cite{fasttrackurl}. |
263 |
It is not clear whether the power-law method is scalable or not, |
It is not clear whether the power-law method is scalable or not, |
264 |
as the majority of the query requests are sent only to the high degree peers while making |
as the majority of the query requests are sent only to the high degree peers while making |
265 |
these peers to bear the load of the entire system. |
these peers to bear the load of the entire system. |
323 |
a large \emph{identifier space} by the overlay. Globally unique identifiers |
a large \emph{identifier space} by the overlay. Globally unique identifiers |
324 |
are also assigned to application-specific data items, \emph{keys}, |
are also assigned to application-specific data items, \emph{keys}, |
325 |
which are selected from the same identifier space. For instance, globally unique keys can be created |
which are selected from the same identifier space. For instance, globally unique keys can be created |
326 |
using a cryptographic content hash (e.g., \cite{fips-sha-1}) over the contents of a data item. |
using a cryptographic content hash (e.g., SHA-1 \cite{fips-sha-1}) over the contents of a data item. |
327 |
The form of identifier space differs between proposed systems. Geometrical circular form of identifier space (and variants) |
The form of identifier space differs between proposed systems. Geometrical circular form of identifier space (and variants) |
328 |
is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde}, |
is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde}, |
329 |
Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry} |
Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry} |
340 |
process of data to key mapping in a tightly structured overlay. |
process of data to key mapping in a tightly structured overlay. |
341 |
Also, each peer in the tightly structured overlay maintains a \emph{routing table}, which |
Also, each peer in the tightly structured overlay maintains a \emph{routing table}, which |
342 |
consists of identifiers and IP addresses of other peers in the overlay. Entries of the routing |
consists of identifiers and IP addresses of other peers in the overlay. Entries of the routing |
343 |
table represent peer's neighbors in the overlay network. |
table represent peer's neighbors in the overlay network. |
344 |
|
|
345 |
|
Currently, all proposed tightly structured overlays provide at least |
346 |
|
poly--logarithmical data lookup operations. However, there are some key |
347 |
|
differences in the data structures representing the identifier space. |
348 |
|
For example, Chord \cite{stoica01chord}, Skip graphs \cite{AspnesS2003} and SkipNet \cite{harvey03skipnet2} maintain a |
349 |
|
distributed data structure which resembles Skip list \cite{78977}. |
350 |
|
In figure \ref{fig:structured_query}, we present an overview of Chord's lookup process. |
351 |
|
On the right side of Chord's lookup process, the same data lookup process |
352 |
|
is shown as a binary-tree abstraction. It can be noticed, that in each step, the distance |
353 |
|
decreases with a logarithmic efficiency. |
354 |
|
|
355 |
|
\begin{figure} |
356 |
|
\centering |
357 |
|
\includegraphics[width=10cm, height=6cm]{structured_query.eps} |
358 |
|
\caption{Chord's simplified data lookup process on top of tightly structured overlay.} |
359 |
|
\label{fig:structured_query} |
360 |
|
\end{figure} |
361 |
|
|
362 |
|
Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry |
363 |
|
\cite{zhao01tapestry} uses balanced $k$-trees to implement the overlay. Figure |
364 |
|
\ref{fig:kademlia_lookup} shows the process of Kademlia's |
365 |
|
data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}), |
366 |
|
which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup |
367 |
|
efficiency. Koorde \cite{kaashoek03koorde}, a recent modification of Chord, uses de Bruijn graphs |
368 |
|
\cite{debruijn46graph} to maintain local routing tables. Koorde \cite{kaashoek03koorde} requires |
369 |
|
each peer to have only about two links to other peers to provide $O(\log{n})$ performance. |
370 |
|
|
371 |
|
|
372 |
|
\begin{figure} |
373 |
|
\centering |
374 |
|
\includegraphics[width=10cm, height=8cm]{kademlia_lookup.eps} |
375 |
|
\caption{Kademlia's simplified data lookup process on top of tightly structured overlay.} |
376 |
|
\label{fig:kademlia_lookup} |
377 |
|
\end{figure} |
378 |
|
|
379 |
\begin{figure} |
\begin{figure} |
380 |
\centering |
\centering |
383 |
\label{fig:structured_hashing} |
\label{fig:structured_hashing} |
384 |
\end{figure} |
\end{figure} |
385 |
|
|
|
Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} have listed four requirements |
|
|
for tightly structured overlays\footnote{Authors use the term 'DHT' in their text, but in this context |
|
|
it doesn't matter as they list \emph{general} properties of tightly structured overlays.} which have to be addressed in order |
|
|
to perform efficient data lookups in tightly structured overlays. |
|
|
First, mapping of keys to peers must be done in a load-balanced |
|
|
way. Second, the overlay must be able to forward a data lookup for a |
|
|
specific key to an appropriate peer. Third, overlay must |
|
|
support efficient distance function. Finally, routing tables for each peer |
|
|
must be constructed and maintained adaptively. |
|
|
|
|
386 |
Currently, there are only three higher level abstractions which tightly structured overlays provide |
Currently, there are only three higher level abstractions which tightly structured overlays provide |
387 |
\cite{zhao03api}. Each of these abstractions represent a storage layer in the overlay, but |
\cite{zhao03api}. Each of these abstractions represent a storage layer in the overlay, but |
388 |
have semantical differences in the \emph{usage} of the overlay. |
have semantical differences in the \emph{usage} of the overlay. |
447 |
\label{fig:Strucutred_lookup_using_DOLR_model} |
\label{fig:Strucutred_lookup_using_DOLR_model} |
448 |
\end{figure} |
\end{figure} |
449 |
|
|
|
Currently, all proposed tightly structured overlays provide at least |
|
|
poly--logarithmical data lookup operations. However, there are some key |
|
|
differences in the data structures representing the identifier space. |
|
|
For example, Chord \cite{stoica01chord}, Skip graphs \cite{AspnesS2003} and SkipNet \cite{harvey03skipnet2} maintain a |
|
|
distributed data structure which resembles Skip lists \cite{78977}. |
|
|
In figure \ref{fig:structured_query}, we present an overview of Chord's lookup process. |
|
|
On the right side of Chord's lookup process, the same data lookup process |
|
|
is shown as a binary-tree abstraction. It can be noticed, that in each step, the distance |
|
|
decreases with a logarithmic efficiency. |
|
|
|
|
|
\begin{figure} |
|
|
\centering |
|
|
\includegraphics[width=10cm, height=6cm]{structured_query.eps} |
|
|
\caption{Chord's simplified data lookup process on top of tightly structured overlay.} |
|
|
\label{fig:structured_query} |
|
|
\end{figure} |
|
|
|
|
|
Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry |
|
|
\cite{zhao01tapestry} uses balanced $k$-trees to implement the overlay. Figure |
|
|
\ref{fig:kademlia_lookup} shows the process of Kademlia's |
|
|
data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}), |
|
|
which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup |
|
|
efficiency. Koorde \cite{kaashoek03koorde}, a recent modification of Chord, uses de Bruijn graphs |
|
|
\cite{debruijn46graph} to maintain local routing tables. Koorde \cite{kaashoek03koorde} requires |
|
|
each peer to have only about two links to other peers to provide $O(\log{n})$ performance. |
|
|
|
|
450 |
|
|
451 |
\begin{figure} |
In tightly structured system, messages are routed across the overlay towards peers, whose |
|
\centering |
|
|
\includegraphics[width=10cm, height=8cm]{kademlia_lookup.eps} |
|
|
\caption{Kademlia's simplified data lookup process on top of tightly structured overlay.} |
|
|
\label{fig:kademlia_lookup} |
|
|
\end{figure} |
|
|
|
|
|
|
|
|
All messages are routed across the overlay towards peers, whose |
|
452 |
peer identifier is gradually ''closer'' to the key's identifier |
peer identifier is gradually ''closer'' to the key's identifier |
453 |
in the identifier space. The distance can be measured by numerical |
in the identifier space. The distance can be measured by numerical |
454 |
difference between identifiers (e.g., Chord \cite{stoica01chord}), number of |
difference between identifiers (e.g., Chord \cite{stoica01chord}), number of |
482 |
overlays, i.e., $O(\log{n})$ space required for maintaining information about other peers in |
overlays, i.e., $O(\log{n})$ space required for maintaining information about other peers in |
483 |
the system and $O(\log{n})$ data lookup efficiency. |
the system and $O(\log{n})$ data lookup efficiency. |
484 |
|
|
485 |
|
Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} have listed four requirements |
486 |
|
for tightly structured overlays\footnote{Authors use the term 'DHT' in their text, but in this context |
487 |
|
it doesn't matter as they list \emph{general} properties of tightly structured overlays.} which have to be addressed in order |
488 |
|
to perform efficient data lookups in tightly structured overlays. |
489 |
|
First, mapping of keys to peers must be done in a load-balanced |
490 |
|
way. Second, the overlay must be able to forward a data lookup for a |
491 |
|
specific key to an appropriate peer. Third, overlay must |
492 |
|
support efficient distance function. Finally, routing tables for each peer |
493 |
|
must be constructed and maintained adaptively. |
494 |
|
|
495 |
Additionally, authors argue in \cite{balakrishnan03semanticfree} that tightly structured systems |
Additionally, authors argue in \cite{balakrishnan03semanticfree} that tightly structured systems |
496 |
are suitable for next generation Reference Resolutions Services (RRS)\footnote{ |
are suitable for next generation Reference Resolutions Services (RRS)\footnote{ |
497 |
Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system in the Internet.}. They present |
Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system in the Internet.}. They present |
522 |
whether all proposed algorithms can preserve logarithmic efficiency and scalability properties |
whether all proposed algorithms can preserve logarithmic efficiency and scalability properties |
523 |
in real-life applications or not; several tightly structured systems |
in real-life applications or not; several tightly structured systems |
524 |
assume that participating peers are homogeneous, and the rate of join or leave operation is low \cite{gurmeet03symphony, |
assume that participating peers are homogeneous, and the rate of join or leave operation is low \cite{gurmeet03symphony, |
525 |
libennowell01observations}. |
libennowell01observations, rowston03controlloingreliability}. |
526 |
|
|
527 |
To end user, the biggest difference between these systems is how data lookups are performed. Loosely |
To end user, the biggest difference between these systems is how data lookups are performed. Loosely |
528 |
structured systems provide more rich and user friendly way of searching data than tightly structured systems |
structured systems provide more rich and user friendly way of searching data than tightly structured systems |
529 |
as they have a support for keyword searches. Tightly structured |
as they have a support for keyword searches \cite{yang02efficientsearch, lv02searchreplication}. Tightly structured |
530 |
systems support only exact key lookups since each data item is identified by globally unique keys. |
systems support only exact key lookups since each data item is identified by globally unique keys \cite{balakrishanarticle03lookupp2p, |
531 |
|
harren02complex, ansaryefficientbroadcast03}. |
532 |
|
|
533 |
Table \ref{table_comparison_approach} lists the key differences between the loosely structured |
Table \ref{table_comparison_approach} lists the key differences between the loosely structured |
534 |
approach and the tightly structured approach. |
approach and the tightly structured approach. |
583 |
\parbox{100pt}{Yes} |
\parbox{100pt}{Yes} |
584 |
\\ \hline |
\\ \hline |
585 |
|
|
|
\parbox{90pt}{Construction and maintenance of the overlay} & |
|
|
\parbox{100pt}{Uncontrolled and ad hoc} & |
|
|
\parbox{100pt}{Controlled and structured} |
|
|
\\ \hline |
|
586 |
|
|
587 |
\parbox{90pt}{Scalable query lookup model} & |
\parbox{90pt}{Scalable data lookup model} & |
588 |
\parbox{100pt}{No} & |
\parbox{100pt}{No} & |
589 |
\parbox{100pt}{Yes} |
\parbox{100pt}{Yes} |
590 |
\\ \hline |
\\ \hline |
591 |
|
|
592 |
\parbox{90pt}{Data item mapped into the overlay} & |
\parbox{90pt}{Data items mapped into the overlay} & |
593 |
\parbox{100pt}{No} & |
\parbox{100pt}{No} & |
594 |
\parbox{100pt}{Yes} |
\parbox{100pt}{Yes} |
595 |
\\ \hline |
\\ \hline |
826 |
In this chapter, we discuss open problems in Peer-to-Peer research. |
In this chapter, we discuss open problems in Peer-to-Peer research. |
827 |
Note that the open problems list considered here is not meant |
Note that the open problems list considered here is not meant |
828 |
to be an exhaustive survey of \emph{all} open problems in Peer-to-Peer domain; |
to be an exhaustive survey of \emph{all} open problems in Peer-to-Peer domain; |
829 |
we focus our attention to some security, scalability, usability and performance related |
we focus our attention to some issues related security, scalability, usability and performance. |
830 |
issues only. |
|
831 |
|
|
832 |
\section{Overview} |
\section{Overview} |
833 |
|
|
843 |
|
|
844 |
In tightly structured system the main concern is to make overlay's data lookup process |
In tightly structured system the main concern is to make overlay's data lookup process |
845 |
more fault tolerant against hostile attacks. Other key problems in tightly structured |
more fault tolerant against hostile attacks. Other key problems in tightly structured |
846 |
systems are the lack of keyword searches, support for heterogeneous peers and load balancing |
systems are the lack of keyword searches \cite{harren02complex, ansaryefficientbroadcast03}, support for heterogeneous peers |
847 |
\cite{balakrishanarticle03lookupp2p}. |
\cite{rowston03controlloingreliability} and load balancing \cite{balakrishanarticle03lookupp2p, byers03dhtbalancing}. |
848 |
|
|
849 |
\section{Security problems in Peer-to-Peer} |
\section{Security problems in Peer-to-Peer} |
850 |
|
|
864 |
the Distributed Denial of Service attack. |
the Distributed Denial of Service attack. |
865 |
|
|
866 |
In the Sybil attack model \cite{douceur02sybil}, a hostile entity presents multiple |
In the Sybil attack model \cite{douceur02sybil}, a hostile entity presents multiple |
867 |
entities, i.e., when a peer selects a subset of entities to perform a operation, a peer can select the same |
entities, i.e., when a peer communicates with a subset of other participating entities to perform a operation, a peer communicates |
868 |
hostile entity multiple times. Therefore, one hostile entity can control a large fraction of Peer-to-Peer system thereby |
only with the same hostile entity. Therefore, one hostile entity can control a large fraction of Peer-to-Peer system while |
869 |
repressing the redundancy of the system. Unfortunately, currently there are no realizable techniques for against the Sybil |
repressing the redundancy of the system. Authors argue in \cite{douceur02sybil} that without a centralized authority, Sybil attacks are always possible in a Peer-to-Peer |
870 |
attack: without a centralized authority, Sybil attacks are always possible in a Peer-to-Peer |
system except under extreme and unrealistic assumptions of resource parity and coordination among entities. According to \cite{douceur02sybil}, u |
871 |
system except under extreme and unrealistic assumptions of resource parity and coordination among entities \cite{douceur02sybil}. |
nrealistic assumptions include: all entities should be nearly homogeneous, all identities can be validated simultaneously by all |
872 |
Castro et al. \cite{castro02securerouting} suggest the use of cryptographic content hashes in the creation process of peer identifier |
entities across the system and when accepting identities that are not directly validated, the required number of certificates exceeds |
873 |
against the Sybil attack. According to authors, in this technique the IP address of a peer can be verified by the other peer. |
the number of systemwide failures. Castro et al. \cite{castro02securerouting} suggest the use of cryptographic content hashes in the |
874 |
|
creation process of peer identifier against the Sybil attack. According to authors, in this technique the IP address of a peer can be verified by the other peer. |
875 |
They call this method as a one form of \emph{self-certifying data}. |
They call this method as a one form of \emph{self-certifying data}. |
876 |
|
|
877 |
In the Fail-stop attack model, cited in \cite{naor03simpledht}, a faulty peer is deleted from the Peer-to-Peer system. Thus, |
In the Fail-stop attack model, cited in \cite{naor03simpledht}, a faulty peer is deleted from the Peer-to-Peer system. Thus, |
878 |
a specific data item can be lost from the system temporaraly (or permanently). The reason for the faultiness of a peer can be a |
a specific data item can be lost from the system temporaraly (or permanently). The reason for the faultiness of a peer can be a |
879 |
software failure or a hostile attack. The Byzantine attack model \cite{357176} is closely related to Fail-stop model. The Byzantine model can |
software failure or a hostile attack. The Byzantine attack model \cite{357176} is closely related to Fail-stop model. In the Byzantine attack model |
880 |
be seen as more severe than Fail-stop model as there are no restrictions over the behavior of faulty peers; for instance, |
$3f + 1$ is the minimum number of peers that allow system to provide the safety and liveness properties when up to $f$ peers are faulty \cite{357176}. |
881 |
the cooperation between multiple malicious faulty peers is possible \cite{357176}. A practical solution for the Byzantine failures have been |
The Byzantine model can be seen as more severe than Fail-stop model as there are no restrictions over the behavior of faulty peers, e.g., the cooperation |
882 |
proposed by Castro et al. \cite{296824}. |
between multiple \emph{malicious} faulty peers is possible \cite{357176}. A practical solution for the Byzantine failures have been |
883 |
|
proposed by Castro et al. \cite{296824}. Authors use in their work replication algorithm to tolerate Byzantine faults and cryptographic |
884 |
|
certificate techniques to prevent spoofing and replays and to detect corrupted messages. |
885 |
|
|
886 |
The Spam generating attack \cite{naor03simpledht} is an another known attack model against Peer-to-Peer system. In the Spam |
The Spam generating attack \cite{naor03simpledht} is an another known attack model against Peer-to-Peer system. In the Spam |
887 |
attack, a hostile or faulty peer may produce false information of the data, or refuses to (or is not able to) reply to requests. |
attack, a hostile or faulty peer may produce false information of the data, or refuses to (or is not able to) reply to requests. |
888 |
Possible solution against this attack is that peer should not trust a single entity. Instead, a peer should get |
Naor et al. \cite{naor03simpledht} have proposed a partial solution against Spam attack in a \emph{faulty} peer environment (not hostile). |
|
information from multiple entities and trust on the majority's opinion. This method requires more messages to be |
|
|
sent to the network while increasing the system load. However, if the Spam attack is combined with the Sybil attack, obviously |
|
|
the previously mentioned solution doesn't work. Naor et al. \cite{naor03simpledht} have proposed a partial solution against Spam attack |
|
|
in a \emph{faulty} peer environment (not hostile). |
|
889 |
|
|
890 |
Overloading of targeted peers is a form of Distributed Denial of Service attack (DDoS) (see, e.g., \cite{372148}). For instance, |
Overloading of targeted peers is a form of Distributed Denial of Service attack (DDoS) (see, e.g., \cite{372148}). For instance, |
891 |
a hostile entity can attempt to burden targeted peers with garbage network packets. As a consequence, peers may act incorrectly or |
a hostile entity can attempt to burden targeted peers with garbage network packets. As a consequence, peers may act incorrectly or |
901 |
According to \cite{aberer01trust}, mutual trust ''...allows agents to cooperate in a game-theoretic situation that corresponds |
According to \cite{aberer01trust}, mutual trust ''...allows agents to cooperate in a game-theoretic situation that corresponds |
902 |
to the repeated prisoners dilemma and leads in the long term to an increased aggregated utility for the participating agents''. |
to the repeated prisoners dilemma and leads in the long term to an increased aggregated utility for the participating agents''. |
903 |
They define \emph{trust management} as a mechanism that allows to establish mutual trust. Furthermore, \emph{reputation} is a measure |
They define \emph{trust management} as a mechanism that allows to establish mutual trust. Furthermore, \emph{reputation} is a measure |
904 |
that is derived from knowledge on interactions in the past \cite{aberer01trust} In this subsection, we discuss mechanisms to maintain |
that is derived from knowledge on interactions in the past \cite{aberer01trust}. In this subsection, we discuss mechanisms to maintain |
905 |
trust in Peer-to-Peer systems. |
trust in Peer-to-Peer systems. |
906 |
|
|
907 |
Trust in Peer-to-Peer systems is based on \emph{reputation}. Little research has been done on reputation models in Peer-to-Peer |
Trust in Peer-to-Peer systems is based on \emph{reputation}. Little research has been done on reputation models in Peer-to-Peer |
954 |
distributed systems which are able to provide some level of anonymity (e.g., \cite{mneturl}). |
distributed systems which are able to provide some level of anonymity (e.g., \cite{mneturl}). |
955 |
|
|
956 |
Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no |
Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no |
957 |
such a system which is able to provide all types of anonymity. Specifically, the conflicts |
such a system which is able to provide complete anonymity. Specifically, the conflicts |
958 |
between anonymity and other properties of Peer-to-Peer system require more research work. |
between anonymity and other properties of Peer-to-Peer system require more research work. |
959 |
|
|
960 |
|
|
974 |
|
|
975 |
\subsection{Hostile entities} |
\subsection{Hostile entities} |
976 |
|
|
977 |
One serious problem in Peer-to-Peer systems is the inability to identify hostile entities. |
One serious problem in Peer-to-Peer systems is the inability to distinguish hostile entities from regular entities |
978 |
|
trustworthy. |
979 |
One possible solution is to use a self-monitoring system, such as SOMO \cite{zhang03somo}, in which a self-monitoring overlay |
One possible solution is to use a self-monitoring system, such as SOMO \cite{zhang03somo}, in which a self-monitoring overlay |
980 |
constantly analyses the Peer-to-Peer overlay. Self-monitoring overlay is built on top of Peer-to-Peer overlay. Authors in |
constantly analyses the Peer-to-Peer overlay. Self-monitoring overlay is built on top of Peer-to-Peer overlay. Authors in |
981 |
\cite{sit02securitycons} suggest the use of system invariants. They emphasize that system invariants should be veriable, and if |
\cite{sit02securitycons} suggest the use of system invariants. They emphasize that system invariants should be veriable, and if |
992 |
|
|
993 |
\subsection{Secure query routing} |
\subsection{Secure query routing} |
994 |
|
|
995 |
By secure routing, we mean that a Peer-to-Peer system is able to deliver a network message |
Secure query routing is essential to any Peer-to-Peer system. By secure routing in this context, we mean that a Peer-to-Peer system |
996 |
thoughout the overlay to a correct destination. |
is able to deliver a network message thoughout the overlay to a correct destination efficiently. |
997 |
|
|
998 |
Aspnes et al. in \cite{aspnes02faultrouting} and Kaashoek et al. in \cite{kaashoek03koorde} formally |
Aspnes et al. in \cite{aspnes02faultrouting} and Kaashoek et al. in \cite{kaashoek03koorde} formally |
999 |
prove the lower and upper bounds for the space requirements of locating a specific data item reliable in a |
prove the lower and upper bounds for the space requirements of locating a specific data item reliable in a |
1635 |
\section{Xanalogical storage model} |
\section{Xanalogical storage model} |
1636 |
|
|
1637 |
Xanalogical storage model \cite{nelson99xanalogicalneeded} is a different kind of model for |
Xanalogical storage model \cite{nelson99xanalogicalneeded} is a different kind of model for |
1638 |
presenting data and relationships between data. \emph{Enfilade}, |
presenting data and relationships between data, e.g., while in the World Wide Web links are |
1639 |
can be considered as a mutable ''virtual file'' (or part of one), which is a list |
between \emph{documents}, in xanalogical storage model links are between individual |
1640 |
|
\emph{characters}. \emph{Enfilade},can be considered as a mutable ''virtual file'' (or part of one), which is a list |
1641 |
of fluid media content. Fluid media is the smallest units of data in xanalogical storage |
of fluid media content. Fluid media is the smallest units of data in xanalogical storage |
1642 |
model. \emph{Transclusion} is an inclusion in |
model. \emph{Transclusion} is an inclusion in |
1643 |
enfilade of contents already used in another enfilade. With the transclusion, a system |
enfilade of contents already used in another enfilade. With the transclusion, a system |
1644 |
implementing xanalogical storage model is able to show all data content that share the same |
implementing xanalogical storage model is able to show \emph{all} data content that share the same |
1645 |
fluid media with current data content (e.g., all documents in a system containing a specific |
fluid media with current data content (e.g., all documents in a system containing document's text). |
1646 |
document's text). |
Figure \ref{fig:xanalogical_model} |
|
|
|
|
While in the World Wide Web links are |
|
|
between \emph{documents}, in xanalogical storage model links are between individual |
|
|
\emph{characters}. Figure \ref{fig:xanalogical_model} |
|
1647 |
illustrates xanalogical storage model with documents, text and characters. |
illustrates xanalogical storage model with documents, text and characters. |
1648 |
Links between data are external |
|
1649 |
|
In xanalogical storage model, links between data are external |
1650 |
and bidirectional. A link is shown between any two data contents |
and bidirectional. A link is shown between any two data contents |
1651 |
containing a specific \emph{fluid media unit} (e.g., a character) that the link connects. |
containing a specific \emph{fluid media unit} (e.g., a character) that the link connects. |
1652 |
Each fluid media unit in xanalogical storage model has a |
Each fluid media unit in xanalogical storage model has a |
1686 |
content hash, all identifiers are directly the data verifiers as well. The uniquess of blocks creates |
content hash, all identifiers are directly the data verifiers as well. The uniquess of blocks creates |
1687 |
a basis for implementing xanalogical storage model in the Fenfire system. Storm blocks have much in common with regular files as they |
a basis for implementing xanalogical storage model in the Fenfire system. Storm blocks have much in common with regular files as they |
1688 |
both contain the data. The main difference is that Storm blocks are \emph{immutable} since any |
both contain the data. The main difference is that Storm blocks are \emph{immutable} since any |
1689 |
change to the byte sequence would change block's hash value (globally unique identifier). |
change to the byte sequence would change block's hash value (i.e., globally unique identifier). |
1690 |
|
|
1691 |
Support for immutable data is built on the immutable abstraction. Storm uses |
Support for immutable data is built on the immutable abstraction. Storm uses |
1692 |
\emph{pointers} and \emph{diffs} for dealing with this kind of data. Using diffs |
\emph{pointers} and \emph{diffs} for dealing with this kind of data. Using diffs |
1695 |
More information about diffs can be found from \cite{fallenstein03storm}. |
More information about diffs can be found from \cite{fallenstein03storm}. |
1696 |
|
|
1697 |
\emph{Pointer} \cite{benja02urn5, fallenstein03storm} is a semantic-free updatable reference to |
\emph{Pointer} \cite{benja02urn5, fallenstein03storm} is a semantic-free updatable reference to |
1698 |
Storm data block, i.e., Storm scroll block. Pointer is unique reference to the data and it is usually |
Storm data block. Pointer is a unique reference to the data and it is usually |
1699 |
represented as a random string. Storm pointers are rather a \emph{concept} of data (e.g., ''The first page of the most recent |
represented as a random string. Storm pointers are rather a \emph{concept} of data (e.g., ''The front page of the most recent |
1700 |
version of New York Times newspaper'') whereas scroll blocks \emph{contain} the data |
version of New York Times newspaper'') whereas scroll blocks \emph{contain} the data |
1701 |
(''New York Times newspaper, 10.10.2002, version 1.0''). |
(''New York Times newspaper, 10.10.2002, version 1.0''). |
1702 |
Figure \ref{fig:storm_model} illustrates simplified Storm storage model with pointers. |
Figure \ref{fig:storm_model} illustrates Storm storage model with pointers. |
1703 |
|
|
1704 |
Each pointer is associated with a collection of \emph{pointer blocks}. |
Each pointer is \emph{linked} to a collection of \emph{pointer blocks}. |
1705 |
Pointers can be created by a user, before the creation of scroll blocks. Pointer blocks |
Pointers can be created by a user, before the creation of scroll blocks. Pointer blocks |
1706 |
are created automatically by Storm when a scroll block is associated with a pointer |
are created automatically by Storm when a scroll block is created and associated with a pointer |
1707 |
(e.g., by a user when creating a concept of ''The first page of the most recent |
(e.g., a user creates a scroll block associated with the concept ''The front page of the most recent |
1708 |
version of New York Times newspaper''). Pointer block has always a single target (i.e., a scroll block) |
version of New York Times newspaper''). Pointer block has always a single target (i.e., a scroll block) |
1709 |
for the pointer, saying that pointer $P$ targets block $B$. In addition to this, pointer block |
for the pointer, saying that pointer $P$ targets block $B$. In addition to this, pointer block |
1710 |
may contain a list of zero or more obsoleted pointer blocks: when a new version of pointer |
may contain a list of zero or more obsoleted pointer blocks: when a new version of pointer |
1736 |
\chapter{Evaluation of Peer-to-Peer for Fenfire} |
\chapter{Evaluation of Peer-to-Peer for Fenfire} |
1737 |
|
|
1738 |
In this chapter we evaluate Fenfire in Peer-to-Peer environment. |
In this chapter we evaluate Fenfire in Peer-to-Peer environment. |
1739 |
We start by giving a problem overview. Then, we define Fenfire's special needs and evaluate existing |
We start by giving a problem overview. Then, we define special needs and evaluate existing |
1740 |
Peer-to-Peer approaches in light of these requirements. After that, we propose a combination |
Peer-to-Peer approaches in light of these requirements. After that, we propose a combination |
1741 |
of Peer-to-Peer techniques reviewed in this thesis to be used with Fenfire and present simple methods to perform data |
of Peer-to-Peer techniques reviewed in this thesis to be used with Fenfire and present simple methods to perform data |
1742 |
lookups (data lookups are required by Alph module). In the end of this chapter, we discuss possible problems of using Fenfire |
lookups. Data lookups are required by Alph module which implements the xanalogical storage model in Fenfire. |
1743 |
|
In the end of this chapter, we discuss possible problems of using Fenfire |
1744 |
in Peer-to-Peer environment. |
in Peer-to-Peer environment. |
1745 |
|
|
1746 |
|
|
1761 |
participants responded whether Peer-to-Peer systems are suitable for hypermedia |
participants responded whether Peer-to-Peer systems are suitable for hypermedia |
1762 |
publishing or not. |
publishing or not. |
1763 |
|
|
|
|
|
1764 |
In Peer-to-Peer environment, our objectives are simple but yet hard to fulfill. |
In Peer-to-Peer environment, our objectives are simple but yet hard to fulfill. |
1765 |
First, as discussed in chapter 4, xanalogical document is a ''virtual |
First, as discussed in chapter 4, xanalogical document is a ''virtual |
1766 |
file'', in which parts of the document are fetched from a |
file'', in which parts of the document are fetched from a |
1767 |
\emph{global} data repository\footnote{Global repository is not a requirement. Locally constructed xanalogical |
\emph{global} data repository\footnote{Global repository is not a requirement. Locally constructed xanalogical |
1768 |
documents are feasible and they can be assembled without any global data.}. Thus, system implementing xanalogical storage model \emph{must} |
documents are feasible and they can be assembled without global data repository.}. Thus, system implementing xanalogical storage model \emph{must} |
1769 |
support global data lookups order to assemble the ''virtual file'' from fragments of data. |
support global data lookups order to assemble the ''virtual file'' from fragments of data. |
1770 |
Specifically, our task is to locate and fetch (i.e., obtain) \emph{all} Storm scroll blocks, associated to a specific ''virtual |
Specifically, our task is to locate and fetch (i.e., obtain) \emph{all} Storm blocks\footnote{These blocks are called \emph{scroll} blocks.}, |
1771 |
file'' from the Peer-to-Peer once the construction of ''virtual'' file |
associated to a specific ''virtual file'' from the Peer-to-Peer once the construction of ''virtual'' file |
1772 |
is resolved (i.e., we know what scroll blocks are required to assemble the ''virtual file''). Also, in addition to the |
is resolved (i.e., we know what blocks are required to assemble the ''virtual file''). Also, in addition to the |
1773 |
\emph{direct} scroll block obtaining using globally unique identifier of Storm scroll block, |
\emph{direct} block obtaining using globally unique identifier of Storm block, |
1774 |
we also must support the \emph{indirect} obtaining of Storm scroll block using the pointers. |
we also must support the \emph{indirect} obtaining of Storm block using the pointer mechanism. |
1775 |
Second, we want that users' operations in Fenfire |
Second, we want that users' operations in Fenfire |
1776 |
are location transparent: data lookups have to be efficient, since constructing |
are location transparent: data lookups have to be efficient, since constructing |
1777 |
one ''virtual file'' may need obtaining several Storm blocks, which are distributed |
one ''virtual file'' may need obtaining several Storm blocks, which are distributed |
1853 |
hot spots in the system \cite{ratnasamy02routing}. Current client-server implementation of Fenfire uses |
hot spots in the system \cite{ratnasamy02routing}. Current client-server implementation of Fenfire uses |
1854 |
standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content |
standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content |
1855 |
hash for verifying the integrity of data by recomputing the content hash |
hash for verifying the integrity of data by recomputing the content hash |
1856 |
for block. In face of multisource downloads, Fenfire must support |
for Storm block. In face of multisource downloads, Fenfire must support |
1857 |
tree-based hashes\footnote{With multisource downloads, tree-based hash functions can be used |
tree-based hashes\footnote{With multisource downloads, tree-based hash functions can be used |
1858 |
to verify fixed length segments of data. If hash value of data segment is incorrect, |
to verify fixed length segments of data. If hash value of data segment is incorrect, |
1859 |
we need only to fetch \emph{segment} of data (instead of whole data) from |
we need only to fetch \emph{segment} of data (instead of whole data) from |
1869 |
severe problems with load balancing in a highly heterogeneous environment \cite{rao03loadbalancing}. The problem is caused by peers |
severe problems with load balancing in a highly heterogeneous environment \cite{rao03loadbalancing}. The problem is caused by peers |
1870 |
which may not be able to store relatively large blocks, assigned randomly by the mapping function of the overlay. |
which may not be able to store relatively large blocks, assigned randomly by the mapping function of the overlay. |
1871 |
|
|
1872 |
In our method, each peer maintains the following local data structures for local operations: a data structure for listing all |
For simplicity, we assume that we have resolved the construction of the ''virtual file'' before locating any Storm blocks, i.e., |
|
key-value pairs which are assigned by the overlay; a data structure for listing all key-value pairs which are assigned by the overlay |
|
|
in the chronological order (the most recent block is topmost). We use Storm blocks' identifiers and pointers |
|
|
as \emph{keys} of the overlay. |
|
|
|
|
|
We assume that we have resolved the construction of the ''virtual file'' before locating any Storm blocks, i.e., |
|
1873 |
when assembling the ''virtual file'' we know all the Storm blocks, which are required to complete the ''virtual file''. |
when assembling the ''virtual file'' we know all the Storm blocks, which are required to complete the ''virtual file''. |
1874 |
Also, we don't respond to the security issues related to Peer-to-Peer systems, since there is no working solution |
Also, we don't respond to the security issues related to Peer-to-Peer systems, since there is no working solution |
1875 |
available yet. Thus, we either assume that Fenfire has a reliable technique for identifying individual entities, or |
available yet. Thus, we either assume that Fenfire has a reliable technique for identifying individual entities, or |
1878 |
|
|
1879 |
|
|
1880 |
\begin{itemize} |
\begin{itemize} |
1881 |
\item Data lookup with a given identifier of Storm scroll block. |
\item Data lookup with a given identifier of Storm block. |
1882 |
\begin{enumerate} |
\begin{enumerate} |
1883 |
\item Submit the data lookup using scroll block's identifier. |
\item Submit the data lookup using block's identifier. |
1884 |
\item Each peer forwards the data lookup to a closer peer which hosts the given scroll block identifier instructed by the overlay. |
\item Each peer forwards the data lookup to a closer peer which hosts the given block identifier instructed by the overlay. |
1885 |
\item The pointer peer returns value (e.g., the IP address of provider peer) to the query originator throughout the overlay. |
\item The pointer peer returns value (e.g., the IP address of provider peer) to the query originator throughout the overlay. |
1886 |
\item The query originator requests the provider peer to return the scroll block. |
\item The query originator requests the provider peer to return the block. |
1887 |
\end{enumerate} |
\end{enumerate} |
1888 |
\end{itemize} |
\end{itemize} |
1889 |
|
|
1890 |
|
|
1891 |
\begin{itemize} |
\begin{itemize} |
1892 |
\item Data lookup with a given pointer returning most recent scroll block. |
\item Data lookup with a given pointer returning most recent block. |
1893 |
\begin{enumerate} |
\begin{enumerate} |
1894 |
\item The query originator locally computes a hash over given pointer. |
\item The query originator locally computes a hash over given pointer. |
1895 |
\item Each peer forwards the data lookup to a closer peer which hosts the given hash of pointer instructed by the overlay. |
\item Each peer forwards the data lookup to a closer peer which hosts the given hash of pointer instructed by the overlay. |
1896 |
\item The pointer peer returns most recent pointer block's key-value pair (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes throughout the overlay. |
\item The pointer peer returns most recent pointer block's key-value pair (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes throughout the overlay. |
1897 |
\item The query originator requests the provider peer to return the scroll block. |
\item The query originator requests the provider peer to return the block. |
1898 |
\end{enumerate} |
\end{enumerate} |
1899 |
\end{itemize} |
\end{itemize} |
1900 |
|
|
1901 |
\begin{itemize} |
\begin{itemize} |
1902 |
\item Data lookup with a given pointer returning scroll block(s) for a given date or time range. |
\item Data lookup with a given pointer returning block(s) for a given date or time range. |
1903 |
\begin{enumerate} |
\begin{enumerate} |
1904 |
\item The query originator locally computes a hash over given pointer. |
\item The query originator locally computes a hash over given pointer. |
1905 |
\item Each peer forwards the data lookup to a closer peer which hosts the given hash of pointer instructed by the overlay. |
\item Each peer forwards the data lookup to a closer peer which hosts the given hash of pointer instructed by the overlay. |
1906 |
\item Pointer peer returns pointer block's key-value pair(s) (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes throughout the overlay. |
\item Pointer peer returns pointer block's key-value pair(s) (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes throughout the overlay. |
1907 |
\item The query originator requests the provider peer to return the scroll block. |
\item The query originator requests the provider peer to return the block. |
1908 |
\end{enumerate} |
\end{enumerate} |
1909 |
\end{itemize} |
\end{itemize} |
1910 |
|
|
1927 |
security technologies. For the Fenfire system, one security related problem occurs when a user wants to |
security technologies. For the Fenfire system, one security related problem occurs when a user wants to |
1928 |
perform a global data lookup with a given pointer; how the user is able to verify |
perform a global data lookup with a given pointer; how the user is able to verify |
1929 |
the correctness of the search results, i.e., how she or he knows which one is the |
the correctness of the search results, i.e., how she or he knows which one is the |
1930 |
correct Storm scroll block ? Another problem related to the Fenfire's |
correct Storm block ? Another problem related to the Fenfire's |
1931 |
security is that if a user downloads data from the network to local computer |
security is that if a user downloads data from the network to local computer |
1932 |
and after a network disconnection, user wants to verify \emph{off line} the |
and after a network disconnection, user wants to verify \emph{off line} the |
1933 |
authenticity of data. Finally, if a data lookup is performed by a user, but there is no reply |
authenticity of data. Finally, if a data lookup is performed by a user, but there is no reply |
1968 |
|
|
1969 |
Our future work includes a support for searching transclusions and xanalogical |
Our future work includes a support for searching transclusions and xanalogical |
1970 |
links in Peer-to-Peer environment. Preliminary analysis have shown |
links in Peer-to-Peer environment. Preliminary analysis have shown |
1971 |
that these questions are rather different than locating scroll or pointer |
that these questions are rather different than locating Storm blocks |
1972 |
blocks from Peer-to-Peer environment. Techniques used in distributed |
from Peer-to-Peer environment. Techniques used in distributed |
1973 |
database systems may prove to be useful. Some fundamental results |
database systems may prove to be useful. Some fundamental results |
1974 |
regarding Peer-to-Peer and database systems have already been |
regarding Peer-to-Peer and database systems have already been |
1975 |
presented in \cite{gribble01p2pdatabase}. |
presented in \cite{gribble01p2pdatabase}. |