42 |
\abstract{ |
\abstract{ |
43 |
In this thesis, we review existing Peer-to-Peer approaches, algorithms and their |
In this thesis, we review existing Peer-to-Peer approaches, algorithms and their |
44 |
key properties. We summarize open problems in Peer-to-Peer systems and divide |
key properties. We summarize open problems in Peer-to-Peer systems and divide |
45 |
problems into three sub-categories. We observe that there are many |
problems into three sub-categories. We observe that there are many problems with |
46 |
problems which have no solutions at all, or problems that have proposed |
either no solutions at all, or only practically unrealizable ones. |
|
solutions but which are practically unrealizable. |
|
47 |
|
|
48 |
Then, we give an overview of the Fenfire system. We evaluate existing |
Then, we give an overview of the Fenfire system. We evaluate existing |
49 |
Peer-to-Peer approaches-- loosely and tightly structured overlays-- with regard |
Peer-to-Peer approaches-- loosely and tightly structured overlays-- with regard |
83 |
academia \cite{projectirisurl} and industry \cite{p2pworkinggroup, jxtaurl} for a |
academia \cite{projectirisurl} and industry \cite{p2pworkinggroup, jxtaurl} for a |
84 |
number of reasons. The lack of centralization in Peer-to-Peer systems |
number of reasons. The lack of centralization in Peer-to-Peer systems |
85 |
means that the participants can form a distributed system without any investment to |
means that the participants can form a distributed system without any investment to |
86 |
centralized hardware which would coordinate it by sharing their services |
centralized hardware, which would coordinate it by sharing their services |
87 |
and connecting to each other directly. Additionally, the distributed and ad hoc nature of |
and connecting to each other directly. Additionally, the distributed and ad hoc nature of |
88 |
Peer-to-Peer improves scalability and avoids single points of failure. |
Peer-to-Peer improves scalability and avoids single points of failure. |
89 |
|
|
94 |
capabilities and responsibilities. Each entity, i.e., \emph{peer}, may contribute services |
capabilities and responsibilities. Each entity, i.e., \emph{peer}, may contribute services |
95 |
to the overall system. |
to the overall system. |
96 |
|
|
97 |
The Fenfire project is an attempt to build hyperstructured, seamlessly interoperating desktop |
The Fenfire project is an attempt to build a hyperstructured, seamlessly interoperating desktop |
98 |
environment. In Fenfire, all data is stored as data blocks. |
environment. In the Fenfire, all data is stored as data blocks. |
99 |
All data blocks have globally unique identifiers and they can be referred by pointer blocks. |
All data blocks have globally unique identifiers and they can be referred by pointer blocks. |
100 |
Other features of Fenfire include innovative user |
Other features of the Fenfire include innovative user |
101 |
interfaces for viewing data and usage of Peer-to-Peer networking for network transparency. |
interfaces for viewing data and usage of Peer-to-Peer networking for network transparency. |
102 |
|
|
103 |
In this thesis, we\footnote{Use of the plural is customary even if research |
In this thesis, we evaluate existing Peer-to-Peer approaches and |
|
paper is authored solely.} evaluate existing Peer-to-Peer approaches and |
|
104 |
choose the best alternative to Fenfire's needs. |
choose the best alternative to Fenfire's needs. |
105 |
|
|
106 |
We start by reviewing existing Peer-to-Peer approaches, algorithms and their key properties. |
We start by reviewing existing Peer-to-Peer approaches, algorithms and their key properties. |
107 |
We observe that despite the great amount of proposed Peer-to-Peer systems, all systems fall either to |
We observe that despite the great amount of proposed Peer-to-Peer systems, all systems fall either to loosely or |
108 |
loosely structured approach or tightly structured approach. We also discuss open problems in |
tightly structured approach. We also discuss open problems in |
109 |
Peer-to-Peer systems and divide problems into three sub-categories: security problems, |
Peer-to-Peer systems and divide problems into three sub-categories: security, performance, and miscellaneous |
110 |
performance problems and miscellaneous problems. |
problems. |
111 |
|
|
112 |
Then, we give an overview of Fenfire project, and evaluate Peer-to-Peer approaches to Fenfire's |
Then, we give an overview of the Fenfire project, and evaluate Peer-to-Peer approaches to Fenfire's |
113 |
needs. Finally, we propose simple but yet efficient methods to be used for data lookups in Peer-to-Peer |
needs. Finally, we propose simple but yet efficient methods to be used for data lookups in Peer-to-Peer |
114 |
environment. |
environment. |
115 |
|
|
118 |
information can be found from the references. |
information can be found from the references. |
119 |
|
|
120 |
There are three research problems discussed in this thesis. First research problem |
There are three research problems discussed in this thesis. First research problem |
121 |
is to find the most efficient way to locate and fetch Fenfire data blocks from a |
is finding the most efficient way to locate and fetch Fenfire data blocks from a |
122 |
Peer-to-Peer network, where the block's identifier is given. Second, we want |
Peer-to-Peer network, where the block's identifier is given. Second, we want |
123 |
to find the most efficient way to locate and fetch the most recent Fenfire data block from a |
to find the most efficient way to locate and fetch the most recent Fenfire data block from a |
124 |
Peer-to-Peer network referred by a pointer block. The third problem |
Peer-to-Peer network referred by a pointer block. The third problem |
128 |
This thesis is structured as follows. In the next chapter, we give an overview of |
This thesis is structured as follows. In the next chapter, we give an overview of |
129 |
existing Peer-to-Peer approaches, algorithms and key differences between them. In chapter 3, we |
existing Peer-to-Peer approaches, algorithms and key differences between them. In chapter 3, we |
130 |
address open problems in Peer-to-Peer domain and divide problems into three |
address open problems in Peer-to-Peer domain and divide problems into three |
131 |
sub-categories. Chapter 4 gives an overview of Fenfire system. In chapter |
sub-categories. Chapter 4 gives an overview of the Fenfire system. In chapter |
132 |
5, we evaluate existing Peer-to-Peer approaches with regard to Fenfire system. |
5, we evaluate existing Peer-to-Peer approaches with regard to the Fenfire system. |
133 |
Finally, in chapter 6 we conclusions and future work. |
Finally, in chapter 6 we conclusions and future work. |
134 |
|
|
135 |
|
|
136 |
\chapter{Peer-to-Peer architectures} |
\chapter{Peer-to-Peer architectures} |
137 |
In this chapter we will give a brief history and overview of Peer-to-Peer networks, |
In this chapter we will give a brief history and overview of Peer-to-Peer networks, |
138 |
review most important Peer-to-Peer algorithms and list key differences between |
review most the important Peer-to-Peer algorithms and list key differences between the |
139 |
two main approaches. |
two main approaches. |
140 |
|
|
141 |
\section{Brief history and overview} |
\section{Brief history and overview} |
142 |
|
|
143 |
The Internet has been originally established in the late 1960s. The objective |
The Internet was originally established in the late 1960s. The objective |
144 |
of the ARPANET-project was to share computers' resources among military computers |
of the ARPANET-project was to share computers' resources among military computers |
145 |
around the United States. The most challenging purpose of ARPANET was to integrate |
around the United States. The most challenging purpose of ARPANET was to integrate |
146 |
different kinds of existing network technologies with one common network architecture. |
different kinds of existing network technologies with one common network architecture. |
147 |
The ARPANET connected the first few hosts together not in client/server relationship, |
The ARPANET connected the first few hosts together not in client/server relationship, |
148 |
but rather as equal networking \emph{peers}. This could be seen as starting point |
but rather as equal networking \emph{peers}. This could be seen as the starting point |
149 |
both of Peer-to-Peer concept and the Internet \cite{oram01harnessingpower}. |
of both the Peer-to-Peer concept and the Internet \cite{oram01harnessingpower}. |
150 |
|
|
151 |
In subsequent years, the Internet became more restricted to client--server based |
In subsequent years, the Internet became more restricted to client--server based |
152 |
applications. In recent years, however, Peer-to-Peer systems have again emerged |
applications. In recent years, however, Peer-to-Peer systems have again emerged |
153 |
room in computing world. Indeed, Peer-to-Peer has had significant social and technical |
in computing world. Indeed, Peer-to-Peer has had significant social and technical |
154 |
attention in academia \cite{projectirisurl}, industry \cite{p2pworkinggroup}, |
attention in academia \cite{projectirisurl} and industry \cite{p2pworkinggroup, jxtaurl}. |
155 |
\cite{jxtaurl}. Already deceased Napster \cite{napsterurl}, |
The deceased Napster \cite{napsterurl}, |
156 |
launched in 1999, was a new starting point for modern Peer-to-Peer computing. After |
launched in 1999, was a new starting point for modern Peer-to-Peer computing. After |
157 |
Napster, hundreds of Peer-to-Peer systems have been developed and proposed. |
Napster, hundreds of Peer-to-Peer systems have been developed and proposed. |
158 |
|
|
159 |
Modern Peer-to-Peer system is composed of \emph{application} level overlay network. |
A modern Peer-to-Peer system is composed of an \emph{application} level overlay network. |
160 |
Figure \ref{fig:application_level} illustrates the analogy of Peer-to-Peer network with |
Figure \ref{fig:application_level} illustrates the analogy of Peer-to-Peer network with |
161 |
regard to OSI model. Compared to ARPANET's Peer-to-Peer functionality, modern Peer-to-Peer systems |
regard to OSI model. Compared to ARPANET's Peer-to-Peer functionality, modern Peer-to-Peer systems |
162 |
are ad hoc, i.e., peers join and leave the system constantly in a dynamic manner. This |
are ad hoc, i.e., peers join and leave the system constantly in a dynamic manner. This |
163 |
fact constitutes challenging requirements for efficient construction and maintenance |
fact constitutes challenging requirements for efficient construction and maintenance |
164 |
of the overlay network. Even more demanding tasks are how to perform efficient data |
of the overlay network. Even more demanding tasks are performing efficient data |
165 |
lookup and maintain security in a varying distributed environment. The most popular |
lookup and maintaining security in a varying distributed environment. The most popular |
166 |
form of modern Peer-to-Peer computing is file-sharing. In this scenario, participants |
form of modern Peer-to-Peer computing is file-sharing. In this scenario, participants |
167 |
of Peer-to-Peer network share their file resources to other participants while obtaining |
of Peer-to-Peer network share their file resources with other participants. |
168 |
more resources from others. This can be seen as a variant of distributed file system |
This can be seen as a variant of distributed file system |
169 |
(e.g., \cite{levy90distributedfilesystems}). |
(e.g., \cite{levy90distributedfilesystems}). |
170 |
|
|
171 |
\begin{figure} |
\begin{figure} |
177 |
|
|
178 |
|
|
179 |
|
|
180 |
In a development of modern Peer-to-Peer systems, lot of influences has been attained from |
In the development of modern Peer-to-Peer systems, lot of influence has been attained from |
181 |
other research areas than computer science. Research has been conducted regarding |
other research areas than computer science. Research has been conducted regarding |
182 |
to self-organizing nature of complex networks \cite{albert-02-statistical}, \cite{albert-00-tolerance}, \cite{watts00dynamics}. |
the self-organizing nature of complex networks \cite{albert-02-statistical, albert-00-tolerance, watts00dynamics}. |
183 |
It's interesting to realize that chemical properties of cells, the Internet, ad hoc |
It is interesting to realize that chemical properties of biological cells, the Internet, ad hoc |
184 |
Peer-to-Peer systems, and social networks have all in common that they self-organize based on same |
Peer-to-Peer systems, and social networks have all in common that they self-organize based on the same |
185 |
principles. Furthermore, the association between social relationships among people |
principles. Furthermore, the association between social relationships among people |
186 |
and Peer-to-Peer overlay topology has been studied recently \cite{watts00dynamics}, |
and Peer-to-Peer overlay topology has been studied recently \cite{watts00dynamics, kleinberg99small, nips02-Kleinberg}. |
187 |
\cite{kleinberg99small}, \cite{nips02-Kleinberg}. This insight is motivated |
This insight is motivated by Milgram, who noticed that people are very effective in locating other people in a wide scale |
|
by Milgram, who noticed that people are very effective to locate other people in a wide scale |
|
188 |
based on local knowledge. This phenomenon is called as ''small-world phenomenon'' |
based on local knowledge. This phenomenon is called as ''small-world phenomenon'' |
189 |
\cite{milgram67smallworld}. As a consequence, many modern Peer-to-Peer systems |
\cite{milgram67smallworld}. As a consequence, many modern Peer-to-Peer systems |
190 |
have applied techniques outside of computer science when constructing and maintaining |
have applied techniques outside of computer science when constructing and maintaining |
191 |
the application level overlay network. |
the application level overlay network. |
192 |
|
|
193 |
In the end, however, there are two main approaches in which all modern Peer-to-Peer |
In the end, however, there are two main approaches in which all modern Peer-to-Peer |
194 |
systems fall: loosely structured approach and tightly structured approach. In loosely |
systems fall: the loosely structured approach and the tightly structured approach. In the loosely |
195 |
structured approach the construction and the maintenance of the overlay is controlled |
structured approach the construction and the maintenance of the overlay is controlled |
196 |
loosely. This approach gives freedom for participating peers |
loosely. This approach gives freedom for participating peers |
197 |
to perform certain tasks in Peer-to-Peer network. On the other hand, tightly structured |
to perform certain tasks in a Peer-to-Peer network. On the other hand, the tightly structured |
198 |
approach has some rules, which all participating peers have to obey. In the following |
approach has some rules, which all participating peers have to obey. |
|
sections, we will discuss in more detail both approaches and key differences between them. |
|
199 |
|
|
200 |
|
|
201 |
\section{Centralized} |
\section{Centralized} |
204 |
historical value (see previous section).} \cite{napsterurl} was designed to allow |
historical value (see previous section).} \cite{napsterurl} was designed to allow |
205 |
people to share music. It was a hybrid Peer-to-Peer file-sharing system, i.e., the search |
people to share music. It was a hybrid Peer-to-Peer file-sharing system, i.e., the search |
206 |
index was centralized and the distribution of storage and serving of files was distributed. |
index was centralized and the distribution of storage and serving of files was distributed. |
207 |
Peers in the Napster network performed requests to the central directory server to find |
Peers in the Napster network made requests to the central directory server to find |
208 |
other peers hosting desirable content. Since service requests were totally based on |
other peers hosting desirable content. Since service requests were totally based on a |
209 |
centralized index, Napster didn't scale well because of constantly updated central |
centralized index, Napster didn't scale well because of constantly updated central |
210 |
directory, and had a single point of failure. |
directory, and had a single point of failure. |
211 |
|
|
217 |
The construction and maintenance of Gnutella network is extremely ad hoc, since participating |
The construction and maintenance of Gnutella network is extremely ad hoc, since participating |
218 |
peers can form the overlay network based on \emph{local} knowledge. Figure \ref{fig:gnutella_overlay} |
peers can form the overlay network based on \emph{local} knowledge. Figure \ref{fig:gnutella_overlay} |
219 |
illustrates how peers form an overlay network. Initially, peer 1 creates the overlay, since |
illustrates how peers form an overlay network. Initially, peer 1 creates the overlay, since |
220 |
it's the first participating peer. Then, repeatedly new peers join the network and connect to |
it is the first participating peer. Then, repeatedly new peers join the network and connect to |
221 |
other peers in a random manner. Thus, Gnutella can be considered as a variation of \emph{scale-free |
other peers in a random manner. Thus, Gnutella can be considered as a variation of \emph{scale-free |
222 |
graph}\footnote{In scale-free graphs (also known as power-law graphs) only a few peers have high number of neighbor |
graph}\footnote{In scale-free graphs (also known as power-law graphs) only a few peers have high number of neighbor |
223 |
links and major of peers have low number of neighbor links.}. |
links and the majority of peers have low number of neighbor links.}. |
224 |
|
|
225 |
\begin{figure} |
\begin{figure} |
226 |
\centering |
\centering |
230 |
\end{figure} |
\end{figure} |
231 |
|
|
232 |
|
|
233 |
In Gnutella, each participating peer maintains local index of its own shared content. Also, |
In Gnutella, each participating peer maintains a local index of its own shared content. Also, |
234 |
each peer has a few connections to other peer, i.e., peer's \emph{neighbors}. Basic Gnutella |
each peer has a few connections to other peers, i.e., peer's \emph{neighbors}. Basic Gnutella |
235 |
data lookup works as follows: peer broadcasts a query request to its neighbors, which in turn |
data lookup works as follows: peer broadcasts a query request to its neighbors, which in turn |
236 |
forwards the query to their neighbors. This leads to a situation where number of messages |
forward the query to their neighbors. This leads to a situation where the number of messages |
237 |
in the network can grow with $O(n^{2})$, where $n$ is the number of participating peers in |
in the network can grow with $O(n^{2})$, where $n$ is the number of participating peers in |
238 |
Gnutella network. To limit the amount of network traffic, Gnutella uses Time-To-Live-limited |
Gnutella network. To limit the amount of network traffic, Gnutella uses Time-To-Live-limited |
239 |
(TTL) flooding to distribute queries. Therefore, Gnutella uses a Breadth-First-Search (BFS) algorithm |
(TTL) flooding to distribute queries. Therefore, Gnutella uses a Breadth-First-Search (BFS) algorithm |
250 |
\label{fig:gnutella_query} |
\label{fig:gnutella_query} |
251 |
\end{figure} |
\end{figure} |
252 |
|
|
253 |
According to \cite{lv02searchreplication}, Gnutella's way to perform data lookups, \emph{flooding}, has |
According to \cite{lv02searchreplication}, Gnutella's way to perform data lookups, \emph{flooding}, has the |
254 |
following limitations. First, choosing the appropriate TTL in practice is not easy. If the |
following limitations. First, choosing the appropriate TTL in practice is not easy. If the |
255 |
TTL is too high, query originator may unnecessarily strain the network. If the TTL is too |
TTL is too high, query originator may unnecessarily strain the network. If the TTL is too |
256 |
low, the query originator might not find the desired data even if it's available somewhere |
low, the query originator might not find the desired data even if it is available somewhere |
257 |
in the network. Second, there are many duplicate messages generated by flooding, especially |
in the network. Second, there are many duplicate messages generated by flooding, especially |
258 |
in high connectivity graphs. It is obvious that with these limitations, flooding creates |
in high connectivity graphs. It is obvious that with these limitations, flooding creates |
259 |
significant message processing overhead for each data lookup. Even worse, flooding may increase |
significant message processing overhead for each data lookup. Even worse, flooding may increase |
260 |
the load on participating to the point, where it has to leave the network. |
the load on participating peer to the point where it has to leave the network. |
261 |
|
|
262 |
Lately, there has been done lot of research to improve Gnutella's data lookup efficiency |
Lately, Gnutella's data lookup efficiency and scalability has been deeply researched. |
263 |
and scalability. Adamic et al. \cite{adamic99small}, \cite{adamic02localsearch}, |
Adamic et al. \cite{adamic99small, adamic02localsearch, adamic01powerlawsearch} |
264 |
\cite{adamic01powerlawsearch} have studied different data lookup methods in power-law |
have studied different data lookup methods in power-law networks and have found that by |
265 |
networks and have found that by |
instructing the peers that forward data lookups to select high degree peers, the performance of data lookup |
|
instructing peers forwarding data lookups to select high degree peers, the performance of data lookup |
|
266 |
increases significantly. As a result, some of the most recent loosely |
increases significantly. As a result, some of the most recent loosely |
267 |
structured Peer-to-Peer systems have adopted this method with some modifications |
structured Peer-to-Peer systems have adopted this method with some modifications |
268 |
\cite{gnutella2url}, \cite{shareazaurl}, \cite{fasttrackurl}, \cite{morpheusurl}, |
\cite{gnutella2url, shareazaurl, fasttrackurl, morpheusurl, kazaaurl, waterhouse02searchp2p, botros01jxtasearch, |
269 |
\cite{kazaaurl}, \cite{waterhouse02searchp2p}, \cite{botros01jxtasearch}, |
ganesan02yappers}. |
|
\cite{ganesan02yappers}. |
|
270 |
Figures \ref{fig:gnutella_overlay_supernodes} and \ref{fig:gnutella_overlay_cluster} |
Figures \ref{fig:gnutella_overlay_supernodes} and \ref{fig:gnutella_overlay_cluster} |
271 |
illustrates simplified variations of power-law overlay networks. Figure \ref{fig:gnutella_powerlaw} |
illustrates simplified variations of power-law overlay networks. Figure \ref{fig:gnutella_powerlaw} |
272 |
presents pure topology of power-law network. All the systems |
presents pure topology of power-law network. |
273 |
share the property of that high degree peers maintain index of all other peers |
|
274 |
they know about. However, it's not clear whether this algorithm is scalable or not, |
It is not clear whether this algorithm is scalable or not, |
275 |
as majority of the query requests are sent only to the high degree peers, making |
as the majority of the query requests are sent only to the high degree peers, making |
276 |
them stress the load of entire system. |
them stress the load of entire system. |
277 |
|
|
278 |
\begin{figure} |
\begin{figure} |
297 |
\end{figure} |
\end{figure} |
298 |
|
|
299 |
Previously presented improvements are only partial solutions. More advanced techniques |
Previously presented improvements are only partial solutions. More advanced techniques |
300 |
to improve data lookup of loosely structured systems are presented in chapter 3. |
to improve data lookup of loosely structured systems are discussed in chapter 3. |
301 |
|
|
302 |
|
|
303 |
\subsection{Sketch of formal definition} |
\subsection{Sketch of a formal definition} |
304 |
|
|
305 |
In this subsection we formalize loosely structured overlay's main components. This |
In this subsection we formalize loosely structured overlay's main components. This |
306 |
model is based on original Gnutella overlay network with power-law improvements. |
model is based on original Gnutella overlay network with power-law improvements. |
307 |
|
|
308 |
Let $S$ be the aggregate of all services $s$ in system. Let $P$ be the aggregate of |
Let $S$ be the aggregate of all services $s$ in system. Let $P$ be the aggregate of |
309 |
all peers $p$ in system. Then, $\forall s \in S$, there is a provider of the service, |
all peers $p$ in system. Then, $\forall s \in S$, there is a provider of the service, |
310 |
expressed as $p = provider(s)$. Every $p$ has neighbor(s), named as $neighbor$, which |
expressed as $p = provider(s)$. Every $p$ has neighbor(s), named as $p_n$, which |
311 |
is $P$ = \{$p \in P: \exists neighbor$, which is randomly chosen from $P$\}. |
is $P$ = \{$p \in P: \exists neighbor$, which is randomly chosen from $P$\}. |
312 |
\emph{Super peer} is a peer, which hosts the indices of other peers, $si = summaryindex(provider(s))$ |
\emph{Super peer} is a peer, which hosts the indices of other peers, $si = summaryindex(provider(s))$ |
313 |
and $\forall$ regular peer $p$, there is super peer, which has has a index of regular |
and $\forall$ regular peer $p$, there is super peer, which has has a index of regular |
326 |
Skip Graphs \cite{AspnesS2003}, SkipNet \cite{harvey03skipnet2}, |
Skip Graphs \cite{AspnesS2003}, SkipNet \cite{harvey03skipnet2}, |
327 |
Symphony \cite{gurmeet03symphony}, SWAN \cite{bonsma02swan}, Tapestry |
Symphony \cite{gurmeet03symphony}, SWAN \cite{bonsma02swan}, Tapestry |
328 |
\cite{zhao01tapestry}, Viceroy \cite{malkhi02viceroy} and others \cite{freedman02trie}. |
\cite{zhao01tapestry}, Viceroy \cite{malkhi02viceroy} and others \cite{freedman02trie}. |
329 |
The biggest difference compared to loosely structured approach is that with tightly structured systems, |
The biggest difference compared to the loosely structured approach is that with tightly structured systems, |
330 |
it is now feasible to perform \emph{global} data lookups in the overlay. |
it is now feasible to perform \emph{global} data lookups in the overlay. |
331 |
While there are significant differences among proposed tighty structured systems, they all have in common |
While there are significant differences among proposed tighty structured systems, they all have in common |
332 |
that \emph{peer identifiers} are assigned to participating peers from |
that \emph{peer identifiers} are assigned to participating peers from |
336 |
space differs between proposed systems. Circular identifier space (and variants) |
space differs between proposed systems. Circular identifier space (and variants) |
337 |
is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde}, |
is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde}, |
338 |
Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry} |
Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry} |
339 |
and Viceroy \cite{malkhi02viceroy} use circular identifier space of $n$-bit integers modulo $2^{n}$. The |
and Viceroy \cite{malkhi02viceroy} use a circular identifier space of $n$-bit integers modulo $2^{n}$. The |
340 |
value of $n$ varies among systems. Again, CAN \cite{ratnasamy01can} uses a $d$-dimensional Cartesian |
value of $n$ varies among systems. Again, CAN \cite{ratnasamy01can} uses a $d$-dimensional Cartesian |
341 |
model to implement identifier space. |
model to implement identifier space. |
342 |
|
|
343 |
To store data into tightly structured overlay, each application-specific |
To store data into a tightly structured overlay, each application-specific |
344 |
unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent |
unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent |
345 |
hashing \cite{258660}) by the overlay to an existing peer in the overlay. Thus, tightly |
hashing \cite{258660}) by the overlay to an existing peer in the overlay. Thus, tightly |
346 |
structured overlay assigns a subset of all possible keys to every participating peer |
structured overlay assigns a subset of all possible keys to every participating peer. |
347 |
\footnote{We say that a peer is \emph{responsible} for the keys which are assigned by the overlay.}. |
We say that a peer is \emph{responsible} for the keys which are assigned by the overlay. |
348 |
Also, each peer in tightly structured overlay maintains a \emph{routing table}, which |
Also, each peer in tightly structured overlay maintains a \emph{routing table}, which |
349 |
consists of identifiers and IP addresses of other peers in the overlay. Entries of routing |
consists of identifiers and IP addresses of other peers in the overlay. Entries of the routing |
350 |
table are peer's neighbors in the overlay network. Figure \ref{fig:structured_hashing} illustrates the |
table represents peer's neighbors in the overlay network. Figure \ref{fig:structured_hashing} illustrates the |
351 |
process of data to key mapping in tightly structured overlays. |
process of data to key mapping in a tightly structured overlay. |
352 |
|
|
353 |
\begin{figure} |
\begin{figure} |
354 |
\centering |
\centering |
361 |
peer identifier is gradually ''closer'' to the key's identifier |
peer identifier is gradually ''closer'' to the key's identifier |
362 |
in the identifier space. The distance can be measured by numerical |
in the identifier space. The distance can be measured by numerical |
363 |
difference between identifiers (e.g., Chord \cite{stoica01chord}), number of |
difference between identifiers (e.g., Chord \cite{stoica01chord}), number of |
364 |
same prefix bits between identifiers (e.g., Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry}) or |
same prefix bits between identifiers (e.g., Pastry \cite{rowston01pastry} and Tapestry |
365 |
bit-wise exclusive or (XOR) (e.g., Kademlia \cite{maymounkov02kademlia}). |
\cite{zhao01tapestry}) or bit-wise exclusive or (XOR) (e.g., Kademlia \cite{maymounkov02kademlia}). |
366 |
Because of XOR-metric, Kademlia's distance function is both unidirectional |
Chord's \cite{stoica01chord} distance function does have the property of unidirection |
367 |
(for a given point $p_i$ in the identifier space and distance $d$ > 0, there |
(for a given point $p_i$ in the identifier space and distance $d$ > 0, there |
368 |
is exactly one point $p_j$ in a way that the distance between $p_i$ and $p_j$ |
is exactly one point $p_j$ in a way that the distance between $p_i$ and $p_j$ |
369 |
is $d$) and symmetric (the distance from $p_i$ to $p_j$ is same as the |
is $d$), but doesn't have symmetry (the distance from $p_i$ to $p_j$ is same as the |
370 |
distance from $p_j$ to $p_i$) \cite{maymounkov02kademlia}. On the other |
distance from $p_j$ to $p_i$). Pastry's \cite{rowston01pastry} distance function supports |
371 |
hand, Chord's \cite{stoica01chord} distance function does have the property |
symmetry, but doesn't support unidirection. Because of XOR-metric, Kademlia's distance |
372 |
of unidirection, but doesn't have symmetry. Pastry's \cite{rowston01pastry} distance |
function is both unidirectional and symmetric. Moreover, Kademlia's \cite{maymounkov02kademlia} |
373 |
function supports symmetry, but doesn't support unidirection. As a consequence, |
XOR-based metric doesn't need stabilization (like in Chord \cite{stoica01chord}) and backup links |
|
Kademlia's \cite{maymounkov02kademlia} XOR-based metric doesn't need |
|
|
stabilization (like in Chord \cite{stoica01chord}) and backup links |
|
374 |
(like in Pastry \cite{rowston01pastry}) \cite{balakrishanarticle03lookupp2p}. |
(like in Pastry \cite{rowston01pastry}) \cite{balakrishanarticle03lookupp2p}. |
375 |
However, in all previously schemes each |
However, in all previous schemes each hop in the overlay shortens the distance between |
376 |
hop in the overlay shortens the distance between current peer working with the data lookup |
current peer working with the data lookup and the key which was looked up in the identifier space. |
|
and the key which was looked up in the identifier space. |
|
377 |
|
|
378 |
Skip Graphs \cite{AspnesS2003} and SWAN \cite{bonsma02swan} employ a key space very similar to a tightly structured |
Skip Graphs \cite{AspnesS2003} and SWAN \cite{bonsma02swan} employ a key space very similar to a tightly structured |
379 |
overlay, but in which queries are routed to \emph{keys}. In these systems |
overlay, but in which queries are routed to \emph{keys}. In these systems |
380 |
peer occupies several positions in the identifier space, one for each |
a peer occupies several positions in the identifier space, one for each |
381 |
application-specific key. The indirection of placing close keys in the |
application-specific key. The indirection of placing close keys in the |
382 |
custody of a storing peer\footnote{Storing peer is the peer in the overlay which is responsible for the |
custody of a storing peer is removed at the cost of each peer maintaining one |
383 |
assigned keys.} is removed at the cost of each peer maintaining one |
''resource peer'' in the overlay network for each data item it publishes. Provider peer is the peer |
384 |
''resource peer'' in the overlay network for each data item it publishes. |
in the overlay which is responsible for the assigned keys |
385 |
|
|
386 |
PeerNet \cite{eriksson03peernet} differs from other tightly structured overlays in that it operates |
PeerNet \cite{eriksson03peernet} differs from other tightly structured overlays in that it operates |
387 |
at the \emph{network} layer. PeerNet makes an explicit distinction |
at the \emph{network} layer. PeerNet makes an explicit distinction |
391 |
for maintaining information about other peers in the system and |
for maintaining information about other peers in the system and |
392 |
$O(\log{n})$ data lookup efficiency. |
$O(\log{n})$ data lookup efficiency. |
393 |
|
|
394 |
Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} have listed |
Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} which have to be |
|
four requirements for tightly structured overlays, which have to be |
|
395 |
addressed in order to perform efficient data lookups in tightly structured overlays. |
addressed in order to perform efficient data lookups in tightly structured overlays. |
396 |
First, mapping of keys to peers must be done in a load-balanced |
First, mapping of keys to peers must be done in a load-balanced |
397 |
way. Second, the overlay must be able to forward a lookup for a |
way. Second, the overlay must be able to forward a lookup for a |
398 |
specific key to an appropriate peer. Third, overlay must have a |
specific key to an appropriate peer. Third, overlay must have |
399 |
support for a distance function. Finally, routing tables for each peer |
support for a efficient distance function. Finally, routing tables for each peer |
400 |
must be constructed and maintained adaptively. |
must be constructed and maintained adaptively. |
401 |
|
|
402 |
Currently, all proposed tightly structured overlays provide at least |
Currently, all proposed tightly structured overlays provide at least |
407 |
In figure \ref{fig:structured_query}, we present an overview of Chord's lookup process. |
In figure \ref{fig:structured_query}, we present an overview of Chord's lookup process. |
408 |
On the right side of Chord's lookup process, the same data lookup process |
On the right side of Chord's lookup process, the same data lookup process |
409 |
is shown as a binary-tree abstraction. It can be noticed, that in each step, the distance |
is shown as a binary-tree abstraction. It can be noticed, that in each step, the distance |
410 |
decreases between with a logarithmic efficiency. |
decreases with a logarithmic efficiency. |
411 |
|
|
412 |
Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry |
Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry |
413 |
\cite{zhao01tapestry} uses balanced $k$-trees as routing table's data structure. Figure |
\cite{zhao01tapestry} uses balanced $k$-trees as routing table's data structure. Figure |
414 |
\ref{fig:kademlia_lookup} shows the process of Kademlia's |
\ref{fig:kademlia_lookup} shows the process of Kademlia's |
415 |
data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}), |
data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}), |
416 |
which requires only constant number of neighbor peers while providing $O(\log{n})$ data lookup |
which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup |
417 |
efficiency. Koorde \cite{kaashoek03koorde}, recent modification of Chord, uses de Bruijn graphs |
efficiency. Koorde \cite{kaashoek03koorde}, a recent modification of Chord, uses de Bruijn graphs |
418 |
\cite{debruijn46graph} to maintain local routing tables. Koorde \cite{kaashoek03koorde} requires |
\cite{debruijn46graph} to maintain local routing tables. Koorde \cite{kaashoek03koorde} requires |
419 |
each peer to have only about two links to other peers to provide $O(\log{n})$ performance. |
each peer to have only about two links to other peers to provide $O(\log{n})$ performance. |
420 |
|
|
442 |
\texttt{insert(key)}. As the name suggests, DHT implements the same functionality |
\texttt{insert(key)}. As the name suggests, DHT implements the same functionality |
443 |
as a regular hash table, by storing the mapping between a key and a value. DHT's |
as a regular hash table, by storing the mapping between a key and a value. DHT's |
444 |
\emph{interface} is generic; values can be any size and type. Figure \ref{fig:Structured_lookup_using_DHT_model} |
\emph{interface} is generic; values can be any size and type. Figure \ref{fig:Structured_lookup_using_DHT_model} |
445 |
shows the DHT abstraction of tightly structured overlay. Second, Decentralized |
shows the DHT abstraction of the tightly structured overlay. Second, Decentralized |
446 |
Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is distributed |
Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is a distributed |
447 |
directory service. DOLR stores \emph{pointers} to where data items are stored |
directory service. DOLR stores \emph{pointers} to data items throughout the overlay. DOLR's main |
448 |
throughout the overlay. DOLR's main operations are \texttt{publish(key)}, |
operations are \texttt{publish(key)}, \texttt{removePublished(key)} and \texttt{sendToObject(key)}. The key |
|
\texttt{removePublished(key)} and \texttt{sendToObject(key)}. The key |
|
449 |
difference between DHT and DOLR abstraction is that DOLR routes overlay's messages |
difference between DHT and DOLR abstraction is that DOLR routes overlay's messages |
450 |
to nearest available peer, hosting a specific data item. This form of locality |
to nearest available peer, hosting a specific data item. This form of locality |
451 |
is not supported by DHT. Finally, tightly structured overlay can be used for |
is not supported by DHT. Finally, tightly structured overlay can be used for |
474 |
\end{figure} |
\end{figure} |
475 |
|
|
476 |
|
|
477 |
\subsection{Sketch of formal definition} |
\subsection{Sketch of a formal definition} |
478 |
|
|
479 |
In this subsection we formalize tightly structured overlay's main features, i.e., |
In this subsection, we formalize the main features of tightly structured overlay, i.e., |
480 |
identifiers, identifier space and mapping function. |
identifiers, identifier space and mapping function. |
481 |
|
|
482 |
Let $S$ be the aggregate of all services $s$ in system. Let $P$ be the aggregate of |
Let $S$ be the aggregate of all services $s$ in the system. Let $P$ be the aggregate of |
483 |
all peers $p$ in system. Let $I$ be the aggregate of all identifiers $i$ in system. |
all peers $p$ in system. Let $I$ be the aggregate of all identifiers $i$ in system. |
484 |
Let $IS$ be the aggregate of all identifier points $ip$ in system. Then, $\forall s \in S$, |
Let $IS$ be the aggregate of all identifier points $ip$ in system. Then, $\forall s \in S$, |
485 |
there is a provider of the service, expressed as $p = provider(s)$. Service's identifier |
there is a provider of the service, expressed as $p = provider(s)$. Service's identifier |
495 |
|
|
496 |
\section{Summary} |
\section{Summary} |
497 |
|
|
498 |
In this section we compare loosely structured approach and tightly structured approach. |
In this section we compare the loosely structured approach and the tightly structured approach. |
499 |
We also summarize proposed Peer-to-Peer algorithms and their key properties with regard |
We also summarize proposed Peer-to-Peer algorithms and their key properties with regard |
500 |
to performance and scalability aspects. |
to performance and scalability aspects. |
501 |
|
|
502 |
\subsection{Differences} |
\subsection{Differences} |
503 |
|
|
504 |
Even though loosely structured and tightly structured approach are both Peer-to-Peer schemes, they |
Even though the loosely structured and the tightly structured approach are both Peer-to-Peer schemes, they |
505 |
have very little in common. Indeed, the only thing they share is the fact that no other peer is more |
have very little in common. Indeed, the only thing they share is the fact that no other peer is more |
506 |
important than an other in the Peer-to-Peer network. Fault tolerance \emph{may} |
important than any other in the Peer-to-Peer network. Fault tolerance \emph{may} |
507 |
be an area, in which approaches have similar properties (e.g., no single point of failure). |
be an area, in which approaches have similar properties (e.g., no single point of failure). |
508 |
Fault tolerance properties of both approaches are currently only initial calculations, or |
Fault tolerance properties of both approaches are currently only initial calculations, or |
509 |
experimented in simulation environments. In real-life, however, measuring fault tolerance is much more |
experimented in simulation environments. In real-life, however, measuring fault tolerance is a much more |
510 |
challenging task and requires more research to get reliable answers. |
challenging task and requires more research to get reliable answers. |
511 |
|
|
512 |
The most important difference between approaches is performance and scalability properties. While |
The most important difference between approaches is performance and scalability properties. Generally |
513 |
performance of loosely structured approach is not always even linear, generally tightly structured |
tightly structured systems can perform all internal operations in poly-logarithmic time\footnote{However, it is unknown |
514 |
approach can perform all internal operations in poly-logarithmic time\footnote{However, it is unknown |
whether all proposed algorithms can preserve logarithmic properties in real-life applications or not.} |
515 |
whether all proposed algorithms can preserve logarithmic properties in real-life applications or not.}. |
while the performance of loosely structured systems is not always even linear, . |
516 |
Moreover, loosely structured systems scale to millions of peers, whereas tightly structured systems are able |
Moreover, loosely structured systems scale to millions of peers, whereas tightly structured systems are able |
517 |
to cope with billions of concurrent peers \cite{osokine02distnetworks}, \cite{kubiatowicz00oceanstore}. |
to cope with billions of concurrent peers \cite{osokine02distnetworks}, \cite{kubiatowicz00oceanstore}. |
518 |
|
|
519 |
To end user, biggest difference between these systems is how data lookups are performed. Loosely |
To end user, biggest difference between these systems is how data lookups are performed. Loosely |
520 |
structured systems provide more richer and user friendly way of searching data as they |
structured systems provide a more rich and user friendly way of searching data as they |
521 |
have support for keyword search. On the other hand, tightly structured systems support |
have support for keyword search than tightly structured systems. On the other hand, tightly structured |
522 |
only exact key lookups as each data item is identified by globally unique keys. |
systems support only exact key lookups as each data item is identified by globally unique keys. |
523 |
|
|
524 |
In the end, both systems have open problems and issues. We will discuss these aspects in more detail in |
In the end, both systems have open problems and issues. We will discuss these aspects in more detail in |
525 |
chapter 3. Table \ref{table_comparison_approach} lists the key differences between loosely structured |
chapter 3. Table \ref{table_comparison_approach} lists the key differences between the loosely structured |
526 |
approach and tightly structured approach. |
approach and the tightly structured approach. |
527 |
|
|
528 |
|
|
529 |
\scriptsize |
\scriptsize |
614 |
|
|
615 |
Table \ref{table_Peer-to-Peer_algorithms} lists proposed Peer-to-Peer algorithms |
Table \ref{table_Peer-to-Peer_algorithms} lists proposed Peer-to-Peer algorithms |
616 |
and their key properties with regard to performance and scalability. List |
and their key properties with regard to performance and scalability. List |
617 |
includes algorithms from both loosely and tightly structured approaches. However, majority of the algorithms |
includes algorithms from both loosely and tightly structured approaches. The list doesn't |
|
listed above belongs to tightly structured approach since there has been active |
|
|
research being pursued towards tightly structured approach lately. List doesn't |
|
618 |
include \emph{all} proposed Peer-to-Peer algorithms. Only the ones which already have |
include \emph{all} proposed Peer-to-Peer algorithms. Only the ones which already have |
619 |
been widely deployed in real life, or the ones which may be promising in the future |
been widely deployed in real life, or the ones which may be promising in the future |
620 |
Peer-to-Peer systems are included in this thesis. |
Peer-to-Peer systems are included in this thesis. |
621 |
|
|
622 |
We decided to follow the guidelines from \cite{kaashoek03koorde} when |
We decided to follow the guidelines from \cite{kaashoek03koorde} in measuring |
623 |
measuring properties of different Peer-to-Peer systems. However, we dropped |
the properties of different Peer-to-Peer systems. However, we dropped |
624 |
out fault tolerance and load balancing properties, since they are hard to measure |
out fault tolerance and load balancing properties, since they are hard to measure |
625 |
in face of real life requirements. Additionally, however, we decided to include |
in face of real life requirements. Additionally, however, we decided to include |
626 |
the number of \emph{real} network connections for each peer in the overlay. |
the number of \emph{real} network connections for each peer in the overlay. |
628 |
Here, we describe the listed properties of Peer-to-Peer algorithms: |
Here, we describe the listed properties of Peer-to-Peer algorithms: |
629 |
|
|
630 |
\begin{itemize} |
\begin{itemize} |
631 |
\item \textbf{Lookup}: number of messages required when a data lookup is performed |
\item \textbf{Lookup}: the number of messages required when a data lookup is performed |
632 |
\item \textbf{Space}: number of neighbors which peers knows about (neighbors) |
\item \textbf{Space}: the number of neighbors which peers knows about (neighbors) |
633 |
\item \textbf{Insert/delete}: number of messages required when a peer joins or leaves the network |
\item \textbf{Insert/delete}: the number of messages required when a peer joins or leaves the network |
634 |
\item \textbf{Number of network connections}: number of concurrent network connections required to maintain correct neighbor information |
\item \textbf{Number of network connections}: the number of concurrent network connections required to maintain correct neighbor information |
635 |
\end{itemize} |
\end{itemize} |
636 |
|
|
637 |
\scriptsize |
\scriptsize |
666 |
\parbox{37pt}{$O$($d$)} & |
\parbox{37pt}{$O$($d$)} & |
667 |
\parbox{37pt}{$O(dn^{\frac{1}{d}})$} & |
\parbox{37pt}{$O(dn^{\frac{1}{d}})$} & |
668 |
\parbox{85pt}{2$d$} & |
\parbox{85pt}{2$d$} & |
669 |
\parbox{85pt}{The performance of system may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, where $d$ is the dimension of virtual key space} |
\parbox{85pt}{System performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, where $d$ is the dimension of virtual key space} |
670 |
\\ \hline |
\\ \hline |
671 |
|
|
672 |
\parbox{37pt}{Chord \cite{stoica01chord}} & |
\parbox{37pt}{Chord \cite{stoica01chord}} & |
674 |
\parbox{37pt}{$O(\log{n}$} & |
\parbox{37pt}{$O(\log{n}$} & |
675 |
\parbox{37pt}{$O(\log{n})$} & |
\parbox{37pt}{$O(\log{n})$} & |
676 |
\parbox{85pt}{2$(\log{n})$} & |
\parbox{85pt}{2$(\log{n})$} & |
677 |
\parbox{85pt}{The performance of system may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner} |
\parbox{85pt}{System performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner} |
678 |
\\ \hline |
\\ \hline |
679 |
|
|
680 |
|
|
710 |
\parbox{37pt}{$O$($\sqrt{n}$)} & |
\parbox{37pt}{$O$($\sqrt{n}$)} & |
711 |
\parbox{37pt}{$O(1)$} & |
\parbox{37pt}{$O(1)$} & |
712 |
\parbox{85pt}{$\frac{n}{\sqrt{n}} + c*(\sqrt{n}-1) + \frac{Totalnumber of files}{\sqrt{n}}$, where n is the number of peers and c the number of contacts/foreign affinity group} & |
\parbox{85pt}{$\frac{n}{\sqrt{n}} + c*(\sqrt{n}-1) + \frac{Totalnumber of files}{\sqrt{n}}$, where n is the number of peers and c the number of contacts/foreign affinity group} & |
713 |
\parbox{85pt}{Insert/delete overhead is constant and performed in the background, the performance of system may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner} |
\parbox{85pt}{Insert/delete overhead is constant and performed in the background, system performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner} |
714 |
\\ \hline |
\\ \hline |
715 |
|
|
716 |
\parbox{37pt}{Koorde \cite{kaashoek03koorde}} & |
\parbox{37pt}{Koorde \cite{kaashoek03koorde}} & |
735 |
\parbox{37pt}{$O(\log{n})$} & |
\parbox{37pt}{$O(\log{n})$} & |
736 |
\parbox{37pt}{$O(\log{n})$} & |
\parbox{37pt}{$O(\log{n})$} & |
737 |
\parbox{85pt}{$(2^{b - 1})\frac{\log{n}}{b}$, where $b$ is a configurable parameter for tuning digit-fixing properties (routing table)} & |
\parbox{85pt}{$(2^{b - 1})\frac{\log{n}}{b}$, where $b$ is a configurable parameter for tuning digit-fixing properties (routing table)} & |
738 |
\parbox{85pt}{The performance of system performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, based on Plaxton's algorithm} |
\parbox{85pt}{System performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, based on Plaxton's algorithm} |
739 |
\\ \hline |
\\ \hline |
740 |
|
|
741 |
|
|
784 |
\parbox{37pt}{$O(\log{n})$} & |
\parbox{37pt}{$O(\log{n})$} & |
785 |
\parbox{37pt}{$O(\log{n})$} & |
\parbox{37pt}{$O(\log{n})$} & |
786 |
\parbox{85pt}{$2k+2+f$, where k = long range connections, 2 = peer's neighbors, f = fault tolerance connections)} & |
\parbox{85pt}{$2k+2+f$, where k = long range connections, 2 = peer's neighbors, f = fault tolerance connections)} & |
787 |
\parbox{85pt}{Space can be also $O(1)$. Additional space of can be used as a lookahead list for better performance, not necessarily fault-tolerant because of constant degree of neighbors} |
\parbox{85pt}{Space can also be $O(1)$. Additional space of can be used as a lookahead list for better performance} |
788 |
\\ \hline |
\\ \hline |
789 |
|
|
790 |
\parbox{37pt}{SWAN \cite{bonsma02swan}} & |
\parbox{37pt}{SWAN \cite{bonsma02swan}} & |
801 |
\parbox{37pt}{$O(\log{n})$} & |
\parbox{37pt}{$O(\log{n})$} & |
802 |
\parbox{37pt}{$O(\log{n})$} & |
\parbox{37pt}{$O(\log{n})$} & |
803 |
\parbox{85pt}{$(2^{b - 1})\frac{\log{n}}{b}$, where $b$ is a configurable parameter for tuning digit-fixing properties (routing table)} & |
\parbox{85pt}{$(2^{b - 1})\frac{\log{n}}{b}$, where $b$ is a configurable parameter for tuning digit-fixing properties (routing table)} & |
804 |
\parbox{85pt}{The system performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, based on Plaxton's algorithm} |
\parbox{85pt}{System performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, based on Plaxton's algorithm} |
805 |
\\ \hline |
\\ \hline |
806 |
|
|
807 |
\parbox{37pt}{Viceroy \cite{malkhi02viceroy}} & |
\parbox{37pt}{Viceroy \cite{malkhi02viceroy}} & |
809 |
\parbox{37pt}{$O(1)$} & |
\parbox{37pt}{$O(1)$} & |
810 |
\parbox{37pt}{$O(\log{n})$} & |
\parbox{37pt}{$O(\log{n})$} & |
811 |
\parbox{85pt}{11} & |
\parbox{85pt}{11} & |
812 |
\parbox{85pt}{The system performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, not necessarily fault-tolerant because of constant degree of neighbors} |
\parbox{85pt}{System performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, not necessarily fault-tolerant because of constant degree of neighbors} |
813 |
\\ \hline |
\\ \hline |
814 |
|
|
815 |
|
|
825 |
\chapter{Open Problems in Peer-to-Peer} |
\chapter{Open Problems in Peer-to-Peer} |
826 |
|
|
827 |
In this chapter, we discuss open problems in Peer-to-Peer research. We describe |
In this chapter, we discuss open problems in Peer-to-Peer research. We describe |
828 |
open problems and their proposed solutions. Then, we list all issues in |
the open problems and their proposed solutions. Then, we list all issues in |
829 |
tables; we list description of the problem, solution and comments on that |
tables. Note that the open problems list considered here is not meant |
|
specific open problem. Note that open problems list considered here is not meant |
|
830 |
to be an exhaustive survey of \emph{all} open problems in Peer-to-Peer domain; |
to be an exhaustive survey of \emph{all} open problems in Peer-to-Peer domain; |
831 |
we focus our attention to security, scalability, usability and performance issues |
we focus our attention to security, scalability, usability and performance issues |
832 |
only. |
only. |
833 |
|
|
834 |
\section{Overview} |
\section{Overview} |
835 |
|
|
836 |
Partly due to the non-maturity of modern Peer-to-Peer technology, it has several |
Partly due to the non-maturity of modern Peer-to-Peer technology, there are several |
837 |
open problems to be solved. The most severe problems are related to performance, scalability, usability |
open problems to be solved. The most severe problems are related to performance, scalability, usability |
838 |
and security. More important, many techniques developed for traditional distributed |
and security. Also, many techniques developed for traditional distributed |
839 |
systems may no longer apply with Peer-to-Peer systems. Therefore, new solutions are |
systems may no longer apply with Peer-to-Peer systems. Therefore, new solutions are |
840 |
needed to make Peer-to-Peer systems more secure and efficient. |
needed to make Peer-to-Peer systems more secure and efficient. |
841 |
|
|
842 |
Both loosely structured and tightly structured approach have their own specific problems. |
Both the loosely structured and the tightly structured approach have their own specific problems. |
843 |
Since Napster \cite{napsterurl} and Gnutella \cite{gnutellaurl} was first introduced |
Since Napster \cite{napsterurl} and Gnutella \cite{gnutellaurl} were first introduced |
844 |
to public, researchers' main concern has been the scalability problem of loosely structured |
to the public, researchers' main concern has been the scalability problem of the loosely structured |
845 |
approach. However, people often misunderstand the scalability problem of loosely structured |
approach. However, people often misunderstand the scalability problem of the loosely structured |
846 |
approach; \emph{network} of loosely structured systems is scalable, but the \emph{data lookup model} is not. |
approach; \emph{network} of loosely structured systems is scalable, but the \emph{data lookup model} is not. |
847 |
The main concern of tightly structured system is to make overlay's data lookup process |
The main concern of the tightly structured system is to make overlay's data lookup process |
848 |
more fault tolerant against hostile attacks. Other key problems in tightly structured |
more fault tolerant against hostile attacks. Other key problems in tightly structured |
849 |
systems are the lack of keyword searches, support for heterogeneous peers and load balancing |
systems are the lack of keyword searches, support for heterogeneous peers and load balancing |
850 |
\cite{balakrishanarticle03lookupp2p}. |
\cite{balakrishanarticle03lookupp2p}. |
851 |
|
|
852 |
To make Peer-to-Peer systems even more popular (e.g., in industry), Peer-to-Peer domain |
To make Peer-to-Peer systems even more popular (e.g., in industry), Peer-to-Peer domain |
853 |
needs better infrastructures to deal with security issues. There has been done some |
needs better infrastructures to deal with security issues. Some research has been done regarding |
854 |
research regarding anonymity, access control, data availability and data integrity but as |
anonymity, access control, data availability and data integrity but as |
855 |
we state in the following sections, much more research work is required to solve these issues. |
we state in the following sections, much more research work is required to solve these issues. |
856 |
|
|
857 |
\section{Security problems in Peer-to-Peer} |
\section{Security problems in Peer-to-Peer} |
864 |
Fail-stop attack, Spam attack \cite{naor03simpledht}, Byzantine attack \cite{357176} and \cite{296824}, and |
Fail-stop attack, Spam attack \cite{naor03simpledht}, Byzantine attack \cite{357176} and \cite{296824}, and |
865 |
general Distributed Denial of Service attack. |
general Distributed Denial of Service attack. |
866 |
|
|
867 |
In Sybil attack model, hostile entity presents multiple |
In Sybil attack model, a hostile entity presents multiple |
868 |
entities. Therefore, one hostile entity can control a large fraction of the Peer-to-Peer system. Optimal |
entities. Therefore, one hostile entity can control a large fraction of the Peer-to-Peer system. Possible solution to |
869 |
possible solution to Sybil attack would be that system could \emph{distinct} entities of the system reliably. Unfortunately, |
Sybil attack would be that the system could distinguish entities of the system reliably. Unfortunately, |
870 |
currently there are no realizable techniques for this task. Partial solutions for Sybil attack is to replicate |
currently there are no realizable techniques for this task. Partial solutions for Sybil attack is to replicate |
871 |
and fragment data randomly among several participating peer. However, both suggestions assume that two different |
and fragment data randomly among several participating peers. However, both suggestions assume that two different |
872 |
remote entities are actually different; Sybil attacks are still possible and therefore, would need centralized |
remote entities are actually different; Sybil attacks are still possible and therefore would need centralized |
873 |
authority for reliable authentication. As author argues in \cite{douceur02sybil}, without centralized authority, |
authority for reliable authentication. As the author argues in \cite{douceur02sybil}, without centralized authority, |
874 |
Sybil attacks are always possible in Peer-to-Peer system except under extreme and unrealistic assumptions of |
Sybil attacks are always possible in a Peer-to-Peer system except under extreme and unrealistic assumptions of |
875 |
resource parity and coordination among entities. |
resource parity and coordination among entities. |
876 |
|
|
877 |
In random fail-stop model, cited in \cite{naor03simpledht}, faulty peer is deleted from the Peer-to-Peer system. |
In random fail-stop model, cited in \cite{naor03simpledht}, a faulty peer is deleted from the Peer-to-Peer system. |
878 |
The reason for faultiness of peer can be a software failure, a hostile attack, or external threat such as virus or |
The reason for the faultiness of a peer can be a software failure, a hostile attack, or an external threat such as virus or |
879 |
trojan. Closely related to fail-stop model is the Byzantine attack model |
trojan. The Byzantine attack model \cite{357176} closely related to fail-stop model. Byzantine model can be seen as more |
880 |
\cite{357176}. Byzantine model can be seen as more severe than fail-stop model as there are no restrictions over |
severe than fail-stop model as there are no restrictions over the behavior of faulty peers. Practical but partial |
881 |
the behavior of faulty peers. Practical, but partial solution for Byzantine failures has been proposed by Castro et |
solution for Byzantine failures has been proposed by Castro et al. \cite{296824}. |
|
al. \cite{296824}. |
|
882 |
|
|
883 |
Spam generating attack is another known attack model against Peer-to-Peer system. In Spam |
Spam generating attack is another known attack model against Peer-to-Peer system. In Spam |
884 |
attack, hostile or faulty peer may produce false information of the data, or refuses/is not able to reply to requests. |
attack, a hostile or faulty peer may produce false information of the data, or refuses to (or is not able to) reply to requests. |
885 |
Possible solution against this attack is that peer should not trust to single entity. Instead, peer should get |
Possible solution against this attack is that peer should not trust a single entity. Instead, a peer should get |
886 |
information from multiple entities and trust on majority's opinion. This method requires more messages to be |
information from multiple entities and trust on the majority's opinion. This method requires more messages to be |
887 |
sent to network while increasing system load. However, if Spam attack is combined with Sybil attack, obviously |
sent to the network while increasing the system load. However, if the Spam attack is combined with the Sybil attack, obviously |
888 |
previously mentioned solution doesn't work. Naor et al. \cite{naor03simpledht} have proposed a partial solution against Spam attack |
the previously mentioned solution doesn't work. Naor et al. \cite{naor03simpledht} have proposed a partial solution against Spam attack |
889 |
in \emph{faulty} peer environment (not hostile). |
in \emph{faulty} peer environment (not hostile). |
890 |
|
|
891 |
Traditional overloading of targeted peers is best known form of distributed Denial of Service attack (DDoS). For example, |
Traditional overloading of targeted peers is the best known form of distributed Denial of Service attack (DDoS). For example, |
892 |
hostile entity can attempt to burden targeted peers with garbage network packets. As an implication, peers may act |
a hostile entity can attempt to burden targeted peers with garbage network packets. As an implication, peers may act |
893 |
incorrectly or stop working. DDoS attack may be very severe, especially if rate of replication and caching |
incorrectly or stop working. DDoS attack may be very severe, especially if the rate of replication and caching |
894 |
in Peer-to-Peer system is low. This may lead to data loss in the Peer-to-Peer system. Daswani et al. |
in the Peer-to-Peer system is low. This may lead to data loss in the Peer-to-Peer system. Daswani et al. |
895 |
\cite{daswani02queryflooddos} have done research regarding to this subject. Authors suggest efficient load balancing |
\cite{daswani02queryflooddos} suggest efficient load balancing |
896 |
policies for Peer-to-Peer system in order to prevent massive system failures. Sit et al. \cite{sit02securitycons} |
policies for Peer-to-Peer system in order to prevent massive system failures. Sit et al. \cite{sit02securitycons} |
897 |
suggest that identifier assignment algorithm for peers would assign identifier with respect to network topology |
suggest that identifier assignment algorithm for peers would assign identifier with respect to network topology |
898 |
and replicas should be located physically to different locations. |
and replicas should be located physically to different locations. |
899 |
|
|
900 |
As stated in \cite{naor03simpledht}, an important aspect is that when it comes to general security aspects and |
As stated in \cite{naor03simpledht}, an important aspect is that when it comes to general security aspects and |
901 |
Byzantine faults in any Peer-to-Peer system, there should be a clear distinction between attacks on the |
Byzantine faults in any Peer-to-Peer system, there should be a clear distinction between attacks on the |
902 |
algorithms assuming the construction of overlay is correct, and attacks on the construction itself. Clearly, Sybil |
algorithms assuming the construction of the overlay is correct, and attacks on the construction itself. Clearly, Sybil |
903 |
and Spam attack belongs to the first category, and rest of the attacks to the latter category. |
and Spam attacks belong to the first category, and the rest of the attacks to the latter category. |
904 |
|
|
905 |
\subsection{Trust, data authenticity and integrity} |
\subsection{Trust, data authenticity and integrity} |
906 |
|
|
907 |
Trust in Peer-to-Peer systems is based on \emph{reputation}. Proposed reputation methods focus either |
Trust in Peer-to-Peer systems is based on \emph{reputation}. Proposed reputation methods focus either |
908 |
on the semantic properties, or the data management properties of the trust model. Some research has been |
on the semantic properties or the data management properties of the trust model. Some research has been |
909 |
done on reputation models in Peer-to-Peer systems, such as \cite{aberer01trust}, \cite{cornelli02reputableservents}. |
done on reputation models in Peer-to-Peer systems, such as \cite{aberer01trust}, \cite{cornelli02reputableservents}. |
910 |
One implementation include Advogato \cite{advogatourl}. None of the current proposals or implementations |
One implementation include Advogato \cite{advogatourl}. None of the current proposals or implementations |
911 |
based on reputation address trust in a reliable, practical way. |
based on reputation address trust in a reliable, practical way. |
912 |
|
|
913 |
Optimal solution for trust in Peer-to-Peer systems would be certificate based security models. |
Optimal solution for trust in Peer-to-Peer systems would be certificate based security models. |
914 |
Quite recently, widely used Public Key Infrastructure (PKI) has been deployed in distributed |
Quite recently, widely used Public Key Infrastructure (PKI) has been deployed in distributed |
915 |
systems \cite{rivest96sdsi}, \cite{spkiworkinggroup}. PKI is reliable technology for securing |
systems \cite{rivest96sdsi}, \cite{spkiworkinggroup}. PKI is a reliable technology for securing |
916 |
data in rather \emph{static} computing systems, such as in the Internet. However, in Peer-to-Peer |
data in rather \emph{static} computing systems, such as the Internet. However, in Peer-to-Peer |
917 |
network, the problem of key based security mechanism is the maintenance of the keys as participating |
networks, the problem of key based security mechanism is the maintenance of the keys as participating |
918 |
peers constantly join and leave the system. Specifically, the distribution of key changes comes an essential |
peers constantly join and leave the system. Specifically, the distribution of key changes becomes an essential |
919 |
problem in ad hoc environments. These include revocation of keys and new key distribution in hostile |
problem in ad hoc environments. These include revocation of keys and new key distribution in hostile |
920 |
environment. |
environment. |
921 |
|
|
922 |
ConChord \cite{ajmani02conchord} is the first Peer-to-Peer system which has a support for PKI based |
ConChord \cite{ajmani02conchord} is the first Peer-to-Peer system which has a support for PKI based |
923 |
security infrastructure. Still, however, ConChord \cite{ajmani02conchord} is in early phase of development and lacks of |
security infrastructure. Still, however, ConChord \cite{ajmani02conchord} is in early phase of development and lacks |
924 |
important features of PKI to be fully usable yet. Furthermore, the hierarchy of Simple Distributed Security Infrastructure |
important features of PKI to be fully usable yet. Furthermore, the hierarchy of Simple Distributed Security Infrastructure |
925 |
(SDSI) \cite{rivest96sdsi} and Simple Public Key Infrastructure (SPKI) \cite{spkiworkinggroup} may be a problem for |
(SDSI) \cite{rivest96sdsi} and Simple Public Key Infrastructure (SPKI) \cite{spkiworkinggroup} may be a problem for |
926 |
Peer-to-Peer systems, in which hierarchy is intentionally missing. |
Peer-to-Peer systems, in which hierarchy is intentionally missing. |
929 |
\cite{fips-sha-1}, their variations \cite{merkle87hashtree} and implementation techniques \cite{mohr02thex}, |
\cite{fips-sha-1}, their variations \cite{merkle87hashtree} and implementation techniques \cite{mohr02thex}, |
930 |
are efficient and reliable methods for identifying the integrity of data in Peer-to-Peer systems. One |
are efficient and reliable methods for identifying the integrity of data in Peer-to-Peer systems. One |
931 |
possible application of cryptographic content hashes may be in peer identifier creation process, in which |
possible application of cryptographic content hashes may be in peer identifier creation process, in which |
932 |
IP address of peer can be verified by the other peer. This is one form of \emph{self-certifying data}. |
the IP address of a peer can be verified by the other peer. This is one form of \emph{self-certifying data}. |
933 |
|
|
934 |
|
|
935 |
\subsection{Anonymity} |
\subsection{Anonymity} |
936 |
|
|
937 |
According to \cite{dingledine00free}, there exists several kinds of anonymity. Author-anonymity is a form |
According to \cite{dingledine00free}, there exist several kinds of anonymity. Author-anonymity is a form |
938 |
of anonymity in which no one can link author to a specific document. In publisher-anonymity system, |
of anonymity in which no one can link the author to a specific document. In publisher-anonymity system, |
939 |
no one is able to link publisher to a specific document. Reader-anonymity means that a specific |
no one is able to link the publisher to a specific document. Reader-anonymity means that a specific |
940 |
document cannot be linked to document's readers. This form of anonymity protects the privacy of |
document cannot be linked to the readers of a document. This form of anonymity protects the privacy of |
941 |
users of the system. Furthermore, peer-anonymity means that no peer can be linked to a specific document, i.e., |
the users of the system. Furthermore, peer-anonymity means that no peer can be linked to a specific document, i.e., |
942 |
no one is able to determine the peer, where the document was originally published. Document-anonymity |
no one is able to determine the peer, where the document was originally published. Document-anonymity |
943 |
means that peer doesn't know which data it is currently hosting. Finally, query-anonymity is a form |
means that a peer doesn't know which data it is currently hosting. Finally, query-anonymity is a form |
944 |
of document-anonymity; when other peers performs data lookups, peer doesn't know which data it serves |
of document-anonymity; when other peers perform data lookups, a peer doesn't know which data it serves |
945 |
to the data lookup originators. As the authors cite, some forms of anonymity may imply each other and |
to the data lookup originators. As the authors cite in \cite{dingledine00free}, some forms of anonymity |
946 |
possible issues raised by this property is one area of future work. |
may imply each other and possible issues raised by this property is one area of future work. |
947 |
|
|
948 |
With regard to anonymity in Peer-to-Peer systems, there has been done much research work both at network |
With regard to anonymity in Peer-to-Peer systems, much research has been done both at the network |
949 |
level layer \cite{tarzan:ccs9} and at application level layer \cite{reiter98crowds}, \cite{mixminionurl}. |
level layer \cite{tarzan:ccs9} and at the application level layer \cite{reiter98crowds}, \cite{mixminionurl}. |
950 |
Research on anonymity outside of Peer-to-Peer context have been done also \cite{352607}, \cite{293447}. |
Anonymity outside of Peer-to-Peer context has also been researched \cite{352607}, \cite{293447}. |
951 |
|
|
952 |
Obviously, providing several types of anonymity, it often conflicts with other key properties of |
Obviously, existance of several types of anonymity often conflicts with other key properties of |
953 |
Peer-to-Peer system. Let's consider anonymity and efficient data lookup. In efficient data lookup, we must know |
Peer-to-Peer systems. Let us consider anonymity and efficient data lookup. In efficient data lookup, we must know |
954 |
the peers responsible for given data in Peer-to-Peer system. Of course, when we know the peers responsible |
the peers responsible for given data. Of course, when we know the peers responsible |
955 |
for the data, the anonymity of peer is lost. Fortunately, there are partial solutions to previously |
for the data, the anonymity of peer is lost. Fortunately, there are partial solutions to previously |
956 |
mentioned situations, such as pseudonym which is a partial form of anonymity. For instance, pseudonym can be used for |
mentioned situations, such as pseudonymity which is a partial form of anonymity. For instance, pseudonymity can be used for |
957 |
addressing peer-anonymity by providing anonymous-like identifiers to peers (e.g., peer identifiers of tightly |
addressing peer-anonymity by providing anonymous-like identifiers to peers (e.g., peer identifiers of a tightly |
958 |
structured system). |
structured system). |
959 |
|
|
960 |
Anonymity is widely used in those Peer-to-Peer system in which data publication and non-censorship are important properties |
Anonymity is widely used in Peer-to-Peer system in which data publication and non-censorship are important properties |
961 |
of the system. These include |
of the system. These include |
962 |
Freenet \cite{clarke00freenet}, Publius \cite{pub00}, Free haven \cite{dingledine00free}, Crowds \cite{reiter98crowds}, |
Freenet \cite{clarke00freenet}, Publius \cite{pub00}, Free Haven \cite{dingledine00free}, Crowds \cite{reiter98crowds}, |
963 |
Tangler \cite{502002} and upcoming Mnet \cite{mneturl}. Forwarding proxies are used in Freenet, Crowds and |
Tangler \cite{502002} and upcoming Mnet \cite{mneturl}. Forwarding proxies are used in Freenet, Crowds and |
964 |
Free Haven in order to provide various types of anonymity. Tangler and Publius uses cryptographic |
Free Haven in order to provide various types of anonymity. Tangler and Publius use cryptographic |
965 |
sharing methods to split data into fragments \cite{Shamir1979a}. Mix mailer networks, such as |
sharing methods to split data into fragments \cite{Shamir1979a}. Mix mailer networks, such as |
966 |
\cite{mixminionurl}, are commonly used in distributed systems, which are able to provide some level |
\cite{mixminionurl}, are commonly used in distributed systems, which are able to provide some level |
967 |
of anonymity. |
of anonymity. |
968 |
|
|
969 |
Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no |
Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no |
970 |
such system which is able to provide all kinds of anonymity as listed above. Specifically, the conflicts |
such system which is able to provide all kinds of anonymity as listed above. Specifically, the conflicts |
971 |
between anonymity and other Peer-to-Peer system properties requires more research work. |
between anonymity and other Peer-to-Peer system properties require more research work. |
972 |
|
|
973 |
|
|
974 |
\subsection{Access control} |
\subsection{Access control} |
975 |
|
|
976 |
Any distributed computing system must support different levels of access control. For instance, in Peer-to-Peer |
Any distributed computing system must support different levels of access control. For instance, in a Peer-to-Peer |
977 |
system, we may want to restrict the accessibility of data to only limited amount of participating peers. Yet, Peer-to-Peer |
system, we may want to restrict the accessibility of data to only limited amount of participating peers. Yet, Peer-to-Peer |
978 |
systems do not have working and distributed access control scheme. Moreover, |
systems do not have a working and distributed access control scheme. Moreover, |
979 |
there has been a lot of violation of copyright laws by users of Peer-to-Peer file sharing systems. As a |
there has been a lot of violations of copyright laws by users of Peer-to-Peer file sharing systems. As a |
980 |
consequence, some law suits have been created against the companies who have build popular file-sharing programs. |
consequence, some law suits have been filed against the companies who have build popular file-sharing programs. |
981 |
|
|
982 |
To our knowledge, Nejdl et al. \cite{nejdl03accesscontrol} have proposed very recently first practical solution to access |
To our knowledge, Nejdl et al. \cite{nejdl03accesscontrol} have very recently proposed the first practical solution to access |
983 |
control problem in Peer-to-Peer systems. They use Resource Description Framework (RDF) \cite{w3rdfurl} based |
control problem in Peer-to-Peer systems. They use Resource Description Framework (RDF) \cite{w3rdfurl} based |
984 |
schema policies to restrict access to certain data. Unfortunately, their current early prototype version works only in |
schema policies to restrict access to certain data. Unfortunately, their current early prototype version only works in |
985 |
loosely structured systems. |
loosely structured systems. |
986 |
|
|
987 |
|
|
988 |
\subsection{Hostile entities} |
\subsection{Hostile entities} |
989 |
|
|
990 |
One serious problem in Peer-to-Peer system is lack of ability to identify hostile entities trustworthy. |
One serious problem in Peer-to-Peer systems is the inability to identify hostile entities as trustworthy. |
991 |
Possible solutions include self-monitoring systems \cite{zhang03somo}, maintaining system invariants as |
Possible solutions include self-monitoring systems \cite{zhang03somo}, maintaining system invariants as |
992 |
proposed in \cite{sit02securitycons}, distributed and secure peer identifier assignment |
proposed in \cite{sit02securitycons}, distributed and secure peer identifier assignment |
993 |
\cite{castro02securerouting}, \cite{clarke00freenet} and self-certifying data using cryptographic |
\cite{castro02securerouting}, \cite{clarke00freenet} and self-certifying data using cryptographic |
994 |
content hashes (e.g., SHA-1 \cite{fips-sha-1}). Identification of hostile entities is essential in tightly structured |
content hashes (e.g., SHA-1 \cite{fips-sha-1}). Identification of hostile entities is essential in the tightly structured |
995 |
approach, in which fundamental (and implicit) assumption is that there is a random, uniform distribution |
approach, in which the fundamental (and implicit) assumption is that there is a random, uniform distribution |
996 |
of peer identifiers that cannot be controlled by hostile entity. |
of peer identifiers that cannot be controlled by a hostile entity. |
997 |
|
|
998 |
Of course centralized authorities could be used for assignment of peer identifiers, but they may not be suitable |
Naturally, centralized authorities could be used for the assignment of peer identifiers, but they may not be suitable |
999 |
for ad hoc Peer-to-Peer environrment and have property of single point of failure. Moreover, distributed peer |
for ad hoc Peer-to-Peer environrment and have property of single point of failure. Moreover, distributed peer |
1000 |
identification assignment can be problematic as long as Sybil attack \cite{douceur02sybil} remains unsolved. |
identification assignment can be problematic as long as Sybil attack \cite{douceur02sybil} remains unsolved. |
1001 |
However, there are some partial solutions for controlling the rate at which hostile entity is able to obtain peer |
However, there are some partial solutions for controlling the rate at which hostile entity is able to obtain peer |
1002 |
identifier, such as crypto-based puzzles \cite{juels99clientpuzzles}. |
identifier, such as crypto-based puzzles \cite{juels99clientpuzzles}. |
1003 |
|
|
1004 |
In the end, none of the previously mentioned solutions are able to identify hostile entities safely. |
In the end, none of these problems solutions are able to identify hostile entities safely. |
1005 |
|
|
1006 |
|
|
1007 |
\subsection{Secure query routing} |
\subsection{Secure query routing} |
1009 |
Much work has been done on secure routing, especially related to tightly structured systems. In |
Much work has been done on secure routing, especially related to tightly structured systems. In |
1010 |
\cite{castro02securitystructured} and \cite{castro02securerouting}, authors suggest the usage |
\cite{castro02securitystructured} and \cite{castro02securerouting}, authors suggest the usage |
1011 |
of constrained routing tables and diverse routes, and detection of faults during query routing. |
of constrained routing tables and diverse routes, and detection of faults during query routing. |
1012 |
Additionally, authors present in \cite{castro02securerouting} an important aspect of tightly structured approach with regard |
Additionally, authors present in \cite{castro02securerouting} an important aspect of the tightly structured approach with regard |
1013 |
to fault-tolerant query routing: the probability of routing successfully between to arbitrary |
to fault-tolerant query routing: the probability of routing successfully between to arbitrary |
1014 |
correct peers, when a fraction $f$ of the other peers are faulty or hostile, is only $(1-f)^{h-1}$, where |
correct peers, when a fraction $f$ of the other peers are faulty or hostile, is only $(1-f)^{h-1}$, where |
1015 |
$h$ is the number of hops in the overlay. |
$h$ is the number of hops in the overlay. |
1030 |
Peer-to-Peer system. They show that to provide high degree of fault tolerance and efficiency, each |
Peer-to-Peer system. They show that to provide high degree of fault tolerance and efficiency, each |
1031 |
participating peer must maintain average of $O(\log{n})$ neighbors. |
participating peer must maintain average of $O(\log{n})$ neighbors. |
1032 |
|
|
1033 |
Fiat et al. in \cite{fiat02censorship}, \cite{saia02dynamicfaultcontentnetwork} and Datar in \cite{datar02butterflies} |
Fiat et al. in \cite{fiat02censorship, saia02dynamicfaultcontentnetwork} and Datar in \cite{datar02butterflies} |
1034 |
describe tightly structured overlay with analytical results in the presence of hostile entities. However, |
describe tightly structured overlay with analytical results in the presence of hostile entities. However, |
1035 |
none of these proposals address an efficient, dynamic tightly structured overlay and multiple rounds |
none of these proposals address an efficient, dynamic tightly structured overlay and multiple rounds |
1036 |
of hostile attack. Also, above mentioned proposals are not very efficient. In \cite{fiat02censorship}, each peer |
of hostile attack. Also, above mentioned proposals are not very efficient. In \cite{fiat02censorship}, each peer |
1037 |
must maintain information of $O(\log^3{n})$ other peers, and in \cite{datar02butterflies}, $O(\log^2{n})$ is required. |
must maintain information of $O(\log^3{n})$ other peers, and in \cite{datar02butterflies}, $O(\log^2{n})$ is required. |
1038 |
|
|
1039 |
Finally, Ratnasamy and Gavoille \cite{ratnasamy02routing}, \cite{gavoille01routing} list several open problems |
Finally, Ratnasamy and Gavoille \cite{ratnasamy02routing, gavoille01routing} list several open problems |
1040 |
regarding routing in distributed networks. Obviously, more research is required in order to provide secure |
regarding routing in distributed networks. Obviously, more research is required in order to provide secure |
1041 |
data lookup routing possible in Peer-to-Peer networks. |
data lookup routing possible in Peer-to-Peer networks. |
1042 |
|
|
1047 |
the list includes viruses and trojans. Currently, there are not even partial solutions |
the list includes viruses and trojans. Currently, there are not even partial solutions |
1048 |
to the problems mentioned above. General robustness properties of Peer-to-Peer system is able to |
to the problems mentioned above. General robustness properties of Peer-to-Peer system is able to |
1049 |
deal with software failures and hostile attack, but fault tolerance against external threats is unknown. |
deal with software failures and hostile attack, but fault tolerance against external threats is unknown. |
1050 |
The reason for this is that there are no experiences on these kinds of attacks. Possible solution |
The reason for this is that there are no experience on these kinds of attacks. Possible solution |
1051 |
would be distributed anti-virus software, but much more intensive research is required until |
would be distributed anti-virus software, but much more intensive research is required until |
1052 |
this kind of solution would be applicable. |
this kind of solution would be applicable. |
1053 |
|
|
1059 |
\subsection{Efficient data lookup} |
\subsection{Efficient data lookup} |
1060 |
|
|
1061 |
The most intensive research in Peer-to-Peer domain has been focused on efficient data lookup methods, |
The most intensive research in Peer-to-Peer domain has been focused on efficient data lookup methods, |
1062 |
especially with loosely structured approach. In addition to ''super-peer'' method presented in chapter |
especially with the loosely structured approach. In addition to ''super-peer'' method presented in chapter |
1063 |
2, there has been other improvements also. |
2, there has been other improvements also. |
1064 |
In iterative deepening |
In iterative deepening |
1065 |
\cite{yang02improvingsearch}, multiple BFS searches are initiated |
\cite{yang02improvingsearch}, multiple BFS searches are initiated |
1101 |
|
|
1102 |
Since tightly structured systems have efficient data lookup at the application level overlay, |
Since tightly structured systems have efficient data lookup at the application level overlay, |
1103 |
current research efforts are focused on proximity based data lookup. In proximity based data lookup, |
current research efforts are focused on proximity based data lookup. In proximity based data lookup, |
1104 |
peers try to choose routing-tables entries referring to other peers that are \emph{nearby} in the |
peers try to choose entries of routing-tables referring to other peers that are \emph{nearby} in the |
1105 |
underlying network. In this way, tightly structured systems are able to decrease actual |
underlying network. In this way, tightly structured systems are able to decrease actual |
1106 |
lookup \emph{latency}. CAN \cite{ratnasamy01can}, Kademlia \cite{maymounkov02kademlia}, |
lookup \emph{latency}. CAN \cite{ratnasamy01can}, Kademlia \cite{maymounkov02kademlia}, |
1107 |
Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry} have advanced heuristics for |
Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry} have advanced heuristics for |
1108 |
proximity based routing. Additionally, most recent version of Chord uses proximity based |
proximity based routing. Additionally, most recent version of Chord uses proximity based |
1109 |
routing inspired by Karger and Ruhl \cite{karger02findingnearest}. SkipNet \cite{harvey03skipnet1} |
routing inspired by Karger and Ruhl \cite{karger02findingnearest}. SkipNet \cite{harvey03skipnet1} |
1110 |
uses combination of proximity and application level overlay routing when performing data |
uses a combination of proximity and application level overlay routing when performing data |
1111 |
lookups. Authors call this feature \emph{constrained load balancing}. |
lookups. Authors call this feature \emph{constrained load balancing}. |
1112 |
|
|
1113 |
Additional research related to proximity based routing include \cite{karger02findingnearest}, |
Additional research related to proximity based routing include \cite{karger02findingnearest, hildrum02distributedobject, |
1114 |
\cite{hildrum02distributedobject}, \cite{brinkmann02compactplacement}, \cite{rhea02probabilistic}, |
brinkmann02compactplacement, rhea02probabilistic, castro02networkproximity, ng02predicting, pias03lighthouse}. |
|
\cite{castro02networkproximity}, \cite{ng02predicting} and \cite{pias03lighthouse}. |
|
1115 |
|
|
1116 |
\subsection{Fast and usable search} |
\subsection{Fast and usable search} |
1117 |
|
|
1118 |
To make Peer-to-Peer systems even more popular (and usable), these systems have to support flexible, efficient |
To make Peer-to-Peer systems even more popular (and usable), these systems have to support flexible, efficient |
1119 |
and easy to use search methods. For instance, Internet's perhaps the most important feature |
and easy search methods. For instance, Internet's perhaps the most important feature |
1120 |
is the ability to perform keyword searches (e.g., Google \cite{googleurl}). Currently, only loosely |
is the ability to perform keyword searches (e.g., Google \cite{googleurl}). Currently, only loosely |
1121 |
structured systems are able to carry out this requirement. Unfortunately, as discussed in this text, |
structured systems are able to carry out this requirement. Unfortunately, as discussed in this text, |
1122 |
the data lookup model of loosely structured approach doesn't scale. Thus, research efforts have |
the data lookup model of the loosely structured approach doesn't scale. Thus, research efforts have |
1123 |
been focused on tightly structured systems. The main problem with tightly structured systems is the |
been focused on tightly structured systems. The main problem with tightly structured systems is the |
1124 |
fact that tightly structured algorithms perform data lookups based on a globally unique identifier (key). |
fact that tightly structured algorithms perform data lookups based on a globally unique identifier (key). |
1125 |
|
|
1127 |
\cite{li03feasibility} on top of tightly structured overlays. Authors argue, that it is possible to implement |
\cite{li03feasibility} on top of tightly structured overlays. Authors argue, that it is possible to implement |
1128 |
Peer-to-Peer Web-like search with certain compromises. First, Peer-to-Peer search engine may need to |
Peer-to-Peer Web-like search with certain compromises. First, Peer-to-Peer search engine may need to |
1129 |
decrease result quality in order to make searching more efficient. Second, Peer-to-Peer systems must |
decrease result quality in order to make searching more efficient. Second, Peer-to-Peer systems must |
1130 |
observe better the properties of underlying network for better performance. |
observe the properties of underlying network for better performance. |
1131 |
|
|
1132 |
Some studies have been concentrated on SQL-like queries \cite{harren02complex} |
Some studies have been concentrated on SQL-like queries \cite{harren02complex} |
1133 |
in tightly structured overlays. Other approaches include adaption of data lookup model of loosely |
in tightly structured overlays. Other approaches include adaption of data lookup model of the loosely |
1134 |
structured approach into tightly structured systems \cite{ansaryefficientbroadcast03}, \cite{chord:om_p-meng}. |
structured approach into tightly structured systems \cite{ansaryefficientbroadcast03, chord:om_p-meng}. |
1135 |
Some studies include additional layer upon overlay network \cite{kronfol02fasdsearch}, |
Some studies include additional layer upon overlay network \cite{kronfol02fasdsearch, joseph02p2players} |
1136 |
\cite{joseph02p2players} and range queries \cite{andrzejak02rangequeries}. |
and range queries \cite{andrzejak02rangequeries}. |
1137 |
|
|
1138 |
Many techniques have been developed in order to provide more efficient search indexing. As |
Many techniques have been developed in order to provide more efficient search indexing. As |
1139 |
several studies show, the popularity of queries in the Internet follow Zipf-like |
several studies show, the popularity of queries in the Internet follow Zipf-like |
1156 |
|
|
1157 |
Adaptive system management and self-organization are essential properties |
Adaptive system management and self-organization are essential properties |
1158 |
of any Peer-to-Peer system, since centralized control over the system is missing. Loosely structured |
of any Peer-to-Peer system, since centralized control over the system is missing. Loosely structured |
1159 |
systems require less system management properties than tightly structured systems; in loosely |
systems require less system management properties than tightly structured systems; in a loosely |
1160 |
structured system, peers join and leave the system constantly without any restrictions. On the |
structured system, peers join and leave the system constantly without any restrictions. On the |
1161 |
other hand, however, peers in tightly structured system join and leave the system but have less freedom, |
other hand, however, peers in tightly structured system join and leave the system but have less freedom, |
1162 |
i.e. overlay chooses peer's neighbors on behalf of peer itself and maps data items randomly |
i.e. overlay chooses peer's neighbors on behalf of peer itself and maps data items randomly |
1163 |
throughout the overlay network. |
throughout the overlay network. |
1164 |
|
|
1165 |
Current research has been focused on system management of tightly structured systems, and all presented |
Current research has been focused on system management of tightly structured systems, and all presented |
1166 |
algorithms of tightly structured approach have been analyzed under static simulation environments. Furthermore, proposed |
algorithms of the tightly structured approach have been analyzed under static simulation environments. Furthermore, proposed |
1167 |
tightly structured overlays are configured statically to achieve the desired reliability even in uncommon and adverse environment |
tightly structured overlays are configured statically to achieve the desired reliability even in uncommon and adverse environment |
1168 |
\cite{rowston03controlloingreliability}. The most important factor for |
\cite{rowston03controlloingreliability}. The most important factor for |
1169 |
future research is to get real-life experiences from tightly structured systems, when there are frequent |
future research is to get real-life experiences from tightly structured systems, when there are frequent |
1190 |
\cite{sloppy:iptps03}. Another key feature of their work is that peers self-organize into clusters, |
\cite{sloppy:iptps03}. Another key feature of their work is that peers self-organize into clusters, |
1191 |
therefore enabling peers to find nearby data without looking up data from distant peers. |
therefore enabling peers to find nearby data without looking up data from distant peers. |
1192 |
|
|
1193 |
As mentioned before, an implicit assumption of almost every tightly structured system is that there is random, uniform |
As mentioned before, an implicit assumption of almost every tightly structured system is that there is a random, uniform |
1194 |
distribution of peer and key identifiers. Even if participating peers are extremely heterogeneous, e.g., in |
distribution of peer and key identifiers. Even if participating peers are extremely heterogeneous, e.g., in |
1195 |
face of computing power or network bandwidth, all data items are distributed uniformly. Clearly, this is |
face of computing power or network bandwidth, all data items are distributed uniformly. Clearly, this is |
1196 |
a serious problem of tightly structured overlays in face of performance and load balancing. Measurement study |
a serious problem of tightly structured overlays in face of performance and load balancing. Measurement study |
1201 |
|
|
1202 |
Research has been done on self-organization. Ledlie et al. propose techniques for forming and maintaining |
Research has been done on self-organization. Ledlie et al. propose techniques for forming and maintaining |
1203 |
groups in highly dynamic environment \cite{ledlie02selfp2p}. Unfortunately their work relies on idea that |
groups in highly dynamic environment \cite{ledlie02selfp2p}. Unfortunately their work relies on idea that |
1204 |
participating peers would create multiple hierarchical groups. It's not clear whether this approach |
participating peers would create multiple hierarchical groups. It is not clear whether this approach |
1205 |
is fault-tolerant and suitable to Peer-to-Peer environment. More promising work has been done by Rowston et al. |
is fault-tolerant and suitable to Peer-to-Peer environment. More promising work has been done by Rowston et al. |
1206 |
in \cite{rowston03controlloingreliability}. Authors propose techniques for self-tuning, dealing with |
in \cite{rowston03controlloingreliability}. Authors propose techniques for self-tuning, dealing with |
1207 |
uncommon conditions (e.g., network partition and high failure rates). Moreover, authors argue that |
uncommon conditions (e.g., network partition and high failure rates). Moreover, authors argue that |
1227 |
of the most important area of future research is to create common programming abstractions, i.e., |
of the most important area of future research is to create common programming abstractions, i.e., |
1228 |
interfaces, design patters and frameworks. Also, equal benchmarks are needed for comparing |
interfaces, design patters and frameworks. Also, equal benchmarks are needed for comparing |
1229 |
different algorithms. Recently, there have been few proposals towards common programming |
different algorithms. Recently, there have been few proposals towards common programming |
1230 |
guidelines. This list includes \cite{zhao03api}, \cite{frise02p2pframework}, \cite{babaoglu02anthill}. |
guidelines. This list includes \cite{zhao03api, frise02p2pframework, babaoglu02anthill}. |
1231 |
Early experiments with Peer-to-Peer benchmarking include \cite{ratnasamy02routing} and \cite{rhea03benchmarks}. |
Early experiments with Peer-to-Peer benchmarking include \cite{ratnasamy02routing, rhea03benchmarks}. |
1232 |
|
|
1233 |
\subsection{Social behavior} |
\subsection{Social behavior} |
1234 |
|
|
1235 |
Frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate. Another belief |
Frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate. Another belief |
1236 |
is that all peers would behave equally, i.e., all peers both consume and contribute resources. |
is that all peers would behave equally, i.e., all peers both consume and contribute resources. |
1237 |
However, these assumptions are not true as several studies show. Peers rather consume than contribute and |
However, these assumptions are not true as several studies show. Peers rather consume than contribute and |
1238 |
peers are unwilling to cooperate \cite{saroiu02measurementstudyp2p}, \cite{oram01harnessingpower}, |
peers are unwilling to cooperate \cite{saroiu02measurementstudyp2p, oram01harnessingpower, hearn02mojonation}. |
|
\cite{hearn02mojonation}. |
|
1239 |
|
|
1240 |
Somewhat surprisingly little research has been done in this area, especially when considering |
Somewhat surprisingly little research has been done in this area, especially when considering |
1241 |
the possible impact of \emph{unwanted social behavior} to performance of Peer-to-Peer |
the possible impact of \emph{unwanted social behavior} to performance of Peer-to-Peer |
1254 |
Very little research has been done on simulating a Peer-to-Peer system. Presumably, this |
Very little research has been done on simulating a Peer-to-Peer system. Presumably, this |
1255 |
is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very |
is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very |
1256 |
difficult. Floyd et al. has been studying the simulation of the Internet in \cite{504642}. Authors |
difficult. Floyd et al. has been studying the simulation of the Internet in \cite{504642}. Authors |
1257 |
state that simulating the Internet is very challenging task, because of Internet's heterogeneity |
state that simulating the Internet is very challenging task, because of its heterogeneity |
1258 |
and rapid change. Obviously, these factors exist also in Peer-to-Peer systems even with higher |
and rapid change. Obviously, these factors exist also in Peer-to-Peer systems even with higher |
1259 |
rates. |
rates. |
1260 |
|
|
1300 |
|
|
1301 |
|
|
1302 |
|
|
1303 |
\parbox{90pt}{Query routing \cite{sit02securitycons}, \cite{aspnes02faultrouting}, \cite{castro02securerouting}, \cite{ratnasamy02routing}, \cite{gavoille01routing}, |
\parbox{90pt}{Query routing \cite{sit02securitycons, aspnes02faultrouting, castro02securerouting, ratnasamy02routing, gavoille01routing, |
1304 |
\cite{lynch02atomicdataaccess}, \cite{fiat02censorship}, \cite{saia02dynamicfaultcontentnetwork}, \cite{datar02butterflies}} & |
lynch02atomicdataaccess, fiat02censorship, saia02dynamicfaultcontentnetwork, datar02butterflies}} & |
1305 |
\parbox{110pt}{Incorrect forwarding (hostile), incorrect routing (hostile)} & |
\parbox{110pt}{Incorrect forwarding (hostile), incorrect routing (hostile)} & |
1306 |
\parbox{110pt}{Query monitoring, cross check routing tables, verify routing tables, create routing table invariants} & |
\parbox{110pt}{Query monitoring, cross check routing tables, verify routing tables, create routing table invariants} & |
1307 |
\parbox{110pt}{Increases system complexity} |
\parbox{110pt}{Increases system complexity} |
1308 |
\\ \hline |
\\ \hline |
1309 |
|
|
1310 |
|
|
1311 |
\parbox{90pt}{DoS attack \cite{sit02securitycons}, \cite{saia02dynamicfaultcontentnetwork}, \cite{datar02butterflies}, \cite{daswani02queryflooddos}, \cite{juels99clientpuzzles}} & |
\parbox{90pt}{DoS attack \cite{sit02securitycons, saia02dynamicfaultcontentnetwork, datar02butterflies, daswani02queryflooddos, juels99clientpuzzles}} & |
1312 |
\parbox{110pt}{Distributed, controlled burden against specific computer(s)} & |
\parbox{110pt}{Distributed, controlled burden against specific computer(s)} & |
1313 |
\parbox{110pt}{Client puzzles, load balancing, traffic measurements, traffic models, replication} & |
\parbox{110pt}{Client puzzles, load balancing, traffic measurements, traffic models, replication} & |
1314 |
\parbox{110pt}{Only partial solutions, traffic models most effective} |
\parbox{110pt}{Only partial solutions, traffic models most effective} |
1315 |
\\ \hline |
\\ \hline |
1316 |
|
|
1317 |
|
|
1318 |
\parbox{90pt}{Sybil attack \cite{douceur02sybil}, \cite{castro02securerouting}} & |
\parbox{90pt}{Sybil attack \cite{douceur02sybil, castro02securerouting}} & |
1319 |
\parbox{110pt}{Single hostile entity presents multiple entities} & |
\parbox{110pt}{Single hostile entity presents multiple entities} & |
1320 |
\parbox{110pt}{Identify all peers simultaneously across the system, collect pool of peers which are validated, distributed peer ID creation} & |
\parbox{110pt}{Identify all peers simultaneously across the system, collect pool of peers which are validated, distributed peer ID creation} & |
1321 |
\parbox{110pt}{Not practically realizable, research focused on persistence, not on identity distinction} |
\parbox{110pt}{Not practically realizable, research focused on persistence, not on identity distinction} |
1343 |
\\ \hline |
\\ \hline |
1344 |
|
|
1345 |
|
|
1346 |
\parbox{90pt}{Anonymity \cite{dingledine00free}, \cite{tarzan:ccs9}, \cite{pub00}, \cite{clarke00freenet}, \cite{reiter98crowds}, \cite{352607},\cite{502002}} & |
\parbox{90pt}{Anonymity \cite{dingledine00free, tarzan:ccs9, pub00, clarke00freenet, reiter98crowds, 352607, 502002}} & |
1347 |
\parbox{110pt}{Anonymity cannot be provided in all cases} & |
\parbox{110pt}{Anonymity cannot be provided in all cases} & |
1348 |
\parbox{110pt}{Remailers, pre-routing} & |
\parbox{110pt}{Remailers, pre-routing} & |
1349 |
\parbox{110pt}{Total anonymity cannot be provided yet} |
\parbox{110pt}{Total anonymity cannot be provided yet} |
1350 |
\\ \hline |
\\ \hline |
1351 |
|
|
1352 |
|
|
1353 |
\parbox{90pt}{Malicious peers \cite{sit02securitycons}, \cite{castro02securerouting}} & |
\parbox{90pt}{Malicious peers \cite{sit02securitycons, castro02securerouting}} & |
1354 |
\parbox{110pt}{How to identify malicious peers in the system ?} & |
\parbox{110pt}{How to identify malicious peers in the system ?} & |
1355 |
\parbox{110pt}{Create invariants for peer behavior, verify invariants, self-certifying data} & |
\parbox{110pt}{Create invariants for peer behavior, verify invariants, self-certifying data} & |
1356 |
\parbox{110pt}{Partial solutions, self-certifying data most reliable} |
\parbox{110pt}{Partial solutions, self-certifying data most reliable} |
1357 |
\\ \hline |
\\ \hline |
1358 |
|
|
1359 |
|
|
1360 |
\parbox{90pt}{Access Control \cite{nejdl03accesscontrol}, \cite{daswani03openproblems}} & |
\parbox{90pt}{Access Control \cite{nejdl03accesscontrol, daswani03openproblems}} & |
1361 |
\parbox{110pt}{Can we define access control levels in Peer-to-Peer network ?} & |
\parbox{110pt}{Can we define access control levels in Peer-to-Peer network ?} & |
1362 |
\parbox{110pt}{Schema-based rules} & |
\parbox{110pt}{Schema-based rules} & |
1363 |
\parbox{110pt}{Some initial experiences, need more research} |
\parbox{110pt}{Some initial experiences, need more research} |
1415 |
|
|
1416 |
\endfoot |
\endfoot |
1417 |
|
|
1418 |
\parbox{90pt}{Web indexing and searching \cite{li03feasibility}, \cite{Bhattacharjee03resultcache}, \cite{362692}, \cite{CuencaAcuna2002DSIWorkshop}, |
\parbox{90pt}{Web indexing and searching \cite{li03feasibility, Bhattacharjee03resultcache, 362692, CuencaAcuna2002DSIWorkshop, |
1419 |
\cite{rhea02probabilistic}, \cite{joseph02neurogrid}, \cite{crespo02semanticoverlay}, \cite{joseph02p2players}, \cite{chord:om_p-meng}, |
rhea02probabilistic, joseph02neurogrid, crespo02semanticoverlay, joseph02p2players, chord:om_p-meng, wittengigabytes, 338634}} & |
|
\cite{wittengigabytes}, \cite{338634}} & |
|
1420 |
\parbox{110pt}{Perform Web like searches in Peer-to-Peer network} & |
\parbox{110pt}{Perform Web like searches in Peer-to-Peer network} & |
1421 |
\parbox{110pt}{Data compression, view trees, bloom filters and its variations, gap compression, index intersection optimizations, clustering} & |
\parbox{110pt}{Data compression, view trees, bloom filters and its variations, gap compression, index intersection optimizations, clustering} & |
1422 |
\parbox{110pt}{Effective but complex solutions, some compromises have to be done (decrease result quality, modify overlay's structure), more research needed} |
\parbox{110pt}{Effective but complex solutions, some compromises have to be done (decrease result quality, modify overlay's structure), more research needed} |
1423 |
\\ \hline |
\\ \hline |
1424 |
|
|
1425 |
|
|
1426 |
\parbox{90pt}{Efficient and scalable data discovery \cite{lv02searchreplication}, \cite{osokine02distnetworks}, \cite{yang02improvingsearch}, \cite{lv02gnutellascalable}, |
\parbox{90pt}{Efficient and scalable data discovery \cite{lv02searchreplication, osokine02distnetworks, yang02improvingsearch, lv02gnutellascalable, |
1427 |
\cite{ganesan02yappers}, \cite{adamic02localsearch}, \cite{adamic01powerlawsearch}, \cite{ripeanu02mappinggnutella}, \cite{milgram67smallworld}, \cite{adamic99small}, |
ganesan02yappers, adamic02localsearch, adamic01powerlawsearch, ripeanu02mappinggnutella, milgram67smallworld, adamic99small, |
1428 |
\cite{ramanathan02goodpeers}, \cite{kleinberg99small}, \cite{nips02-Kleinberg}, \cite{zhang02using}, \cite{watts00dynamics}, \cite{karger02findingnearest}, |
ramanathan02goodpeers, kleinberg99small, nips02-Kleinberg, zhang02using, watts00dynamics, karger02findingnearest, |
1429 |
\cite{brinkmann02compactplacement}, \cite{rhea02probabilistic}, \cite{castro02networkproximity}, \cite{ng02predicting}, \cite{pias03lighthouse}} & |
brinkmann02compactplacement, rhea02probabilistic, castro02networkproximity, ng02predicting, pias03lighthouse}} & |
1430 |
\parbox{110pt}{Find resources efficiently, if resource exists (loosely structured)} & |
\parbox{110pt}{Find resources efficiently, if resource exists (loosely structured)} & |
1431 |
\parbox{110pt}{Super peers, peer clusters, caching techniques} & |
\parbox{110pt}{Super peers, peer clusters, caching techniques} & |
1432 |
\parbox{110pt}{More efficient, less network traffic, not comparable to the efficiency of tightly structured systems} |
\parbox{110pt}{More efficient, less network traffic, not comparable to the efficiency of tightly structured systems} |
1433 |
\\ \hline |
\\ \hline |
1434 |
|
|
1435 |
|
|
1436 |
\parbox{90pt}{Richness of queries \cite{harren02complex}, \cite{ansaryefficientbroadcast03}, \cite{andrzejak02rangequeries}} & |
\parbox{90pt}{Richness of queries \cite{harren02complex, ansaryefficientbroadcast03, andrzejak02rangequeries}} & |
1437 |
\parbox{110pt}{Query languages should be more powerful in tightly structured overlays} & |
\parbox{110pt}{Query languages should be more powerful in tightly structured overlays} & |
1438 |
\parbox{110pt}{SQL-like queries} & |
\parbox{110pt}{SQL-like queries} & |
1439 |
\parbox{110pt}{Hard to implement, increases system complexity, not much research has been done} |
\parbox{110pt}{Hard to implement, increases system complexity, not much research has been done} |
1440 |
\\ \hline |
\\ \hline |
1441 |
|
|
1442 |
|
|
1443 |
\parbox{90pt}{Robustness \cite{datar02butterflies}, \cite{saia02dynamicfaultcontentnetwork}, \cite{fiat02censorship}, \cite{aspnes02faultrouting}, \cite{albert-00-tolerance}, \cite{libennowell01observations}} & |
\parbox{90pt}{Robustness \cite{datar02butterflies, saia02dynamicfaultcontentnetwork, fiat02censorship, aspnes02faultrouting, albert-00-tolerance, libennowell01observations}} & |
1444 |
\parbox{110pt}{How well system performs under hostile attacks/in the case of severe failure ?} & |
\parbox{110pt}{How well system performs under hostile attacks/in the case of severe failure ?} & |
1445 |
\parbox{110pt}{Self-tuning, backup links, use diverse routing paths, power-law networks/properties} & |
\parbox{110pt}{Self-tuning, backup links, use diverse routing paths, power-law networks/properties} & |
1446 |
\parbox{110pt}{Working solutions} |
\parbox{110pt}{Working solutions} |
1461 |
\\ \hline |
\\ \hline |
1462 |
|
|
1463 |
|
|
1464 |
\parbox{90pt}{Network proximity \cite{pias03lighthouse}, \cite{ng02predicting}, \cite{ratnasamy02ght}, \cite{eriksson03peernet}, \cite{castro02networkproximity}} & |
\parbox{90pt}{Network proximity \cite{pias03lighthouse, ng02predicting, ratnasamy02ght, eriksson03peernet, castro02networkproximity}} & |
1465 |
\parbox{110pt}{Can we take into account the underlying network's properties better when forming overlay network (network-awareness for performance) ?} & |
\parbox{110pt}{Can we take into account the underlying network's properties better when forming overlay network (network-awareness for performance) ?} & |
1466 |
\parbox{110pt}{Global network positioning, lighthouse technique, triangulated heuristics} & |
\parbox{110pt}{Global network positioning, lighthouse technique, triangulated heuristics} & |
1467 |
\parbox{110pt}{Increases system complexity, no real world experience in a wide scale, proposed solutions are susceptible to single point of failure} |
\parbox{110pt}{Increases system complexity, no real world experience in a wide scale, proposed solutions are susceptible to single point of failure} |
1468 |
\\ \hline |
\\ \hline |
1469 |
|
|
1470 |
|
|
1471 |
\parbox{90pt}{Locality \cite{keleher-02-p2p}, \cite{hildrum02distributedobject}, \cite{freedman02trie}, \cite{sloppy:iptps03}, \cite{plaxton97accessingnearby}, \cite{karger02findingnearest}} & |
\parbox{90pt}{Locality \cite{keleher-02-p2p, hildrum02distributedobject, freedman02trie, sloppy:iptps03, plaxton97accessingnearby, karger02findingnearest}} & |
1472 |
\parbox{110pt}{Could tightly structured systems exploit locality properties better ?} & |
\parbox{110pt}{Could tightly structured systems exploit locality properties better ?} & |
1473 |
\parbox{110pt}{Constrained Load Balancing, using network properties for nearest neighbor selection, self-organizing clusters} & |
\parbox{110pt}{Constrained Load Balancing, using network properties for nearest neighbor selection, self-organizing clusters} & |
1474 |
\parbox{110pt}{Working solutions} |
\parbox{110pt}{Working solutions} |
1475 |
\\ \hline |
\\ \hline |
1476 |
|
|
1477 |
|
|
1478 |
\parbox{90pt}{Hot spots \cite{258660}, \cite{sloppy:iptps03}, \cite{maymounkov03ratelesscodes}} & |
\parbox{90pt}{Hot spots \cite{258660, sloppy:iptps03, maymounkov03ratelesscodes}} & |
1479 |
\parbox{110pt}{What will happen if some resource is extremely popular and only one peer is hosting it ?} & |
\parbox{110pt}{What will happen if some resource is extremely popular and only one peer is hosting it ?} & |
1480 |
\parbox{110pt}{Caching, multisource downloads, replication, load balancing, sloppy hashing} & |
\parbox{110pt}{Caching, multisource downloads, replication, load balancing, sloppy hashing} & |
1481 |
\parbox{110pt}{For query hot spots, caching and multisource downloads efficiently reduce hot spots, for routing hot spots, benefits are smaller} |
\parbox{110pt}{For query hot spots, caching and multisource downloads efficiently reduce hot spots, for routing hot spots, benefits are smaller} |
1482 |
\\ \hline |
\\ \hline |
1483 |
|
|
1484 |
|
|
1485 |
\parbox{90pt}{Load balancing \cite{rao03loadbalancing}, \cite{ledlie02selfp2p}, \cite{byers03dhtbalancing}} & |
\parbox{90pt}{Load balancing \cite{rao03loadbalancing, ledlie02selfp2p, byers03dhtbalancing}} & |
1486 |
\parbox{110pt}{Random (but uniformly distributed) identifier selection could cause system inbalance among participants with different capabilities} & |
\parbox{110pt}{Random (but uniformly distributed) identifier selection could cause system inbalance among participants with different capabilities} & |
1487 |
\parbox{110pt}{Caching, virtual server transfers} & |
\parbox{110pt}{Caching, virtual server transfers} & |
1488 |
\parbox{110pt}{Effective, more research required in fully dynamic environment} |
\parbox{110pt}{Effective, more research required in fully dynamic environment} |
1489 |
\\ \hline |
\\ \hline |
1490 |
|
|
1491 |
\parbox{90pt}{System in flux \cite{libennowell01observations}, \cite{571863}, \cite{ledlie02selfp2p}, \cite{albert-02-statistical}} & |
\parbox{90pt}{System in flux \cite{libennowell01observations, 571863, ledlie02selfp2p, albert-02-statistical}} & |
1492 |
\parbox{110pt}{Peers join and leave system constantly. What about load balancing and performance ?} & |
\parbox{110pt}{Peers join and leave system constantly. What about load balancing and performance ?} & |
1493 |
\parbox{110pt}{Half-life phenomenon (for analysis), simple overlay maintenance and construction algorithm} & |
\parbox{110pt}{Half-life phenomenon (for analysis), simple overlay maintenance and construction algorithm} & |
1494 |
\parbox{110pt}{Initial theoretical analysis have been created, but not comprehensive model for analyzing different system states and its variations (e.g. complex usage patterns)} |
\parbox{110pt}{Initial theoretical analysis have been created, but not comprehensive model for analyzing different system states and its variations (e.g. complex usage patterns)} |
1495 |
\\ \hline |
\\ \hline |
1496 |
|
|
1497 |
\parbox{90pt}{Sudden network partition \cite{harvey03skipnet1}, \cite{harvey03skipnet2}, \cite{rowston03controlloingreliability}} & |
\parbox{90pt}{Sudden network partition \cite{harvey03skipnet1, harvey03skipnet2, rowston03controlloingreliability}} & |
1498 |
\parbox{110pt}{Sub network is isolated from other network because of network disconnection} & |
\parbox{110pt}{Sub network is isolated from other network because of network disconnection} & |
1499 |
\parbox{110pt}{Self-tuning, environment observation, localized network connection for minimum latency (backup connections)} & |
\parbox{110pt}{Self-tuning, environment observation, localized network connection for minimum latency (backup connections)} & |
1500 |
\parbox{110pt}{Creates more overhead/space requirements per peer} |
\parbox{110pt}{Creates more overhead/space requirements per peer} |
1501 |
\\ \hline |
\\ \hline |
1502 |
|
|
1503 |
\parbox{90pt}{Fail Stop \cite{rowston03controlloingreliability}, \cite{zhang03somo}} & |
\parbox{90pt}{Fail Stop \cite{rowston03controlloingreliability, zhang03somo}} & |
1504 |
\parbox{110pt}{A faulty peer stops working} & |
\parbox{110pt}{A faulty peer stops working} & |
1505 |
\parbox{110pt}{Failure detectors, informing algorithms} & |
\parbox{110pt}{Failure detectors, informing algorithms} & |
1506 |
\parbox{110pt}{Creates more network traffic, peer's information can be outdated, failure detectors not reliable} |
\parbox{110pt}{Creates more network traffic, peer's information can be outdated, failure detectors not reliable} |
1545 |
\endfoot |
\endfoot |
1546 |
|
|
1547 |
|
|
1548 |
\parbox{90pt}{Mutual distrust \cite{cornelli02reputableservents}, \cite{aberer01trust}} & |
\parbox{90pt}{Mutual distrust \cite{cornelli02reputableservents, aberer01trust}} & |
1549 |
\parbox{110pt}{Nobody trusts anybody} & |
\parbox{110pt}{Nobody trusts anybody} & |
1550 |
\parbox{110pt}{Reputation methods, key infrastructures} & |
\parbox{110pt}{Reputation methods, key infrastructures} & |
1551 |
\parbox{110pt}{Resource demanding, not practical to implement/not working solutions, no real world experience in a wide scale} |
\parbox{110pt}{Resource demanding, not practical to implement/not working solutions, no real world experience in a wide scale} |
1552 |
\\ \hline |
\\ \hline |
1553 |
|
|
1554 |
|
|
1555 |
\parbox{90pt}{Lack of motivation to cooperate \cite{golle01incentivesp2p}, \cite{ngan03enforcefile}, \cite{shneidman03rationality}} & |
\parbox{90pt}{Lack of motivation to cooperate \cite{golle01incentivesp2p, ngan03enforcefile, shneidman03rationality}} & |
1556 |
\parbox{110pt}{All participants do not behave like they should be, instead they go for own profit} & |
\parbox{110pt}{All participants do not behave like they should be, instead they go for own profit} & |
1557 |
\parbox{110pt}{Different reputation methods} & |
\parbox{110pt}{Different reputation methods} & |
1558 |
\parbox{110pt}{No real world experience in a wide scale} |
\parbox{110pt}{No real world experience in a wide scale} |
1559 |
\\ \hline |
\\ \hline |
1560 |
|
|
1561 |
|
|
1562 |
\parbox{90pt}{Heterogeneity \cite{saroiu02measurementstudyp2p}, \cite{brinkmann02compactplacement}, \cite{zhao02brocade},\cite{gurmeet03symphony}, \cite{rowston03controlloingreliability}} & |
\parbox{90pt}{Heterogeneity \cite{saroiu02measurementstudyp2p, brinkmann02compactplacement, zhao02brocade, gurmeet03symphony, rowston03controlloingreliability}} & |
1563 |
\parbox{110pt}{There are different kind of peers in the system, in light of bandwidth and computing power} & |
\parbox{110pt}{There are different kind of peers in the system, in light of bandwidth and computing power} & |
1564 |
\parbox{110pt}{Super peers (loosely structured), clusters (loosely structured) additional layer upon tighty structured systems, structure itself is simple (tighty structured)} & |
\parbox{110pt}{Super peers (loosely structured), clusters (loosely structured) additional layer upon tighty structured systems, structure itself is simple (tighty structured)} & |
1565 |
\parbox{110pt}{Working solutions, increases system complexity (additional layer)} |
\parbox{110pt}{Working solutions, increases system complexity (additional layer)} |
1566 |
\\ \hline |
\\ \hline |
1567 |
|
|
1568 |
|
|
1569 |
\parbox{90pt}{Programming guidelines \cite{zhao03api}, \cite{frise02p2pframework}, \cite{babaoglu02anthill}, \cite{rhea03benchmarks}, \cite{garciamolina03sil}, \cite{balakrishnan03semanticfree}} & |
\parbox{90pt}{Programming guidelines \cite{zhao03api, frise02p2pframework, babaoglu02anthill, rhea03benchmarks, garciamolina03sil, balakrishnan03semanticfree}} & |
1570 |
\parbox{110pt}{Set of programming guidelines/frameworks is needed for better interoperability between different systems} & |
\parbox{110pt}{Set of programming guidelines/frameworks is needed for better interoperability between different systems} & |
1571 |
\parbox{110pt}{Common frameworks and APIs} & |
\parbox{110pt}{Common frameworks and APIs} & |
1572 |
\parbox{110pt}{Common framework/API is still missing, a few proposals have been made (tightly structured)} |
\parbox{110pt}{Common framework/API is still missing, a few proposals have been made (tightly structured)} |
1581 |
|
|
1582 |
|
|
1583 |
\parbox{90pt}{Overlay management and health monitoring \cite{zhang03somo}} & |
\parbox{90pt}{Overlay management and health monitoring \cite{zhang03somo}} & |
1584 |
\parbox{110pt}{System is self-capable to monitor it's status and health for better performance} & |
\parbox{110pt}{System is self-capable to monitor it is status and health for better performance} & |
1585 |
\parbox{110pt}{Build a meta data overlay atop of structured overlay (such as SOMO for structured overlays), make local decisions about overlay (loosely structured)} & |
\parbox{110pt}{Build a meta data overlay atop of structured overlay (such as SOMO for structured overlays), make local decisions about overlay (loosely structured)} & |
1586 |
\parbox{110pt}{For tightly structured overlays, efficient and simple to implement, fault tolerance unknown, for loosely structured not necessarily efficient because decisions are based on local knowledge} |
\parbox{110pt}{For tightly structured overlays, efficient and simple to implement, fault tolerance unknown, for the loosely structured approach not necessarily efficient because decisions are based on local knowledge} |
1587 |
\\ \hline |
\\ \hline |
1588 |
|
|
1589 |
\parbox{90pt}{Locating Peer-to-Peer network} & |
\parbox{90pt}{Locating Peer-to-Peer network} & |
1602 |
|
|
1603 |
\chapter{Fenfire hypermedia system} |
\chapter{Fenfire hypermedia system} |
1604 |
|
|
1605 |
In this chapter we give an overview of Fenfire system. We also |
In this chapter we give an overview of the Fenfire system. We also |
1606 |
describe briefly xanalogical storage model. At the end of this chapter we study Storm, |
describe briefly xanalogical storage model. At the end of this chapter we study Storm, |
1607 |
Fenfire's software module, which is an essential part of Fenfire's Peer-to-Peer |
Fenfire's software module, which is an essential part of Fenfire's Peer-to-Peer |
1608 |
functionality. |
functionality. |
1609 |
|
|
1610 |
\section{Overview} |
\section{Overview} |
1611 |
|
|
1612 |
Fenfire project \cite{fenfireurl} is an effort to build a location transparent, hyperstructured desktop |
The Fenfire project \cite{fenfireurl} is an effort to build a location transparent, hyperstructured desktop |
1613 |
environment. Fenfire's uses xanalogical storage model \cite{ted-xu-model} as a basis for hyperstructured |
environment. Fenfire uses xanalogical storage model \cite{ted-xu-model} as a basis for hyperstructured |
1614 |
media. Fenfire uses innovative user interfaces for displaying data to the end users. All data in Fenfire |
media. Fenfire deploys innovative user interfaces for displaying data to the end users. All data in the Fenfire |
1615 |
is stored in same format, i.e., blocks. This should allow making references between data easier and more |
is stored in a unified format, i.e., blocks. This should allow making references between data easier and more |
1616 |
seamlessly interoperating than in other systems. For location transparency in a distributed system, Fenfire |
seamlessly interoperating than in other systems. For location transparency in a distributed system, Fenfire |
1617 |
uses Peer-to-Peer network for locating and fetching blocks. |
uses Peer-to-Peer network for locating and fetching blocks. |
1618 |
|
|
1619 |
Fenfire is free software and it is licensed under GNU L-GPL. Fenfire was formerly also a partial implementation |
Fenfire is free software and it is licensed under GNU LGPL. Fenfire was formerly also a partial implementation |
1620 |
of the ZigZag\texttrademark --structure, which was originally invented |
of the ZigZag\texttrademark --structure, which was originally invented |
1621 |
by Ted Nelson. Now, however, Fenfire uses Resource Description Framework (RDF) \cite{w3rdfurl} |
by Ted Nelson. Now, however, Fenfire uses Resource Description Framework (RDF) \cite{w3rdfurl} |
1622 |
for representing internal data structures and their relationships. |
for representing internal data structures and their relationships. |
1623 |
|
|
1624 |
Fenfire is high modular software system. It consists of several independent software modules: |
Fenfire is a high modular software system. It consists of several independent software modules: |
1625 |
|
|
1626 |
\begin{itemize} |
\begin{itemize} |
1627 |
\item \textbf{Storm}: distributed storage module for storing arbitrary data items |
\item \textbf{Storm}: distributed storage module for storing arbitrary data items |
1634 |
|
|
1635 |
In this thesis, we focus on Storm and Alph modules, since they are the foundation of Fenfire's |
In this thesis, we focus on Storm and Alph modules, since they are the foundation of Fenfire's |
1636 |
Peer-to-Peer functionality. If not otherwise mentioned, we use term 'Storm' when referring to both |
Peer-to-Peer functionality. If not otherwise mentioned, we use term 'Storm' when referring to both |
1637 |
Storm and Alph software modules. For location transparency in Fenfire system, Storm software module |
Storm and Alph software modules. For location transparency in the Fenfire system, Storm software module |
1638 |
must have support for Peer-to-Peer functionality as it provides low-level data storage operations |
must have support for Peer-to-Peer functionality as it provides low-level data storage operations |
1639 |
in Fenfire system. |
in the Fenfire system. |
1640 |
|
|
1641 |
|
|
1642 |
\section{Xanalogical storage model} |
\section{Xanalogical storage model} |
1665 |
and bidirectional. Xanadu link is an \emph{association} of two enfilades, such as an |
and bidirectional. Xanadu link is an \emph{association} of two enfilades, such as an |
1666 |
annotation to a specific part of a another document. \emph{Transclusion} is an inclusion in |
annotation to a specific part of a another document. \emph{Transclusion} is an inclusion in |
1667 |
enfilade of contents already used in another enfilade, i.e., current fluid media is copied into |
enfilade of contents already used in another enfilade, i.e., current fluid media is copied into |
1668 |
different data contents. By using this mechanism, system implementing xanalogical storage model |
different data contents. By using this mechanism, a system implementing xanalogical storage model |
1669 |
is able to show all data content that share same fluid media with current data content |
is able to show all data content that share same fluid media with current data content |
1670 |
(e.g., all documents containing current document's text). Figure \ref{fig:xanalogical_model} |
(e.g., all documents containing current document's text). Figure \ref{fig:xanalogical_model} |
1671 |
illustrates xanalogical storage model with documents, text and characters. |
illustrates xanalogical storage model with documents, text and characters. |
1685 |
from recent publications: for general discussion about Fenfire in Peer-to-Peer environment, |
from recent publications: for general discussion about Fenfire in Peer-to-Peer environment, |
1686 |
see \cite{lukka02freenetguids}, and for detailed Storm design, see \cite{fallenstein03storm}. |
see \cite{lukka02freenetguids}, and for detailed Storm design, see \cite{fallenstein03storm}. |
1687 |
|
|
1688 |
Storm (for \emph{STORage Module}) is a software module, which is used in Fenfire for |
Storm (for \emph{STORage Module}) is a software module, which is used in the Fenfire for |
1689 |
data storage operations. Storm stores all data as \emph{blocks}, which |
data storage operations. Storm stores all data as \emph{blocks}, which |
1690 |
are immutable byte sequences. SHA-1\footnote{SHA-1 is considered as a collision free |
are immutable byte sequences. SHA-1\footnote{SHA-1 is considered as a collision free |
1691 |
hash function. Therefore, it is very unlikely that two different Storm data blocks |
hash function. Therefore, it is very unlikely that two different Storm data blocks |
1695 |
blocks have much in common with regular files, except that Storm blocks are \emph{immutable} as |
blocks have much in common with regular files, except that Storm blocks are \emph{immutable} as |
1696 |
any change to the byte sequence would change block's hash value, i.e., globally unique |
any change to the byte sequence would change block's hash value, i.e., globally unique |
1697 |
identifier. This mechanism creates a basis for implementing xanalogical storage model in |
identifier. This mechanism creates a basis for implementing xanalogical storage model in |
1698 |
Fenfire system. Figure \ref{fig:storm_model} illustrates simplified Storm storage model. |
in Fenfire system. Figure \ref{fig:storm_model} illustrates simplified Storm storage model. |
1699 |
|
|
1700 |
\begin{figure} |
\begin{figure} |
1701 |
\centering |
\centering |
1712 |
|
|
1713 |
Pointer \cite{benja02urn5} is a semantic-free, updatable reference to |
Pointer \cite{benja02urn5} is a semantic-free, updatable reference to |
1714 |
Storm data block, i.e., Storm scroll block. |
Storm data block, i.e., Storm scroll block. |
1715 |
In practice, pointer is random string, which resembles Universal Resource Names |
In practice, pointer is a random string, which resembles Universal Resource Names |
1716 |
(URN) \cite{rfc2396}. Pointer itself doesn't contain any data, it's rather a \emph{concept} of |
(URN) \cite{rfc2396}. Pointer itself doesn't contain any data, it is rather a \emph{concept} of |
1717 |
data. Pointers are created automatically by Storm and each pointer is |
data. Pointers are created automatically by Storm and each pointer is |
1718 |
associated with a collection of \emph{pointer blocks}. Pointer block has a single |
associated with a collection of \emph{pointer blocks}. Pointer block has a single |
1719 |
target for the pointer. In figure \ref{fig:storm_model}, we present overall |
target for the pointer. In figure \ref{fig:storm_model}, we present overall |
1737 |
In this chapter we evaluate Fenfire in Peer-to-Peer environment. |
In this chapter we evaluate Fenfire in Peer-to-Peer environment. |
1738 |
We start by giving a problem overview when considering Fenfire in Peer-to-Peer |
We start by giving a problem overview when considering Fenfire in Peer-to-Peer |
1739 |
environment. We define Fenfire's special needs and evaluate existing |
environment. We define Fenfire's special needs and evaluate existing |
1740 |
Peer-to-Peer approaches in light of these requirements. After that, we propose system |
Peer-to-Peer approaches in light of these requirements. After that, we propose a system |
1741 |
model for Fenfire in Peer-to-Peer environment and present simple methods to perform data |
model for Fenfire in Peer-to-Peer environment and present simple methods to perform data |
1742 |
lookups in Peer-to-Peer environment. In the end of this chapter, we discuss possible problems of using Fenfire |
lookups in Peer-to-Peer environment. In the end of this chapter, we discuss possible problems of using Fenfire |
1743 |
in Peer-to-Peer environment |
in Peer-to-Peer environment. |
1744 |
|
|
1745 |
|
|
1746 |
\section{Problem overview} |
\section{Problem overview} |
1752 |
from fragments of data. |
from fragments of data. |
1753 |
|
|
1754 |
In the xanalogical storage model, each fragment of data is identified by a globally |
In the xanalogical storage model, each fragment of data is identified by a globally |
1755 |
unique identifier. In Fenfire, data fragments are scroll blocks, generated by Storm storage module. |
unique identifier. In the Fenfire system, data fragments are scroll blocks, generated by Storm storage module. |
1756 |
As we discussed already in chapter 4, Fenfire's Storm design |
As we discussed already in chapter 4, Fenfire's Storm design |
1757 |
uses SHA-1 \cite{fips-sha-1} hash over the contents of a scroll block for creating globally unique |
uses SHA-1 \cite{fips-sha-1} hash over the contents of a scroll block for creating globally unique |
1758 |
identifiers for each scroll block. In our scenario, fragments of data is distributed |
identifiers for each scroll block. In our scenario, fragments of data is distributed |
1790 |
\section{Evaluation of Peer-to-Peer approaches with regard to Fenfire} |
\section{Evaluation of Peer-to-Peer approaches with regard to Fenfire} |
1791 |
|
|
1792 |
In chapter 2, we discussed main differences between loosely and tightly structured |
In chapter 2, we discussed main differences between loosely and tightly structured |
1793 |
approaches. As stated, the most significant difference is that tightly structured |
approaches. As stated, the most significant difference is that the tightly structured |
1794 |
approach has logarithmical properties in all internal operations, while loosely |
approach has logarithmical properties in all internal operations, while the loosely |
1795 |
structured approach doesn't always have even linear properties. Furthermore, the |
structured approach doesn't always have even linear properties. Furthermore, the |
1796 |
data lookup model of tightly structured overlay scales much better than loosely |
data lookup model of tightly structured overlay scales much better than loosely |
1797 |
structured overlays; tightly structured overlay supports global data lookups |
structured overlays; tightly structured overlay supports global data lookups |
1798 |
in the overlay, whereas the data lookup model of loosely structured approach |
in the overlay, whereas the data lookup model of the loosely structured approach |
1799 |
is limited to certain area of overlay\footnote{The area depends on where the query |
is limited to certain area of overlay\footnote{The area depends on where the query |
1800 |
originator is located in the overlay.}. |
originator is located in the overlay.}. |
1801 |
|
|
1802 |
For Fenfire's special needs for \emph{locating} data, an important advantage of |
For Fenfire's special needs for \emph{locating} data, an important advantage of the |
1803 |
tightly structured approach over loosely structured approach is that tightly |
tightly structured approach over the loosely structured approach is that tightly |
1804 |
structured systems use location-independent, globally unique identifiers for |
structured systems use location-independent, globally unique identifiers for |
1805 |
identifying data in the system. Indeed, this |
identifying data in the system. Indeed, this |
1806 |
feature is almost analogical to Fenfire's (and xanalogical storage model's) way of |
feature is almost analogical to Fenfire's (and xanalogical storage model's) way of |
1809 |
Domain Name System (DNS) \cite{rfc1101} is widely used RRS system in the Internet.} |
Domain Name System (DNS) \cite{rfc1101} is widely used RRS system in the Internet.} |
1810 |
\cite{balakrishnan03semanticfree}. Authors argue that next generation RRS must be |
\cite{balakrishnan03semanticfree}. Authors argue that next generation RRS must be |
1811 |
application-independent and references itself should be \emph{unstructured} and |
application-independent and references itself should be \emph{unstructured} and |
1812 |
\emph{semantic free}. Finally, as said, with tightly structured systems, it is feasible to |
\emph{semantically free}. Finally, as said, with tightly structured systems, it is feasible to |
1813 |
perform \emph{global} data lookups in the overlay. To summarize, these aspects may be the most important features |
perform \emph{global} data lookups in the overlay. To summarize, these aspects may be the most important features |
1814 |
of Peer-to-Peer infrastructure with regard to Fenfire as a distributed, location transparent hypermedia system. |
of Peer-to-Peer infrastructure with regard to Fenfire as a distributed, location transparent hypermedia system. |
1815 |
Thus, we see the tightly structured approach as the best alternative to locate data in Peer-to-Peer |
Thus, we see the tightly structured approach as the best alternative to locate data in Peer-to-Peer |
1816 |
environment. |
environment. |
1817 |
|
|
1818 |
Once located, for \emph{fetching} Fenfire related data from the overlay, we can use regular |
Once located, for \emph{fetching} Storm blocks from the overlay, we can use regular |
1819 |
TCP/IP-protocols, such as Hypertext Transfer protocol (HTTP) \cite{rfc2068}. However, HTTP-protocol may |
TCP/IP-protocols, such as Hypertext Transfer protocol (HTTP) \cite{rfc2068}. However, HTTP-protocol may |
1820 |
not be optimal when obtaining large amounts of data from the Peer-to-Peer network (e.g., |
not be optimal when obtaining large amounts of data from the Peer-to-Peer network (e.g., |
1821 |
videos, images or music). In this case, multisource downloads can be very useful |
videos, images or music). In this case, multisource downloads can be very useful |
1822 |
for better efficiency \cite{maymounkov03ratelesscodes}, \cite{bittorrenturl}. Furthermore, |
for better efficiency \cite{maymounkov03ratelesscodes, bittorrenturl}. Furthermore, |
1823 |
multisource downloads can be used for decreasing load of certain peer, thus avoiding query |
multisource downloads can be used for decreasing load of certain peer, thus avoiding query |
1824 |
hot spots in the system \cite{ratnasamy02routing}. Current implementation of Fenfire uses |
hot spots in the system \cite{ratnasamy02routing}. Current implementation of Fenfire uses |
1825 |
standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content |
standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content |
1828 |
tree-based hash\footnote{With multisource downloads, tree based hash functions can be used |
tree-based hash\footnote{With multisource downloads, tree based hash functions can be used |
1829 |
to verify fixed length segments of data. If hash value of data segment is incorrect, |
to verify fixed length segments of data. If hash value of data segment is incorrect, |
1830 |
we need only to fetch \emph{segment} of data (instead of whole data, e.g., a file) from |
we need only to fetch \emph{segment} of data (instead of whole data, e.g., a file) from |
1831 |
other source.}, such as \cite{merkle87hashtree} and \cite{mohr02thex} for reliable and efficient |
other source.}, such as \cite{merkle87hashtree, mohr02thex} for reliable and efficient |
1832 |
data validation. |
data validation. |
1833 |
|
|
1834 |
Again, there are research challenges with tightly structured systems which have to be |
Again, there are research challenges with tightly structured systems which have to be |
1836 |
tolerance when system in presence of system flux, non-optimal distance functions in identifier space, |
tolerance when system in presence of system flux, non-optimal distance functions in identifier space, |
1837 |
proximity routing, hostile entities and flexible search \cite{balakrishanarticle03lookupp2p}. |
proximity routing, hostile entities and flexible search \cite{balakrishanarticle03lookupp2p}. |
1838 |
Additionally, there is only little real world experiments yet with tightly structured systems |
Additionally, there is only little real world experiments yet with tightly structured systems |
1839 |
(e.g., \cite{overneturl}, \cite{edonkey2kurl}). Therefore, we can't say for sure, how well these |
(e.g., \cite{overneturl, edonkey2kurl}). Therefore, we can't say for sure, how well these |
1840 |
systems would perform in real Peer-to-Peer environment. However, we believe that issues are |
systems would perform in real Peer-to-Peer environment. However, we believe that issues are |
1841 |
solved, since there is a strong and wide research community towards to tightly structured |
solved, since there is a strong and wide research community towards to tightly structured |
1842 |
overlays \cite{projectirisurl}. |
overlays \cite{projectirisurl}. |
1854 |
locating data efficiently in the Peer-to-Peer overlay. There are two main |
locating data efficiently in the Peer-to-Peer overlay. There are two main |
1855 |
reasons for this. First, Kademlia's XOR-based distance function is superior |
reasons for this. First, Kademlia's XOR-based distance function is superior |
1856 |
over the distance functions of other systems (see section 2.4). Second, there exist already |
over the distance functions of other systems (see section 2.4). Second, there exist already |
1857 |
deployed real-life systems using Kademlia (e.g., \cite{overneturl}, \cite{edonkey2kurl}, \cite{kashmirurl}, |
deployed real-life systems using Kademlia (e.g., \cite{overneturl, edonkey2kurl, kashmirurl, |
1858 |
\cite{kato02gisp}), which means that Kademlia's algorithm is simple and easy to implement. |
kato02gisp}), which means that Kademlia's algorithm is simple and easy to implement. |
1859 |
|
|
1860 |
On top of Kademlia, we propose the usage of Sloppy hashing \cite{sloppy:iptps03} which |
On top of Kademlia, we propose the usage of Sloppy hashing \cite{sloppy:iptps03} which |
1861 |
is optimized for the DOLR abstraction of tightly structured overlays. With the Sloppy hashing, |
is optimized for the DOLR abstraction of tightly structured overlays. With the Sloppy hashing, |
1877 |
|
|
1878 |
\subsection{Methods} |
\subsection{Methods} |
1879 |
|
|
1880 |
We use the DOLR abstraction of tightly structured approach, i.e., each participating peer hosts |
We use the DOLR abstraction of the tightly structured approach, i.e., each participating peer hosts |
1881 |
the data and overlay maintains only the \emph{pointers} to the data. We decided to use the DOLR |
the data and overlay maintains only the \emph{pointers} to the data. We decided to use the DOLR |
1882 |
abstraction in our model, since DOLR systems locate data without specifying a storage policy explicitly \cite{rhea03benchmarks}. |
abstraction in our model, since DOLR systems locate data without specifying a storage policy explicitly \cite{rhea03benchmarks}. |
1883 |
DHT based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have |
DHT based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have |
1884 |
critical problems with load balancing in highly heterogeneous environment. This problem is caused by peers |
critical problems with load balancing in highly heterogeneous environment. This problem is caused by peers |
1885 |
which may not be able to store relatively large amount of data with key/value pair, assigned randomly by |
which may not be able to store relatively large amount of data with key/value pair, assigned randomly by |
1886 |
mapping function of the overlay. These systems wastes both storage and bandwidth, and |
mapping function of the overlay. These systems waste both storage and bandwidth, and |
1887 |
are sensitive to certain attacks (e.g., DDoS attack). Additionally, we emphasize that we prefer \emph{abstraction} |
are sensitive to certain attacks (e.g., DDoS attack). Additionally, we emphasize that we prefer \emph{abstraction} |
1888 |
level analysis as very recently better and better tightly structured algorihtms have been proposed. |
level analysis as very recently better and better tightly structured algorithms have been proposed. |
1889 |
Thus, we don't want to bind our system proposal to a specific algorithm definitively as we expect |
Thus, we don't want to bind our system proposal to a specific algorithm definitively as we expect |
1890 |
that this development continues. |
that this development continues. |
1891 |
|
|
1916 |
\end{itemize} |
\end{itemize} |
1917 |
|
|
1918 |
Figure \ref{fig:storm_query_blockid} illustrates how Storm scroll block is located |
Figure \ref{fig:storm_query_blockid} illustrates how Storm scroll block is located |
1919 |
in a tightly structured overlay using the DOLR abstraction, where identifier of Storm scroll |
in tightly structured overlay using the DOLR abstraction, where identifier of Storm scroll |
1920 |
block is known. |
block is known. |
1921 |
|
|
1922 |
|
|
1941 |
\end{itemize} |
\end{itemize} |
1942 |
|
|
1943 |
Figure \ref{fig:storm_query_urn5} illustrates how Storm scroll block is located |
Figure \ref{fig:storm_query_urn5} illustrates how Storm scroll block is located |
1944 |
in a tightly structured overlay using the DOLR abstraction, where pointer random string is known. |
in tightly structured overlay using the DOLR abstraction, where pointer random string is known. |
1945 |
|
|
1946 |
Each of these algorithms can locate Fenfire related data in $O(\log{n})$ time at application level overlay: |
Each of these algorithms can locate Fenfire data in $O(\log{n})$ time at application level overlay: |
1947 |
$O(\log{n})$ time for query routing to pointer peer and constant time for |
$O(\log{n})$ time for query routing to pointer peer and constant time for |
1948 |
locating hosting peer with a given reference link. |
locating hosting peer with a given reference link. |
1949 |
|
|
1969 |
security technologies. For instance, online entities cannot be identified |
security technologies. For instance, online entities cannot be identified |
1970 |
safely (e.g., the Sybil attack \cite{douceur02sybil}). For Fenfire, one |
safely (e.g., the Sybil attack \cite{douceur02sybil}). For Fenfire, one |
1971 |
security related problem occurs when user wants to perform global data lookup with a given |
security related problem occurs when user wants to perform global data lookup with a given |
1972 |
pointer random string; how the user is able to verify the correctness |
pointer random string; how can the user verify the correctness |
1973 |
of the search results ? Specifically, how she/he knows which one is the |
of the search results ? Specifically, how she or he knows which one is the |
1974 |
correct Storm scroll block ? Spam attack \cite{naor03simpledht} is a variation of previously |
correct Storm scroll block ? Spam attack \cite{naor03simpledht} is a variation of previously |
1975 |
mentioned problem; data lookup is performed by a user, but there is no reply |
mentioned problem; data lookup is performed by a user, but there is no reply |
1976 |
from the system. How we are able to know if this was a spam attack, or the |
from the system. How are we able to know if this was a spam attack, or the |
1977 |
data really doesn't exist in the system ? Another problem related to Fenfire's |
data really doesn't exist in the system ? Another problem related to the Fenfire's |
1978 |
security is that if a user downloads data from the network to local computer |
security is that if a user downloads data from the network to local computer |
1979 |
and after network disconnection, user wants to verify \emph{off line} the |
and after network disconnection, user wants to verify \emph{off line} the |
1980 |
authenticity of data. Obviously, optimal solution to all security issues would |
authenticity of data. Obviously, optimal solution to all security issues would |
1981 |
be that digital signatures are included to every message sent to the system therefore |
be that digital signatures are included to every message sent to the system therefore |
1982 |
enabling peers to authenticate other peers safely. However, these problems are not |
enabling peers to authenticate other peers safely. However, these problems are not |
1983 |
only limited to Fenfire as it concerns all Peer-to-Peer based computer systems. |
only limited to the Fenfire as it concerns all Peer-to-Peer based computer systems. |
1984 |
|
|
1985 |
As security technologies come more mature, we wish to apply these |
As security technologies come more mature, we wish to apply these |
1986 |
technologies with Fenfire, if applicable. |
technologies with Fenfire, if applicable. |
1995 |
yet, or solutions are only partial. We point out that much research work is required to |
yet, or solutions are only partial. We point out that much research work is required to |
1996 |
solve these problems. |
solve these problems. |
1997 |
|
|
1998 |
Then, we focused our attention to Fenfire system. First, we gave a brief |
Then, we focused our attention to the Fenfire system. First, we gave a brief |
1999 |
overview of Fenfire and xanalogical model. We also described Storm software module. |
overview of Fenfire and xanalogical model. We also described Storm software module. |
2000 |
|
|
2001 |
In the last chapter, we evaluated existing Peer-to-Peer approaches with regard |
In the last chapter, we evaluated existing Peer-to-Peer approaches with regard |
2002 |
to Fenfire's needs. We proposed that tightly structured approach is the |
to Fenfire's needs. We proposed that the tightly structured approach is the |
2003 |
best alternative to Fenfire's needs for the following reasons. First, Storm, xanalogical |
best alternative to Fenfire's needs for the following reasons. First, Storm, xanalogical |
2004 |
model and tightly structured systems use global unique identifiers |
model and tightly structured systems use global unique identifiers |
2005 |
for identifying data. Second, our Storm design uses \emph{semantic-free references} |
for identifying data. Second, our Storm design uses \emph{semantic-free references} |
2009 |
we also agree that tightly structured overlays provide general purpose |
we also agree that tightly structured overlays provide general purpose |
2010 |
interface to next-generation reference resolution services. Third, by using |
interface to next-generation reference resolution services. Third, by using |
2011 |
the DOLR abstraction of tightly structured overlay, we can minimize the lack |
the DOLR abstraction of tightly structured overlay, we can minimize the lack |
2012 |
of locality in tightly structured approach. Finally, we believe that issues |
of locality in the tightly structured approach. Finally, we believe that issues |
2013 |
related to tightly structured overlays are solved in near future, because of |
related to tightly structured overlays are solved in near future, because of |
2014 |
wide and intensive co-operation among research groups. |
wide and intensive co-operation among research groups. |
2015 |
|
|