34 |
\contactinformation{\\ |
\contactinformation{\\ |
35 |
Hermanni Hyytiälä\\ |
Hermanni Hyytiälä\\ |
36 |
Huhtalammentie 5 as. 17\\ |
Huhtalammentie 5 as. 17\\ |
37 |
37637 JYVÄSKYLÄ\\ |
40640 JYVÄSKYLÄ\\ |
38 |
sähköposti: hemppah@cc.jyu.fi} |
sähköposti: hemppah@cc.jyu.fi} |
39 |
|
|
40 |
|
|
43 |
key properties. We summarize open problems in Peer-to-Peer networks and divide |
key properties. We summarize open problems in Peer-to-Peer networks and divide |
44 |
problems into three sub-categories. We observe that there are many |
problems into three sub-categories. We observe that there are many |
45 |
problems, which have not solutions at all, or problems have proposed |
problems, which have not solutions at all, or problems have proposed |
46 |
solutions, but they are practically unrealizable. |
solutions but they are practically unrealizable. |
47 |
|
|
48 |
Then, we give an overview of our Fenfire system. We evaluate existing |
Then, we give an overview of our Fenfire system. We evaluate existing |
49 |
Peer-to-Peer approaches-- loosely and tightly structured overlays-- with regard |
Peer-to-Peer approaches-- loosely and tightly structured overlays-- with regard |
52 |
} |
} |
53 |
\tiivistelma{ |
\tiivistelma{ |
54 |
Tässä opinnäytetyössä arvioimme olemassaolevia vertaisverkkoja, protokollia ja |
Tässä opinnäytetyössä arvioimme olemassaolevia vertaisverkkoja, protokollia ja |
55 |
niiden erikoisominaisuuksia. Teemme yhteenvedon olemassa olevista ongelmista |
niiden erityisominaisuuksia. Teemme yhteenvedon olemassa olevista ongelmista |
56 |
vertaisverkoista ja jaamme ongelmat kolmeen alakategoriaan. Havaitsemme, että |
vertaisverkoissa ja jaamme ongelmat kolmeen alakategoriaan. Havaitsemme, että |
57 |
on olemassa useita ongelmia, joihin ei ole ratkaisua lainkaan, tai on ehdotelma |
on olemassa useita ongelmia, joihin ei ole ratkaisua lainkaan, tai on ehdotelma |
58 |
ongelman ratkaisemiseki, mikä kuitenkin käytännössä on mahdotonta toteuttaa. |
ongelman ratkaisemiseksi, mikä käytännössä on kuitenkin mahdotonta toteuttaa. |
59 |
|
|
60 |
Tämän jälkeen annamme yleiskuvan Fenfire-järjestelmästämme. |
Tämän jälkeen annamme yleiskuvan Fenfire-järjestelmästä. |
61 |
Arvioimme olemassaolevia vertaisverkkoarkkitehtuureja-- löyhästi ja tiukasti |
Arvioimme olemassaolevia vertaisverkkoarkkitehtuureja-- löyhästi ja tiukasti |
62 |
rakennettuja päällysverkkoja-- Fenfiren vaatimusten valossa. Lopuksi ehdotamme |
rakennettuja päällysverkkoja-- Fenfiren vaatimusten valossa. Lopuksi ehdotamme |
63 |
yksin-kertaisia algoritmeja, joiden avulla voidaan tehokkaasti löytää |
yksin-kertaisia algoritmeja, joiden avulla voidaan tehokkaasti löytää |
64 |
vertaisverkosta tietoa Fenfire:n liittyen. |
vertaisverkosta Fenfire:n kannalta olennaista tietoa |
65 |
} |
} |
66 |
|
|
67 |
\begin{document} |
\begin{document} |
81 |
Peer-to-Peer systems have recently received noteworthy attention in both |
Peer-to-Peer systems have recently received noteworthy attention in both |
82 |
academia and industry for a number of reasons. First, the lack of centralization |
academia and industry for a number of reasons. First, the lack of centralization |
83 |
means that participants can form a distributed system without any investment to |
means that participants can form a distributed system without any investment to |
84 |
centralized, high-priced hardware which would to coordinate it. Second, Peer-to-Peer |
centralized, high-priced hardware which would coordinate it. Second, Peer-to-Peer |
85 |
provides new, direct way to achieve interoperability between network participants. |
provides new direct way to achieve interoperability between network participants. |
86 |
Finally, the distributed and ad-hoc nature of Peer-to-Peer improves scalability |
Finally, the distributed and ad-hoc nature of Peer-to-Peer improves scalability |
87 |
and reliability againts certain kinds of faults (e.g., single point of failure). |
and reliability againts certain kinds of faults (e.g., single point of failure). |
88 |
|
|
111 |
problems in easy-to-understand tables. |
problems in easy-to-understand tables. |
112 |
|
|
113 |
Next, we give an overview of our Fenfire system, which implements xanalogical storage model. We |
Next, we give an overview of our Fenfire system, which implements xanalogical storage model. We |
114 |
also describe briefly Storm software module, which is essential part of Fenfire's |
also describe briefly Storm software module, which is an essential part of Fenfire's |
115 |
Peer-to-Peer functionality. We evaluate existing Peer-to-Peer approaches and |
Peer-to-Peer functionality. We evaluate existing Peer-to-Peer approaches and |
116 |
choose the best alternative to our needs. We discover that Storm, xanalogical model and |
choose the best alternative to our needs. We discover that Storm, xanalogical model and |
117 |
tightly structured Peer-to-Peer approach all have similar method to deal with data, |
tightly structured Peer-to-Peer approach all have similar method to deal with data, |
129 |
is to find the most efficient way to locate and fetch Fenfire related data from the |
is to find the most efficient way to locate and fetch Fenfire related data from the |
130 |
Peer-to-Peer network, where Scroll block's identifier is given. Second, we want |
Peer-to-Peer network, where Scroll block's identifier is given. Second, we want |
131 |
to find the most efficient way to locate and fetch most recent Fenfire related data from the |
to find the most efficient way to locate and fetch most recent Fenfire related data from the |
132 |
the Peer-to-Peer network, which is associated with a given urn-5 random string. Final problem |
the Peer-to-Peer network, which is associated with a given urn-5 random string. Third problem |
133 |
is otherwise same as the second problem, except we want to locate and fetch all Fenfire |
is otherwise same as the second problem, except we want to locate and fetch all Fenfire |
134 |
related data from the Peer-to-Peer network, where given date and/or time range is given. |
related data from the Peer-to-Peer network, where given date and/or time range is given. |
135 |
|
|
136 |
When comparing different Peer-to-Peer approaches and algorithms, we will examine their |
When comparing different Peer-to-Peer approaches and algorithms, we will examine their |
137 |
scalability, efficiency, space requirements for neighbor connections and overhead |
scalability, efficiency, space requirements for neighbor connections and overhead |
138 |
associated with system maintenance. When we have solutions to our research |
associated with system maintenance. Finally, when we have solutions to our research |
139 |
problems, we will use best solutions as examples in our algorithm proposals. |
problems, we will use best solutions as examples in our algorithm proposals. |
140 |
|
|
141 |
\section{Thesis overview} |
\section{Thesis overview} |
142 |
This thesis is structured as follows. In next chapter, we give an overview of |
This thesis is structured as follows. In next chapter, we give an overview of |
143 |
existing Peer-to-Peer approaches, algorithms and key differences. In chapter 3, we |
existing Peer-to-Peer approaches, algorithms and key differences. In chapter 3, we |
144 |
address open problems in Peer-to-Peer domain and divide problems into three |
address open problems in Peer-to-Peer domain and divide problems into three |
145 |
sub-categories. Chapter 4 gives an overview of our Fenfire system. In chapter |
sub-categories. Chapter 4 gives an overview of Fenfire system. In chapter |
146 |
5 we evaluate existing Peer-to-Peer approaches with regard to Fenfire system and |
5, we evaluate existing Peer-to-Peer approaches with regard to Fenfire system and |
147 |
propose simple algorithms perform data lookups in Fenfire's Peer-to-Peer enviroment. |
propose simple algorithms to perform data lookups in Fenfire's Peer-to-Peer enviroment. |
148 |
We also discuss open issues and future work. Finally, we present conclusions in chapter |
Then, we discuss open issues and future work. In chapter 6, we present conclusions. |
149 |
6. |
|
150 |
|
|
151 |
\chapter{Peer-to-Peer architectures} |
\chapter{Peer-to-Peer architectures} |
152 |
In this chapter we give brief history and overview of Peer-to-Peer networks, |
In this chapter we will give brief history and overview of Peer-to-Peer networks, |
153 |
review most important Peer-to-Peer algorithms and list key differences between |
review most important Peer-to-Peer algorithms and list key differences between |
154 |
two main approaches. |
two main approaches. |
155 |
|
|
189 |
form of modern Peer-to-Peer computing is file-sharing. In this scenario, participants |
form of modern Peer-to-Peer computing is file-sharing. In this scenario, participants |
190 |
of Peer-to-Peer network share their resources to other participants while obtaining |
of Peer-to-Peer network share their resources to other participants while obtaining |
191 |
more resources from others. This can been seen as a variant of distributed filesystem |
more resources from others. This can been seen as a variant of distributed filesystem |
192 |
(see, e.g., \cite{levy90distributedfilesystems}). |
(e.g., \cite{levy90distributedfilesystems}). |
193 |
|
|
194 |
In a development of modern Peer-to-Peer systems, lot of influences has been attained from |
In a development of modern Peer-to-Peer systems, lot of influences has been attained from |
195 |
other research areas than computer science. There has been done research regarding |
other research areas than computer science. There has been done research regarding |
196 |
to ad-hoc nature of complex networks \cite{albert-02-statistical}, \cite{albert-00-tolerance}, \cite{watts00dynamics}. |
to ad-hoc nature of complex networks \cite{albert-02-statistical}, \cite{albert-00-tolerance}, \cite{watts00dynamics}. |
197 |
It's interesting to realize that chemical properties of cells, the Internet, ad-hoc |
It's interesting to realize that chemical properties of cells, the Internet, ad-hoc |
198 |
Peer-to-Peer networks, have in common that they all self-organize based on same |
Peer-to-Peer networks, have all in common that they self-organize based on same |
199 |
principles. Furthermore, the assocation between social connections among people |
principles. Furthermore, the assocation between social connections among people |
200 |
and Peer-to-Peer overlay topology has been studied recently \cite{watts00dynamics}, |
and Peer-to-Peer overlay topology has been studied recently \cite{watts00dynamics}, |
201 |
\cite{kleinberg99small}, \cite{nips02-Kleinberg}. This insight is motivated |
\cite{kleinberg99small}, \cite{nips02-Kleinberg}. This insight is motivated |
202 |
by Milgram, how noticed that people are very effective to locate other people in a wide scale, |
by Milgram, how noticed that people are very effective to locate other people in a wide scale |
203 |
based on local knowledge. This phenomenon is called as ''small-world phenomenon'' |
based on local knowledge. This phenomenon is called as ''small-world phenomenon'' |
204 |
\cite{milgram67smallworld}. As a consequence, many modern Peer-to-Peer systems |
\cite{milgram67smallworld}. As a consequence, many modern Peer-to-Peer systems |
205 |
have applied techniques outside of computer science when constructing and maintaining |
have applied techniques outside of computer science when constructing and maintaining |
207 |
|
|
208 |
In the end, however, there are two main approaches in which all modern Peer-to-Peer |
In the end, however, there are two main approaches in which all modern Peer-to-Peer |
209 |
systems fall: loosely structured approach and tightly structured approach. In loosely |
systems fall: loosely structured approach and tightly structured approach. In loosely |
210 |
structured approach the construction and the maintenance of the overlay is-- as the name |
structured approach the construction and the maintenance of the overlay-- as the name |
211 |
suggests- is controlled loosely. This approach gives freedom for participating peers |
suggests- is controlled loosely. This approach gives freedom for participating peers |
212 |
to perform certains tasks in Peer-to-Peer network. On the other hand, tightly structured |
to perform certains tasks in Peer-to-Peer network. On the other hand, tightly structured |
213 |
approach has some rules, which all participating peers have to obey. In the following |
approach has some rules, which all participating peers have to obey. In the following |
230 |
\section{Loosely structured} |
\section{Loosely structured} |
231 |
|
|
232 |
Gnutella \cite{gnutellaurl} is well-known example of loosely structured overlay network. As in |
Gnutella \cite{gnutellaurl} is well-known example of loosely structured overlay network. As in |
233 |
other Peer-to-Peer networks, no Peer is more important than any other Peer in the network. |
other Peer-to-Peer networks, no peer is more important than any other peer in the network. |
234 |
The construction and maintenance of Gnutella network is extremely ad hoc, since participating |
The construction and maintenance of Gnutella network is extremely ad-hoc, since participating |
235 |
peers can form the overlay network based on local knowledge. Figure \ref{fig:gnutella_overlay} |
peers can form the overlay network based on \emph{local} knowledge. Figure \ref{fig:gnutella_overlay} |
236 |
illustrates how peers form an overlay network. Initially, peer 1 creates the overlay, since |
illustrates how peers form an overlay network. Initially, peer 1 creates the overlay, since |
237 |
it's the first participating peer. Then, repeatly new peers join the network and connects to |
it's the first participating peer. Then, repeatly new peers join the network and connects to |
238 |
other nodes in a random manner. Thus, gnutella can be considered as a \emph{random graph}. |
other nodes in a random manner. Thus, gnutella can be considered as a variation of \emph{random graph}. |
239 |
|
|
240 |
\begin{figure} |
\begin{figure} |
241 |
\centering |
\centering |
246 |
|
|
247 |
|
|
248 |
In Gnutella, each participating peer maintains local index of its own shared content. Also, |
In Gnutella, each participating peer maintains local index of its own shared content. Also, |
249 |
each peer has a few connections to other peer, i.e., peer's \emph{neighbors}. Basic gnutella |
each peer has a few connections to other peer, i.e. peer's \emph{neighbors}. Basic gnutella |
250 |
data lookup works as follows: peer broadcasts a query request to its neighors, which in turn |
data lookup works as follows: peer broadcasts a query request to its neighors, which in turn |
251 |
forwards the query to their neighbors. This leads in the situation where number of messages |
forwards the query to their neighbors. This leads in the situation where number of messages |
252 |
in the network can grow with $O(n^{2})$, where $n$ is the number of participating peers in the |
in the network can grow with $O(n^{2})$, where $n$ is the number of participating peers in the |
253 |
Gnutella network. To limit the amount of network traffic, Gnutella uses Time-To-Live-limited |
Gnutella network. To limit the amount of network traffic, Gnutella uses Time-To-Live-limited |
254 |
(TTL) flooding to distributed queries. Gnutella uses a breadt-First traversal with depth limit |
(TTL) flooding to distributed queries. Gnutella uses a Breadt-First-Traversal (BFS) with depth limit |
255 |
$T$ (e.g., 7), where T is the system-wide maximum TTL of a message in hops. Therefore, only peers that |
$T$ (e.g., 7), where $T$ is the system-wide maximum TTL of a message in hops. Therefore, only peers that |
256 |
are TTL hops away from the query originator will forward the query or respond to the query. |
are TTL hops away from the query originator will forward the query or respond to the query. |
257 |
In Gnutella network, search results are fast, because breadt-First traversal sends queries to |
In Gnutella network, search results are fast, because BFS sends queries to |
258 |
every possible neighbor. On the other hand, this method wastes resources and doesn't scale well. |
every possible neighbor. Clearly, this method wastes resources and doesn't scale well. |
259 |
Figure \ref{fig:gnutella_query} shows the query lookup process of Gnutella network. |
Figure \ref{fig:gnutella_query} shows the data lookup process of the Gnutella network. |
260 |
|
|
261 |
\begin{figure} |
\begin{figure} |
262 |
\centering |
\centering |
271 |
low, the query originator might not find the desired data even it's available somewhere |
low, the query originator might not find the desired data even it's available somewhere |
272 |
in the network. Second, there are many duplicate messages generated by flooding, especially |
in the network. Second, there are many duplicate messages generated by flooding, especially |
273 |
in high connectivity graphs. It is obvious that with these limitations, flooding creates |
in high connectivity graphs. It is obvious that with these limitations, flooding creates |
274 |
significant message processing overhead for each query. Furthermore, flooding may increase |
significant message processing overhead for each data lookup. Even worse, flooding may increase |
275 |
the load on participating to the point, where it has to leave the network. |
the load on participating to the point, where it has to leave the network. |
276 |
|
|
277 |
Lately, there has been done lot of research to improve Gnutella's data lookup efficiency |
Lately, there has been done lot of research to improve Gnutella's data lookup efficiency |
278 |
and scalability. Adamic et. all \cite{adamic99small}, \cite{adamic02localsearch}, |
and scalability. Adamic et. all \cite{adamic99small}, \cite{adamic02localsearch}, |
279 |
\cite{adamic01powerlawsearch} has been studied different random walk methods in power-law |
\cite{adamic01powerlawsearch} has been studied different random walk methods in power-law |
280 |
networks\footnote{In power-law networks only a few peers have high number of neighbor |
networks\footnote{In power-law networks only a few peers have high number of neighbor |
281 |
links and major of peers have low nuber of neighbor links.} and they have found that by |
links and major of peers have low number of neighbor links.} and they have found that by |
282 |
instructing peers forwarding queries to select high degree peers the data lookup's |
instructing peers forwarding data lookups to select high degree peers, the performance of data lookup |
283 |
performance increases signficantly. As a result, some of the most recent loosely |
increases signficantly. As a result, some of the most recent loosely |
284 |
structured Peer-to-Peer systems have adopted this method with some modifications |
structured Peer-to-Peer systems have adopted this method with some modifications |
285 |
\cite{gnutella2url}, \cite{shareazaurl}, \cite{fasttrackurl}, \cite{morpheusurl}, |
\cite{gnutella2url}, \cite{shareazaurl}, \cite{fasttrackurl}, \cite{morpheusurl}, |
286 |
\cite{kazaaurl}, \cite{jxtaurl}, \cite{jxtaoverview}, \cite{botros01jxtasearch}, |
\cite{kazaaurl}, \cite{jxtaurl}, \cite{jxtaoverview}, \cite{botros01jxtasearch}, |
308 |
|
|
309 |
Previously presented improvements are only partial solutions. Obviously, more |
Previously presented improvements are only partial solutions. Obviously, more |
310 |
research is required to make loosely structured approach's data lookup more |
research is required to make loosely structured approach's data lookup more |
311 |
scalable and effective. |
scalable and effective. More advanced techniques to improve loosely strcutured |
312 |
|
systems' data lookup is presented in chapter 3. |
313 |
|
|
314 |
|
|
315 |
\subsection{Sketch of formal definition} |
\subsection{Sketch of formal definition} |
329 |
|
|
330 |
\section{Tightly structured} |
\section{Tightly structured} |
331 |
|
|
332 |
In recents months, several tightly structured overlays has been proposed. |
Partly due to loosely structured systems' scalability problems, several tightly |
333 |
|
structured overlays has been proposed. |
334 |
This list includes CAN \cite{ratnasamy01can}, Chord \cite{stoica01chord}, |
This list includes CAN \cite{ratnasamy01can}, Chord \cite{stoica01chord}, |
335 |
Kademlia \cite{maymounkov02kademlia}, Kelips \cite{gupta03kelips}, |
Kademlia \cite{maymounkov02kademlia}, Kelips \cite{gupta03kelips}, |
336 |
Koorde \cite{kaashoek03koorde}, ODHDHT \cite{naor03simpledht}, |
Koorde \cite{kaashoek03koorde}, ODHDHT \cite{naor03simpledht}, |
349 |
value of $n$ varies among approaches. Again, CAN uses a $d$-dimensional cartesian |
value of $n$ varies among approaches. Again, CAN uses a $d$-dimensional cartesian |
350 |
to implement identifier space. |
to implement identifier space. |
351 |
|
|
352 |
|
Stoica et al. \cite{balakrishanarticle03lookupp2p} have listed |
353 |
|
four requirements for tightly structured overlays, which have to be |
354 |
|
addressed in order to perform data lookups in tightly structured overlays. |
355 |
|
First, mapping of keys to peers must be done in a load-balanced |
356 |
|
way. Second, the overlay must be able to forward a lookup for a |
357 |
|
specific key to an approriate peer. Third, overlay must have a |
358 |
|
support for a distance function. Finally, routing tables for each peer |
359 |
|
must be constructed and maintained adaptively. |
360 |
|
|
361 |
To store data into tightly structured overlay, each application-specific |
To store data into tightly structured overlay, each application-specific |
362 |
unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent |
unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent |
363 |
hashing \cite{258660}) by the overlay to a existing peer in the overlay. Thus, tightly |
hashing \cite{258660}) by the overlay to a existing peer in the overlay. Thus, tightly |
364 |
structured overlay assigns a subset of all possible keys to every participating peer. |
structured overlay assigns a subset of all possible keys to every participating peer. |
365 |
Furtermore, each peer in the structured overlay maintains a \emph{routing table}, which |
Specifically, each peer in tightly structured overlay maintains a \emph{routing table}, which |
366 |
consists of identifiers and IP addresses of other peers in the overlay. These are peer's |
consists of identifiers and IP addresses of other peers in the overlay. Entries of routing |
367 |
neighbors in the overlay network. Figure \ref{fig:structured_hashing} illustrates the |
table are peer's neighbors in the overlay network. Figure \ref{fig:structured_hashing} illustrates the |
368 |
process of data to key mapping in tightly strucuted overlays. |
process of data to key mapping in tightly strucuted overlays. |
369 |
|
|
370 |
\begin{figure} |
\begin{figure} |
377 |
All messages are routed across overlay links towards peers, whose |
All messages are routed across overlay links towards peers, whose |
378 |
peer identifier is gradually ''closer'' to the key's identifier |
peer identifier is gradually ''closer'' to the key's identifier |
379 |
in the identifier space. Distance can be measured by numerical |
in the identifier space. Distance can be measured by numerical |
380 |
difference between identifiers (e.g., Chord), the number of |
difference between identifiers (e.g., Chord \cite{stoica01chord}), the number of |
381 |
same prefix bits between identifiers (e.g., Pastry and Tapestry), |
same prefix bits between identifiers (e.g., Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry}), |
382 |
bit-wise exclusive or (XOR) (e.g., Kademlia). However, in all |
bit-wise exclusive or (XOR) (e.g., Kademlia \cite{maymounkov02kademlia}). However, in all |
383 |
previously schemes, each hop in the overlay shortens the path |
previously schemes each hop in the overlay shortens the path |
384 |
between current peer working with query and the key which was |
between current peer working with query and the key which was |
385 |
looked up. |
looked up. |
386 |
|
|
387 |
Skip Graphs and Swan employ a key space very similar to a tightly structured |
Skip Graphs and Swan employ a key space very similar to a tightly structured |
388 |
overlay, but in which queries are routed to \emph{identifiers}. In these systems |
overlay, but in which queries are routed to \emph{keys}. In these systems |
389 |
peer occupies several positions in the identifier space, one for each |
peer occupies several positions in the identifier space, one for each |
390 |
application-specific key. The indirection of placing close keys in the |
application-specific key. The indirection of placing close keys in the |
391 |
custody of a storing peer\footnote{Storing peer is the peer in the overlay which stores the |
custody of a storing peer\footnote{Storing peer is the peer in the overlay which stores the |
392 |
assigned keys.} keys is removed at the cost of each peer maintaining one |
assigned keys.} keys is removed at the cost of each peer maintaining one |
393 |
''resource node'' in the overlay network for each resource item pair it publishes. |
''resource peer'' in the overlay network for each resource item pair it publishes. |
394 |
|
|
395 |
PeerNet differs from other tightly structured overlays in that it operates |
PeerNet differs from other tightly structured overlays in that it operates |
396 |
at the \emph{network} level layer. Peernet makes an explicit distinction |
at the \emph{network} level layer. Peernet makes an explicit distinction |
397 |
between peer identity and address, which is supported by standard |
between peer identity and address, which is not supported by standard |
398 |
TCP/IP-algorithms. Otherwise, PeerNet has same performance properties |
TCP/IP-protocols. Otherwise, PeerNet has the same performance properties |
399 |
as other tightly structured overlays, i.e. $O(\log{n})$ space required |
as other tightly structured overlays, i.e. $O(\log{n})$ space required |
400 |
for maintaining information about other peers in the system and |
for maintaining information about other peers in the system and |
401 |
$O(\log{n})$ data lookup efficieny. |
$O(\log{n})$ data lookup efficiency. |
|
|
|
|
Stoica et al. \cite{balakrishanarticle03lookupp2p} have listed |
|
|
four requirements for tightly structured overlays, which have to be |
|
|
addressed in order to perform data lookups in tightly structured overlays. |
|
|
First, mapping of keys to peers must be done in a load-balanced |
|
|
way. Second, the overlay must be able to forward a lookup for a |
|
|
specific key to an approriate peer. Third, overlay must have a |
|
|
support for a distance function. Finally, routing tables for each peer |
|
|
must be constructed and maintained adaptively. |
|
402 |
|
|
403 |
Currently, all proposed tightly structured overlays provide at least |
Currently, all proposed tightly structured overlays provide at least |
404 |
poly--logaritmical data lookup operations. However, there are some key |
poly--logaritmical data lookup operations. However, there are some key |
405 |
differences in the data structure that they use as a routing table. For example, Chord, Skip graphs and |
differences in the data structure that they use as a routing table. For example, Chord |
406 |
Skipnet maintain a local data structure which resembles skip lists \cite{78977}. |
\cite{stoica01chord}, Skip graphs \cite{AspnesS2003} and Skipnet \cite{harvey03skipnet2} maintain a local |
407 |
|
data structure which resembles Skip lists \cite{78977}. |
408 |
In figure \ref{fig:structured_query}, we present overview of Chord's lookup process. |
In figure \ref{fig:structured_query}, we present overview of Chord's lookup process. |
409 |
On the left side of Chord's lookup process, we show the same data lookup process |
On the left side of Chord's lookup process, we show the same data lookup process |
410 |
as binary-tree abstraction. We can notice, that in each step, the distance between |
as binary-tree abstraction. We can notice, that in each step, the distance between |
411 |
the query originator and the target in both methods is halved. Thus, the |
the query originator and the target in both methods is halved. Thus, the |
412 |
locarithmic efficiency. |
locarithmic efficiency. |
413 |
|
|
414 |
Kademlia, Pastry and Tapestry uses balanced $k$-trees |
Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry |
415 |
as routing table's data structure. Figure \ref{fig:kademlia_lookup} shows the process of Kademlia |
\cite{zhao01tapestry} uses balanced $k$-trees as routing table's data structure. Figure |
416 |
data lookup. Viceroy maintains a butterfly data structure (see e.g., \cite{226658}), |
\ref{fig:kademlia_lookup} shows the process of Kademlia |
417 |
|
data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}), |
418 |
which requires only constant number of neighbor peers while providing $O(\log{n})$ data lookup |
which requires only constant number of neighbor peers while providing $O(\log{n})$ data lookup |
419 |
efficiency. Koorde, recent modification of Chord, uses de Bruijn graphs to maintain |
efficiency. Koorde \cite{kaashoek03koorde}, recent modification of Chord, uses de Bruijn graphs |
420 |
local routing tables. Koorde requires each peer to have only about two links to other |
\cite{debruijn46graph} to maintain local routing tables. Koorde requires each peer to have only |
421 |
peers to to provide $O(\log{n})$ performance. |
about two links to other peers to to provide $O(\log{n})$ performance. |
422 |
|
|
423 |
\begin{figure} |
\begin{figure} |
424 |
\centering |
\centering |
453 |
to nearest available peer, hosting a specific data item. This form of locality |
to nearest available peer, hosting a specific data item. This form of locality |
454 |
is not supported by DHT. Finally, tightly structured overlay can be used for |
is not supported by DHT. Finally, tightly structured overlay can be used for |
455 |
scalable group multicast/anycast operations (CAST) (see e.g., \cite{zhuang01bayeux}). |
scalable group multicast/anycast operations (CAST) (see e.g., \cite{zhuang01bayeux}). |
456 |
The basic operation are \texttt{join(groupIdentifier)}, \texttt{leave(groupIdentifier)}, |
The basic operations are \texttt{join(groupIdentifier)}, \texttt{leave(groupIdentifier)}, |
457 |
\texttt{multicast(message, groupIdentifier)}, \texttt{anycast(message, groupIdentifier)}. |
\texttt{multicast(message, groupIdentifier)}, \texttt{anycast(message, groupIdentifier)}. |
458 |
Participating peers may join and leave a group and send multicast messages to |
Participating peers may join and leave the group and send multicast messages to |
459 |
the group, or anycast message to a specific member of the group. DOLR's and CAST's |
the group, or anycast message to a specific member of the group. DOLR's and CAST's |
460 |
have much in common.For instance, they both use network proximity techniques |
have much in common.For instance, they both use network proximity techniques |
461 |
to optimize their operation in the overlay. Figure \ref{fig:Strucutred_lookup_using_DOLR_model} |
to optimize their operation in the overlay. Figure \ref{fig:Strucutred_lookup_using_DOLR_model} |
493 |
$ip = map(identifier(s))$, which maps data items, expressed by a identifier to coordinate |
$ip = map(identifier(s))$, which maps data items, expressed by a identifier to coordinate |
494 |
point $ip$ in $(IS,d)$. Peer's p resources are mapped onto a set $IS$ = \{$ip \in IS: |
point $ip$ in $(IS,d)$. Peer's p resources are mapped onto a set $IS$ = \{$ip \in IS: |
495 |
\exists s \in S$, $ip = map(identifier(s)) \wedge (provider(s) = p)$\}., |
\exists s \in S$, $ip = map(identifier(s)) \wedge (provider(s) = p)$\}., |
496 |
which means that resources that a peer provides into the system are not kept locally. |
which means that resources which peer provides into the system are not kept locally. |
497 |
Every $p$ has neighbor(s), named as $neighbor$, which are $P$ = \{$p \in P: \exists neighbor$, |
Every $p$ has neighbor(s), named as $neighbor$, which are $P$ = \{$p \in P: \exists neighbor$, |
498 |
where $difference(p,p_neighbor)= close$, and $close$ is minimal difference $d$ in $(IS,d)$\}. |
where $difference(p,p_neighbor)= ''close''$, where $''close''$ is small difference $d$ in $(IS,d)$\}. |
499 |
|
|
500 |
|
|
501 |
\section{Summary} |
\section{Summary} |
516 |
|
|
517 |
Thus, there are significant differences between loosely structured and tightly structured approaches. |
Thus, there are significant differences between loosely structured and tightly structured approaches. |
518 |
The most important aspect is the performance and scalability. While loosely structured approach's performance |
The most important aspect is the performance and scalability. While loosely structured approach's performance |
519 |
is not always even linear, generally tightly structured approach can perform all operations in |
is not always even linear, generally tightly structured approach can perform all internal operations in |
520 |
$\Theta(\log{n})$\footnote{However, it is unknown whether all proposed algorithms can preserve |
poly-logarithmic time\footnote{However, it is unknown whether all proposed algorithms can preserve |
521 |
logarithmic properties in real-life applications or not.}. |
logarithmic properties in real-life applications or not.}. |
522 |
|
|
523 |
Another key point is the philosophy how overlay network is constructed and maintained. While loosely |
Another key point is the philosophy how overlay network is constructed and maintained. While loosely |
525 |
structured approach has certain features, in which participating peers have no control at all |
structured approach has certain features, in which participating peers have no control at all |
526 |
(such as mapping of data items). |
(such as mapping of data items). |
527 |
|
|
528 |
To end user, biggest difference between these systems is how data lookups are performed. Looselely |
To end user, biggest difference between these systems is how data lookups are performed. Loosely |
529 |
structured systems provides much more richier and user friendly way of searching data as they |
structured systems provides much more richier and user friendly way of searching data as they |
530 |
have support for keyword search and fuzzy search. On the other, tightly structured systems support |
have support for keyword search and fuzzy search. On the other hand, tightly structured systems support |
531 |
only exact key lookups as each data item is identified by unique keys. |
only exact key lookups as each data item is identified by globally unique keys. |
532 |
|
|
533 |
In the end, both systems have open problems and issues. We will discuss these aspects more detail in |
In the end, both systems have open problems and issues. We will discuss these aspects more detail in |
534 |
chapter 3. Table \ref{table_comparison_approach} lists key differences between loosely structured |
chapter 3. Table \ref{table_comparison_approach} lists key differences between loosely structured |
642 |
and their key properties with regard to performance and scalability. List |
and their key properties with regard to performance and scalability. List |
643 |
includes algorithms from two main approaches. However, majority of the algorithms |
includes algorithms from two main approaches. However, majority of the algorithms |
644 |
listed above belongs to tightly structured approach since there has been active |
listed above belongs to tightly structured approach since there has been active |
645 |
research being pursued lately. List doesn't include \emph{all} proposed Peer-to-Peer |
research being pursued towards tightly structured approach lately. List doesn't |
646 |
systems, only the ones which already have been widely deployed in real-life, or |
include \emph{all} proposed Peer-to-Peer systems. Only the ones which already have |
647 |
the ones which may promising in the future's Peer-to-Peer systems. |
been widely deployed in real-life, or the ones which may promising in the future's |
648 |
|
Peer-to-Peer systems, are included in this thesis. |
649 |
|
|
650 |
We decided to follow the guidelines from \cite{kaashoek03koorde} when |
We decided to follow the guidelines from \cite{kaashoek03koorde} when |
651 |
measuring properties of different Peer-to-Peer systems. However, we dropped |
measuring properties of different Peer-to-Peer systems. However, we dropped |
652 |
out fault tolerance and load balancing properties, since they are hard to measure |
out fault tolerance and load balancing properties, since they are hard to measure |
653 |
in face of real life requirements. Additionally, we decided to include |
in face of real life requirements. Additionally, we decided to include |
654 |
the number of real network connections for each peer in the overlay. |
the number of \emph{real} network connections for each peer in the overlay. |
655 |
|
|
656 |
Here, we describe the listed properties of Peer-to-Peer algorihms: |
Here, we describe the listed properties of Peer-to-Peer algorihms: |
657 |
|
|
659 |
\item \textbf{Lookup}: Number of messages required when a data lookup is performed |
\item \textbf{Lookup}: Number of messages required when a data lookup is performed |
660 |
\item \textbf{Space}: Number of neighbors which peers knows about (neighbors) |
\item \textbf{Space}: Number of neighbors which peers knows about (neighbors) |
661 |
\item \textbf{Insert/delete}: Number of messages required when a peer joins or leaves the network |
\item \textbf{Insert/delete}: Number of messages required when a peer joins or leaves the network |
662 |
\item \textbf{Number of network connections}: Number of concurrent \emph{network} connections required to maintain correct neighbor information |
\item \textbf{Number of network connections}: Number of concurrent network connections required to maintain correct neighbor information |
663 |
\end{itemize} |
\end{itemize} |
664 |
|
|
665 |
\scriptsize |
\scriptsize |
857 |
tables; we list description of the problem, solution and comments on that |
tables; we list description of the problem, solution and comments on that |
858 |
specific open problem. Note that open problems list considered here is not meant |
specific open problem. Note that open problems list considered here is not meant |
859 |
to be an exhaustive survey of \emph{all} open problems in Peer-to-Peer domain; |
to be an exhaustive survey of \emph{all} open problems in Peer-to-Peer domain; |
860 |
we focus our attention to security, scalability and performance related issues |
we focus our attention to security, scalability, usability and performance related issues |
861 |
only. |
only. |
862 |
|
|
863 |
\section{Overview} |
\section{Overview} |
864 |
|
|
865 |
Partly due to the non-maturity of modern Peer-to-Peer technology, it has several |
Partly due to the non-maturity of modern Peer-to-Peer technology, it has several |
866 |
open problems to be solved. Main open problems are related to performance, scalability |
open problems to be solved. Main open problems are related to performance, scalability, usability |
867 |
and security. More important, many techniques developed for traditional distributed |
and security. More important, many techniques developed for traditional distributed |
868 |
systems may no longer apply with Peer-to-Peer systems. Therefore, new solutions are |
systems may no longer apply with Peer-to-Peer systems. Therefore, new solutions are |
869 |
needed to make Peer-to-Peer systems more secure and efficient. |
needed to make Peer-to-Peer systems more secure and efficient. |
872 |
Since Napster \cite{napsterurl} and Gnutella \cite{gnutellaurl} was first time introduced |
Since Napster \cite{napsterurl} and Gnutella \cite{gnutellaurl} was first time introduced |
873 |
to public, researchers' main concern has been scalability problem of loosely structured |
to public, researchers' main concern has been scalability problem of loosely structured |
874 |
approach. However, people often misunderstand the scalability problem of loosely structured |
approach. However, people often misunderstand the scalability problem of loosely structured |
875 |
approach; loosely structured approache's \emph{network} is scalable, but the \emph{query model} is not |
approach; loosely structured systems' \emph{network} is scalable, but the \emph{query model} is not. |
876 |
scalable. Tightly structured approach's main concern is to make overlay's data lookup |
Tightly structured system's main concern is to make overlay's data lookup |
877 |
routing more flexible againts hostile attacks. Another key problems in tightly structured |
routing more flexible againts hostile attacks. Another key problems in tightly structured |
878 |
approach are the lack of keyword searches and support for heterogeneous peers. |
approach are the lack of keyword searches and support for heterogeneous peers. |
879 |
|
|
894 |
general Distrubuted Denial of Service attack. |
general Distrubuted Denial of Service attack. |
895 |
|
|
896 |
In Sybil attack model, hostile entity presents multpile |
In Sybil attack model, hostile entity presents multpile |
897 |
entities. Therefore, one hostile entity can control a large fraction of the Peer-to-Peer system. The best |
entities. Therefore, one hostile entity can control a large fraction of the Peer-to-Peer system. Optimal |
898 |
possible solution to Sybil attack would be that system could \emph{distinct} entities reliably. Unfortunately, |
possible solution to Sybil attack would be that system could \emph{distinct} system's entities reliably. Unfortunately, |
899 |
currently there no realizable techiques for this task. Partial solutions for Sybil is attack is to replicate |
currently there no realizable techiques for this task. Partial solutions for Sybil is attack is to replicate |
900 |
and fragment data randomly among several participating peer. However, both suggestions assume that two different |
and fragment data randomly among several participating peer. However, both suggestions assume that two different |
901 |
remote entities are actually different; Sybil attacks are still possible and therefore, would need centralized |
remote entities are actually different; Sybil attacks are still possible and therefore, would need centralized |
908 |
troijan. Closey related to fail-stop model is the Byzantine attack model |
troijan. Closey related to fail-stop model is the Byzantine attack model |
909 |
\cite{357176}. Byzantine model can been seen more seveve than fail-stop model as there are no restrictions over |
\cite{357176}. Byzantine model can been seen more seveve than fail-stop model as there are no restrictions over |
910 |
the behaviour of faulty peers. Partial, practical solution for byzantine failures has been proposed by Castro et |
the behaviour of faulty peers. Partial, practical solution for byzantine failures has been proposed by Castro et |
911 |
al \cite{296824}. General robustness properties of Peer-to-Peer system is able to deal with software failures and hostile |
al \cite{296824}. |
|
attack, but redundancy againts external threats is unknown. The reason for this is that there are no experiences |
|
|
on these kinds of attacks. Possible solution would be distributed anti-virus software, but much more intensive |
|
|
research is required for solve these problems. |
|
912 |
|
|
913 |
Spam generating attack is another known attack model againts Peer-to-Peer system. In Spam |
Spam generating attack is another known attack model againts Peer-to-Peer system. In Spam |
914 |
attack, hostile or faulty peer may produce false information of the data. Possible solution againts this attack |
attack, hostile or faulty peer may produce false information of the data. Possible solution againts this attack |
928 |
|
|
929 |
\subsection{Trust, data authenticity and integrity} |
\subsection{Trust, data authenticity and integrity} |
930 |
|
|
931 |
Currently, trust in Peer-to-Peer systems is based on \emph{reputation}. Current repuation methods focus either |
Trust in Peer-to-Peer systems is based on \emph{reputation}. Proposed repuation methods focus either |
932 |
on the semantic properties, or data management properties of the trust model. Some research has been |
on the semantic properties, or data management properties of the trust model. Some research has been |
933 |
done on reputation models in Peer-to-Peer systems, such as \cite{aberer01trust}, \cite{cornelli02reputableservents}. |
done on reputation models in Peer-to-Peer systems, such as \cite{aberer01trust}, \cite{cornelli02reputableservents}. |
934 |
Implementations include Advogato \cite{advogatourl}. None of the current proposals or implementations |
Implementations include Advogato \cite{advogatourl}. None of the current proposals or implementations |
945 |
|
|
946 |
ConChord \cite{ajmani02conchord} is the first Peer-to-Peer system which has a support for PKI based |
ConChord \cite{ajmani02conchord} is the first Peer-to-Peer system which has a support for PKI based |
947 |
security infrastructure. Unfortunately, ConChord is in early in development and lacks of important |
security infrastructure. Unfortunately, ConChord is in early in development and lacks of important |
948 |
features of PKI to be fully usable yet. Furthermore, the hierarchy of SDSI/SPKI may a problem for |
features of PKI to be fully usable yet. Furthermore, the hierarchy of SDSI/SPKI \cite{rivest96sdsi}, |
949 |
Peer-to-Peer systems, in which hierarchy is intentionally missing. |
\cite{spkiworkinggroup} may a problem for Peer-to-Peer systems, in which hierarchy is intentionally missing. |
950 |
|
|
951 |
For data integrity, on the other hand, there are few working solutions. Cryptographic content hashes, |
For data integrity, on the other hand, there are few working solutions. Cryptographic content hashes |
952 |
such as \cite{fips-sha-1}, variations \cite{merkle87hashtree} and their implementation techniques \cite{mohr02thex}, |
\cite{fips-sha-1}, variations \cite{merkle87hashtree} and their implementation techniques \cite{mohr02thex}, |
953 |
are efficient and reliable methods for identifying the integrity of data in Peer-to-Peer systems. One |
are efficient and reliable method for identifying the integrity of data in Peer-to-Peer systems. One |
954 |
possible application of cryptographic content hashes may in peer identifier creation process, in which |
possible application of cryptographic content hashes may in peer identifier creation process, in which |
955 |
IP address of peer can be verified by the other peer. This is one form of \emph{self-certifying data}. |
IP address of peer can be verified by the other peer. This is one form of \emph{self-certifying data}. |
956 |
|
|
976 |
Peer-to-Peer system. Let's consider anonymity and efficient data lookup. In efficient lookup, we must know |
Peer-to-Peer system. Let's consider anonymity and efficient data lookup. In efficient lookup, we must know |
977 |
the peers responsible to given data in Peer-to-Peer system. Of course, when we know the peers responsible |
the peers responsible to given data in Peer-to-Peer system. Of course, when we know the peers responsible |
978 |
for the data, the anonymity of peer is lost. Fortunately, there are partial solutions to previously |
for the data, the anonymity of peer is lost. Fortunately, there are partial solutions to previously |
979 |
mentioned situations, i.e. pseudonym which is a partial form of anonymity. For instance, pseudonym can used for |
mentioned situations, i.e. \emph{pseudonym} which is a partial form of anonymity. For instance, pseudonym can used for |
980 |
addressing peer-anonymity by providing anonymous-like identifiers to peers (e.g., tightly structured peer |
addressing peer-anonymity by providing anonymous-like identifiers to peers (e.g., tightly structured system's |
981 |
identifiers). |
peer identifiers). |
982 |
|
|
983 |
Anonymity is widely used in those Peer-to-Peer system in which data publication and non-censorship are important properties |
Anonymity is widely used in those Peer-to-Peer system in which data publication and non-censorship are important properties |
984 |
of the system. These include |
of the system. These include |
986 |
Tangler \cite{502002} and upcoming Mnet \cite{mneturl}. Forwarding proxies are used in Freenet, Crowds and |
Tangler \cite{502002} and upcoming Mnet \cite{mneturl}. Forwarding proxies are used in Freenet, Crowds and |
987 |
Free Haven in order to provide various types of anonymity. Tangler and Publius uses cryptographic |
Free Haven in order to provide various types of anonymity. Tangler and Publius uses cryptographic |
988 |
sharing methods to split a data into data fragments \cite{Shamir1979a}. Mixmailer networks, such as |
sharing methods to split a data into data fragments \cite{Shamir1979a}. Mixmailer networks, such as |
989 |
\cite{mixminionurl}, are commonly used in distributed systems, which are able to provide level |
\cite{mixminionurl}, are commonly used in distributed systems, which are able to provide some level |
990 |
of anonymity |
of anonymity |
991 |
|
|
992 |
Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no |
Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no |
994 |
between anonymity and other Peer-to-Peer system properties requires more research work. |
between anonymity and other Peer-to-Peer system properties requires more research work. |
995 |
|
|
996 |
|
|
997 |
\subsection{Access Control} |
\subsection{Access control} |
998 |
|
|
999 |
Any distributed computing system must support different levels of access control. For instance, we may |
Any distributed computing system must support different levels of access control. For instance, we may |
1000 |
want to restrict the accessibility of data to only limited amount of participating peers. Currently, |
want to restrict the accessibility of data to only limited amount of participating peers. Peer-to-Peer |
1001 |
Peer-to-Peer systems doesn't support working, trusted and distributed access control scheme. Moreover, |
systems doesn't support working and distributed access control scheme. Moreover, |
1002 |
there has been a lot of violation of copyright laws by users of Peer-to-Peer filesharing systems. As a consequence, some |
there has been a lot of violation of copyright laws by users of Peer-to-Peer filesharing systems. As a |
1003 |
lawsuits has been created againts the companies how have build popular file-sharing programs. |
consequence, some lawsuits has been created againts the companies how have build popular file-sharing programs. |
1004 |
|
|
1005 |
To our knowledge, Nejdl et al \cite{nejdl03accesscontrol} have proposed first practical solution to access |
To our knowledge, Nejdl et al \cite{nejdl03accesscontrol} have proposed very recently first practical solution to access |
1006 |
control problem in Peer-to-Peer systems. They use RDF-based schema policies to restrict access to certain |
control problem in Peer-to-Peer systems. They use RDF-based schema policies to restrict access to certain |
1007 |
data. To be distributed system feasible, there must be way of control. Unfortunately, their solution |
data. Unfortunately, their current prototype works only in loosely structured systems. |
|
works only in loosely structured systems. |
|
1008 |
|
|
1009 |
|
|
1010 |
\subsection{Hostile entities} |
\subsection{Hostile entities} |
1013 |
Possible solutions include self-monitoring systems \cite{zhang03somo}, maintaining system invariants as |
Possible solutions include self-monitoring systems \cite{zhang03somo}, maintaining system invariants as |
1014 |
proposed in \cite{sit02securitycons}, distributed and secure peer identifier assignment |
proposed in \cite{sit02securitycons}, distributed and secure peer identifier assignment |
1015 |
\cite{castro02securerouting}, \cite{clarke00freenet} and self-certifying data using cryptographic |
\cite{castro02securerouting}, \cite{clarke00freenet} and self-certifying data using cryptographic |
1016 |
content hashes (e.g., SHA-1). Identification of hostile entities is essential in tightly structured |
content hashes (e.g., SHA-1 \cite{fips-sha-1}). Identification of hostile entities is essential in tightly structured |
1017 |
approach, in which fundamental (and implicit) assumption is that there is random, uniform distribution |
approach, in which fundamental (and implicit) assumption is that there is random, uniform distribution |
1018 |
of peer identifiers that cannot be controlled by hostile entity. |
of peer identifiers that cannot be controlled by hostile entity. |
1019 |
|
|
1020 |
Of course centralized authorities could be used for assignment of peer identifiers, but they have |
Of course centralized authorities could be used for assignment of peer identifiers, but they have |
1021 |
property of single point of failure. Furthermore, distributed peer identification assignment can |
property of single point of failure. Furthermore, distributed peer identification assignment can |
1022 |
be problematic as long as Sybil attack remains unsolved. However, there are some partial solutions |
be problematic as long as Sybil attack \cite{douceur02sybil} remains unsolved. However, there are some partial solutions |
1023 |
for controlling the rate at which and hostile entity is able to obtain peer identifier, such as crypto-based |
for controlling the rate at which and hostile entity is able to obtain peer identifier, such as crypto-based |
1024 |
puzzles \cite{juels99clientpuzzles}. |
puzzles \cite{juels99clientpuzzles}. |
1025 |
|
|
1027 |
efficient way. More research is required to solve this problems. |
efficient way. More research is required to solve this problems. |
1028 |
|
|
1029 |
|
|
1030 |
\subsection{Secure Query Routing} |
\subsection{Secure query routing} |
1031 |
|
|
1032 |
Much work has been done on secure routing, especially in tightly structured systems. In |
Much work has been done on secure routing, especially in tightly structured systems. In |
1033 |
\cite{castro02securitystructured} and \cite{castro02securerouting}, authors suggests the usage |
\cite{castro02securitystructured} and \cite{castro02securerouting}, authors suggests the usage |
1037 |
correct peers, when a fraction $f$ of the other peers are faulty or hostile, is only $(1-f)^{h-1}$. |
correct peers, when a fraction $f$ of the other peers are faulty or hostile, is only $(1-f)^{h-1}$. |
1038 |
|
|
1039 |
Sit and Morris \cite{sit02securitycons} discuss the possibility of allowing query originator |
Sit and Morris \cite{sit02securitycons} discuss the possibility of allowing query originator |
1040 |
to observe lookup progress and cross-check routing tables using random queries. However, Sit and |
to observe lookup progress and cross-check routing tables using random queries. However, their |
1041 |
Morris approach is not very efficient, since proposals create a lot of additional network traffic when |
approach is not very efficient, since proposals create a lot of additional network traffic when |
1042 |
in function. |
in function. |
1043 |
|
|
1044 |
Additionally, Lynch et al. \cite{lynch02atomicdataaccess} propose a solution to secure routing table |
Additionally, Lynch et al. \cite{lynch02atomicdataaccess} propose a solution to secure routing table |
1049 |
|
|
1050 |
Aspnes et al in \cite{aspnes02faultrouting} and Kaashoek et all in \cite{kaashoek03koorde} formally |
Aspnes et al in \cite{aspnes02faultrouting} and Kaashoek et all in \cite{kaashoek03koorde} formally |
1051 |
prove the lower and upper bounds for space requirements of locating a specific date item in |
prove the lower and upper bounds for space requirements of locating a specific date item in |
1052 |
Peer-to-Peer system. They show that to provide high degree of fault tolerance efficiency, a peer |
Peer-to-Peer system. They show that to provide high degree of fault tolerance and efficiency, each |
1053 |
must maintain $O(\log{n})$ neighbors. In addition, most existing |
participating peer must maintain $O(\log{n})$ neighbors. |
1054 |
|
|
1055 |
Fiat et al in \cite{fiat02censorship}, \cite{saia02dynamicfaultcontentnetwork} and Datar in \cite{datar02butterflies} |
Fiat et al in \cite{fiat02censorship}, \cite{saia02dynamicfaultcontentnetwork} and Datar in \cite{datar02butterflies} |
1056 |
describe tightly structured overlay with analytical results in the presence of hostile entities. However, |
describe tightly structured overlay with analytical results in the presence of hostile entities. However, |
1057 |
none of these proposals doesn't address an efficient, dynamic tightly structured overlay and multiple rounds |
none of these proposals doesn't address an efficient, dynamic tightly structured overlay and multiple rounds |
1058 |
of hostile attack. Indeed, above mentioned solutions are not very efficient. In Fiat et al, each node |
of hostile attack. Also, above mentioned propsals are not very efficient. In \cite{fiat02censorship}, each node |
1059 |
must maintain information of $O(\log^3{n})$ other peers, and in Datar $O(\log^3{n})$ is required. |
must maintain information of $O(\log^3{n})$ other peers, and in \cite{datar02butterflies}, $O(\log^2{n})$ is required. |
1060 |
|
|
1061 |
Finally, Ratnasamy and Gavoille \cite{ratnasamy02routing}, \cite{gavoille01routing} list several open problems |
Finally, Ratnasamy and Gavoille \cite{ratnasamy02routing}, \cite{gavoille01routing} list several open problems |
1062 |
regarding routing in distributed networks. Obviously, more research is required for make secure |
regarding routing in distributed networks. Obviously, more research is required in order to provde secure |
1063 |
routing possible in Peer-to-Peer networks. |
data lookup routing possible in Peer-to-Peer networks. |
1064 |
|
|
1065 |
|
|
1066 |
\subsection{Other Security threats} |
\subsection{Other security threats} |
1067 |
|
|
1068 |
Ross Lee graham lists several external threats againts Peer-to-Peer networks \cite{grahamp2psecurity}. The list |
Ross Lee graham lists several external threats againts Peer-to-Peer networks \cite{grahamp2psecurity}. The list |
1069 |
includes viruses, trojans and bugs in Peer-to-Peer software. Currently, there are not even partial solutions |
includes viruses, trojans and bugs in Peer-to-Peer software. Currently, there are not even partial solutions |
1070 |
to the problems mentioned above. |
to the problems mentioned above. General robustness properties of Peer-to-Peer system is able to |
1071 |
|
deal with software failures and hostile attack, but redundancy againts external threats is unknown. |
1072 |
|
The reason for this is that there are no experiences on these kinds of attacks. Possible solution |
1073 |
|
would be distributed anti-virus software, but much more intensive research is required for solve these problems. |
1074 |
|
|
1075 |
|
|
1076 |
|
|
1077 |
|
|
1078 |
\section{Performance and usability problems in Peer-to-Peer} |
\section{Performance and usability problems in Peer-to-Peer} |
1095 |
is similar to iterative deepening techique. With these techniques, search |
is similar to iterative deepening techique. With these techniques, search |
1096 |
may not be fast when desired data item requires many consecutive flooding rounds. |
may not be fast when desired data item requires many consecutive flooding rounds. |
1097 |
|
|
1098 |
Directed breadt-first search \cite{yang02improvingsearch} optimizes the original |
Directed BFS \cite{yang02improvingsearch} optimizes the original |
1099 |
breadt-first search in way that peer selects neighbors with many quality results |
BFS in way that peer selects neighbors with many quality results |
1100 |
may be reached, thereby maintaining the the quality of costs and decreasing the amount |
may be reached, thereby maintaining the the quality of costs and decreasing the amount |
1101 |
of messages sent to network. Alpine \cite{alpineurl} and NeuroGrid \cite{joseph02neurogrid} |
of messages sent to network. Alpine \cite{alpineurl} and NeuroGrid \cite{joseph02neurogrid} |
1102 |
Peer-to-Peer system use somewhat similar method when performing data lookups. |
are Peer-to-Peer system use somewhat similar method when performing data lookups. |
1103 |
|
|
1104 |
Local indices \cite{yang02improvingsearch} in one variation of active caching. |
Local indices \cite{yang02improvingsearch} is one variation of active caching. |
1105 |
In this scheme, each peer maintains an index over the data of all nodes within |
In this scheme, each peer maintains an index over the data of all nodes within |
1106 |
$h$ hops of itself, where $h$ is a system-wide variable, called radius of the |
$h$ hops of itself, where $h$ is a system-wide variable, called radius of the |
1107 |
index\footnote{In normal BFS case, the value of $h$ is 0, as peer only has index |
index\footnote{In normal BFS case, the value of $h$ is 0, as peer only has index |
1123 |
|
|
1124 |
Since tightly structured systems have efficient data lookup at the application level overlay, |
Since tightly structured systems have efficient data lookup at the application level overlay, |
1125 |
current research efforts are focused on proximity based data lookup. In proximity based data lookup, |
current research efforts are focused on proximity based data lookup. In proximity based data lookup, |
1126 |
peers try to choose routing-tables refering to other peers that are \emph{nearby} in the |
peers try to choose routing-tables entries refering to other peers that are \emph{nearby} in the |
1127 |
underlying network. In this way, tightly structured systems are able to decrease actual |
underlying network. In this way, tightly structured systems are able to decrease actual |
1128 |
lookup \emph{latency}. CAN, Kademlia, Pastry and Tapestry have a advanced heuristics for |
lookup \emph{latency}. CAN \cite{ratnasamy01can}, Kademlia \cite{maymounkov02kademlia}, |
1129 |
|
Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry} have a advanced heuristics for |
1130 |
proximity based routing. Additionally, most recent version of Chord uses proximity based |
proximity based routing. Additionally, most recent version of Chord uses proximity based |
1131 |
routing inspired by Karger and Ruhl \cite{karger02findingnearest}. Skipnet \cite{harvey03skipnet1} |
routing inspired by Karger and Ruhl \cite{karger02findingnearest}. Skipnet \cite{harvey03skipnet1} |
1132 |
uses combination of proximity and application level overlay routing when performing data |
uses combination of proximity and application level overlay routing when performing data |
1140 |
|
|
1141 |
\subsection{Fast and usable search} |
\subsection{Fast and usable search} |
1142 |
|
|
1143 |
To make Peer-to-Peer systems usable in a large, these systems have to support flexible, efficient |
To make Peer-to-Peer systems even more popular (and usable), these systems have to support flexible, efficient |
1144 |
and easy to use search methods. For instance, Internet's perhaps the most important feature |
and easy to use search methods. For instance, Internet's perhaps the most important feature |
1145 |
is the ability to perform keyword or fuzzy searches (e.g., Google). Currently, only loosely |
is the ability to perform keyword or fuzzy searches (e.g., Google). Currently, only loosely |
1146 |
structured systems are able carry out this requirement. Unfortunately, as discussed in this text, |
structured systems are able carry out this requirement. Unfortunately, as discussed in this text, |
1147 |
the data loouk model of loosely structured approach is not scalable. Thus, research efforts have |
the data lookup model of loosely structured approach is not scalable. Thus, research efforts have |
1148 |
been focused on tightly structured approach. |
been focused on tightly structured approach. |
1149 |
The main in problem with tightly structured approach is the fact that tightly structured algorihms |
The main in problem with tightly structured approach is the fact that tightly structured algorihms |
1150 |
performs data lookups based on a unique identifier. However, quite recently have been studies |
performs data lookups based on a globally unique identifier. Quite recent study has been focused |
1151 |
on the feasibility of Peer-to-Peer Web-like indexing and searching \cite{li03feasibility}. Authors |
on the feasibility of Peer-to-Peer Web-like indexing and searching \cite{li03feasibility}. Authors |
1152 |
argue, that it is possible to implement Peer-to-Peer Web-like search with certain radical compromises. |
argue, that it is possible to implement Peer-to-Peer Web-like search with certain radical compromises. |
1153 |
First, Peer-to-Peer search enginge may need to decrease result quality in order make searching more |
First, Peer-to-Peer search enginge may need to decrease result quality in order make searching more |
1154 |
efficient. Second, Peer-to-Peer systems must observe better the properties of underlying network for |
efficient. Second, Peer-to-Peer systems must observe better the properties of underlying network for |
1155 |
better performance. |
better performance. |
1156 |
|
|
1157 |
Some studies have been concentraded on SQL-like queries \cite{harren02complex}, |
Some studies have been concentraded on SQL-like queries \cite{harren02complex} |
1158 |
in tightly structured overlays. Another approaches include adapting loosely structured approache's |
in tightly structured overlays. Another approaches includes adapting loosely structured approache's |
1159 |
data lookup model into tightly structured systems \cite{ansaryefficientbroadcast03}, \cite{chord:om_p-meng}. |
data lookup model into tightly structured systems \cite{ansaryefficientbroadcast03}, \cite{chord:om_p-meng}. |
1160 |
Additional studies include additional layer upon overlay network \cite{kronfol02fasdsearch}, |
Additional studies include additional layer upon overlay network \cite{kronfol02fasdsearch}, |
1161 |
\cite{joseph02p2players} and range queries \cite{andrzejak02rangequeries}. |
\cite{joseph02p2players} and range queries \cite{andrzejak02rangequeries}. |
1164 |
studies queries follow Zipf-like distributions \cite{breslau98implications} caching and precomputation |
studies queries follow Zipf-like distributions \cite{breslau98implications} caching and precomputation |
1165 |
can be done for optimizting search indices \cite{li03feasibility}. Regular compression algorithms, |
can be done for optimizting search indices \cite{li03feasibility}. Regular compression algorithms, |
1166 |
Bloom filters \cite{362692}, vector space models \cite{CuencaAcuna2002DSIWorkshop} and view |
Bloom filters \cite{362692}, vector space models \cite{CuencaAcuna2002DSIWorkshop} and view |
1167 |
trees \cite{Bhattacharjee03resultcache} can be used for even better optimizations. In addition, authors |
trees \cite{Bhattacharjee03resultcache} can be used for even better optimizations. Authors |
1168 |
in \cite{li03feasibility} use Gap compression \cite{wittengigabytes}, Adaptive Set Intersection \cite{338634} |
in \cite{li03feasibility} use Gap compression \cite{wittengigabytes}, Adaptive Set Intersection \cite{338634} |
1169 |
and Clustering with their search optimizations. |
and clustering with their search optimizations. |
1170 |
|
|
1171 |
While it is expected that web-like searches can be layered on top of tightly structured overlay, much |
While it is expected that web-like searches can be layered on top of tightly structured overlay, much |
1172 |
more research is required to make indexing and searching more efficient. |
more research is required to make indexing and searching more efficient. |
1183 |
Peer-to-Peer system is \emph{never} in ''ideal'' state as it is always evolving system. |
Peer-to-Peer system is \emph{never} in ''ideal'' state as it is always evolving system. |
1184 |
|
|
1185 |
Current research has been focused on tightly structured systems' system management, since all presented |
Current research has been focused on tightly structured systems' system management, since all presented |
1186 |
algorithms have been analyzed under static simulation environments. Furthermore, propsed tightly structured |
tightly structured approache's algorithms have been analyzed under static simulation environments. Furthermore, propsed tightly structured |
1187 |
overlays are configured statically to achieve the desired reliability even in uncommon and adverse environment |
overlays are configured statically to achieve the desired reliability even in uncommon and adverse environment |
1188 |
\cite{rowston03controlloingreliability}. The most important factor for |
\cite{rowston03controlloingreliability}. The most important factor for |
1189 |
future research is to get real-life experiences from tightly structured system, when there are frequent |
future research is to get real-life experiences from tightly structured system, when there are frequent |
1190 |
joins and leaves in the system. Some research has been done already in this area. |
joins and leaves in the system. Some research has been done already in this area. |
1191 |
|
|
1192 |
A concept of ''half-life'' was introduced by Liben-Nowell \cite{libennowell01observations}. Half-life is defined |
A concept of ''half-life'' was introduced by Liben-Nowell \cite{libennowell01observations}. Half-life is defined |
1193 |
as follows: let there be N live nodes at time t. The doubling from time t is the time that pass before |
as follows: let there be $N$ live nodes at time $t$. The doubling from time $t$ is the time that pass before |
1194 |
N new additional nodes arrive into the system. The halving time from time t is the time |
$N$ new additional nodes arrive into the system. The halving time from time $t$ is the time |
1195 |
requires for half of the living nodes at time t to leave the system. The half-life from |
requires for half of the living nodes at time $t$ to leave the system. The half-life from |
1196 |
time t is smaller of the properties stated above. The half-life of the entire system is the |
time $t$ is smaller of the properties stated above. The half-life of the entire system is the |
1197 |
minimum half-life overl all times t. Concept of half-time can be used as basic for developing |
minimum half-life over all times $t$. Concept of half-time can be used as basic for developing |
1198 |
more efficient analytical tools for modelling complex Peer-to-Peer system. |
more efficient analytical tools for modelling complex Peer-to-Peer system. |
1199 |
|
|
1200 |
Some research has been done with regard to load balancing properties of tightly structured |
Some research has been done with regard to load balancing properties of tightly structured |
1231 |
describe a arbitrary data structure on top of tightly structured overlay \cite{zhang03somo}. They |
describe a arbitrary data structure on top of tightly structured overlay \cite{zhang03somo}. They |
1232 |
call their proposal as \emph{data overlay}, since it support several fundamental data structures. |
call their proposal as \emph{data overlay}, since it support several fundamental data structures. |
1233 |
Authors use this data overlay to build Self-Organized Metadata Overlay (SOMO), which can be used |
Authors use this data overlay to build Self-Organized Metadata Overlay (SOMO), which can be used |
1234 |
for monitoring health of tightly structured overlay. |
for monitoring health of tightly structured overlay. Fault tolerance of SOMO itself is currently |
1235 |
|
unknown. |
1236 |
|
|
1237 |
|
|
1238 |
\section{Miscellaneous problems in Peer-to-Peer} |
\section{Miscellaneous problems in Peer-to-Peer} |
1251 |
|
|
1252 |
\subsection{Social behaviour} |
\subsection{Social behaviour} |
1253 |
|
|
1254 |
Very frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate. Another belief |
Frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate. Another belief |
1255 |
is that all peers would behave equally, i.e. all peers both consume resources and contributes resources. |
is that all peers would behave equally, i.e. all peers both consume resources and contributes resources. |
1256 |
However, these assumptions are not true as several studies show peers rather consume than contribute and |
However, these assumptions are not true as several studies show peers rather consume than contribute and |
1257 |
and peers are unwilling to cooperate \cite{saroiu02measurementstudyp2p}, \cite{oram01harnessingpower}, |
and peers are unwilling to cooperate \cite{saroiu02measurementstudyp2p}, \cite{oram01harnessingpower}, |
1258 |
\cite{hearn02mojonation}. |
\cite{hearn02mojonation}. |
1259 |
|
|
1260 |
Somewhat surprisingly little research has been in this area, especiaclly when considering |
Somewhat surprisingly little research has been in this area, especially when considering |
1261 |
the possible impact of this \emph{unwanted socical behaviour} to performance of Peer-to-Peer |
the possible impact of this \emph{unwanted socical behaviour} to performance of Peer-to-Peer |
1262 |
system. However, problem is addressed by Golle et al. \cite{golle01incentivesp2p}. Some |
system. Problem is addressed by Golle et al. \cite{golle01incentivesp2p}. Some |
1263 |
research has been focused on semantic properties of the overlay in order to increase |
research has been focused on semantic properties of the overlay in order to increase |
1264 |
cooperation among participating peers \cite{crespo02semanticoverlay}. Ramanathan et al. |
cooperation among participating peers \cite{crespo02semanticoverlay}. Ramanathan et al. |
1265 |
\cite{ramanathan02goodpeers} and Bernstein et al. \cite{bernstein03selection} use |
\cite{ramanathan02goodpeers} and Bernstein et al. \cite{bernstein03selection} use |
1266 |
empirical metrics and decision trees when teaching peers to make better decisions |
empirical metrics and decision trees when teaching peers to make better decisions |
1267 |
when contacting other peers in Peer-to-Peer system. Alpine is an example of |
when contacting other peers in Peer-to-Peer system. Alpine \cite{alpineurl} is an example of |
1268 |
Peer-to-Peer system, which uses empirical metrics for peer selection. |
Peer-to-Peer system, which uses empirical metrics for peer selection. |
1269 |
|
|
1270 |
|
|
1271 |
\subsection{Simulating the Peer-to-Peer system} |
\subsection{Simulating the Peer-to-Peer system} |
1272 |
|
|
1273 |
Very little research has been done on simulating the global Peer-to-Peer system. Presumably, this |
Very little research has been done on simulating the \emph{global} Peer-to-Peer system. Presumably, this |
1274 |
is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very |
is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very |
1275 |
diffucult. Floyd et al. has been studying the simulation of the Internet in \cite{504642}. Authors |
diffucult. Floyd et al. has been studying the simulation of the Internet in \cite{504642}. Authors |
1276 |
state that simulating the Internet is very challenging task, because of Internet's heterogeneity |
state that simulating the Internet is very challenging task, because of Internet's heterogeneity |
1279 |
|
|
1280 |
As long as global simulations of Peer-to-Peer systems are lacking, we cannot we make any further |
As long as global simulations of Peer-to-Peer systems are lacking, we cannot we make any further |
1281 |
analysis e.g., on usage patterns in Peer-to-Peer systems. Presumedly, however, we can assume that |
analysis e.g., on usage patterns in Peer-to-Peer systems. Presumedly, however, we can assume that |
1282 |
queries follow the Zipf-like distributions both in the Internet and in Peer-to-Peer systems. |
, e.g., query keywords follow the Zipf-like distributions \cite{breslau98implications} both in the |
1283 |
|
Internet and in Peer-to-Peer systems. |
1284 |
|
|
1285 |
\section{Summary} |
\section{Summary} |
1286 |
|
|