/[gzz]/gzz/Documentation/misc/hemppah-progradu/masterthesis.tex
ViewVC logotype

Diff of /gzz/Documentation/misc/hemppah-progradu/masterthesis.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.152 by hemppah, Mon Mar 17 11:42:01 2003 UTC revision 1.153 by hemppah, Mon Mar 17 15:07:39 2003 UTC
# Line 42  E-mail: hemppah@cc.jyu.fi} Line 42  E-mail: hemppah@cc.jyu.fi}
42  \abstract{  \abstract{
43  In this thesis, we review existing Peer-to-Peer approaches, algorithms and their  In this thesis, we review existing Peer-to-Peer approaches, algorithms and their
44  key properties. We summarize open problems in Peer-to-Peer systems and divide  key properties. We summarize open problems in Peer-to-Peer systems and divide
45  problems into three sub-categories. We observe that there are many  problems into three sub-categories. We observe that there are many problems with
46  problems which have no solutions at all, or problems that have proposed  either no solutions at all, or only practically unrealizable ones.
 solutions but which are practically unrealizable.  
47    
48  Then, we give an overview of the Fenfire system.  We evaluate existing  Then, we give an overview of the Fenfire system.  We evaluate existing
49  Peer-to-Peer approaches-- loosely and tightly structured overlays-- with regard  Peer-to-Peer approaches-- loosely and tightly structured overlays-- with regard
# Line 84  Peer-to-Peer systems have recently recei Line 83  Peer-to-Peer systems have recently recei
83  academia \cite{projectirisurl} and industry \cite{p2pworkinggroup, jxtaurl} for a  academia \cite{projectirisurl} and industry \cite{p2pworkinggroup, jxtaurl} for a
84  number of reasons. The lack of centralization in Peer-to-Peer systems  number of reasons. The lack of centralization in Peer-to-Peer systems
85  means that the participants can form a distributed system without any investment to  means that the participants can form a distributed system without any investment to
86  centralized hardware which would coordinate it by sharing their services  centralized hardware, which would coordinate it by sharing their services
87  and connecting to each other directly. Additionally, the distributed and ad hoc nature of  and connecting to each other directly. Additionally, the distributed and ad hoc nature of
88  Peer-to-Peer improves scalability and avoids single points of failure.  Peer-to-Peer improves scalability and avoids single points of failure.
89    
# Line 95  systems in which all communication is sy Line 94  systems in which all communication is sy
94  capabilities and responsibilities. Each entity, i.e., \emph{peer}, may contribute services  capabilities and responsibilities. Each entity, i.e., \emph{peer}, may contribute services
95  to the overall system.  to the overall system.
96    
97  The Fenfire project is an attempt to build hyperstructured, seamlessly interoperating desktop  The Fenfire project is an attempt to build a hyperstructured, seamlessly interoperating desktop
98  environment. In Fenfire, all data is stored as data blocks.    environment. In the Fenfire, all data is stored as data blocks.  
99  All data blocks have globally unique identifiers and they can be referred by pointer blocks.  All data blocks have globally unique identifiers and they can be referred by pointer blocks.
100  Other features of Fenfire include innovative user  Other features of the Fenfire include innovative user
101  interfaces for viewing data and usage of Peer-to-Peer networking for network transparency.  interfaces for viewing data and usage of Peer-to-Peer networking for network transparency.
102    
103  In this thesis, we\footnote{Use of the plural is customary even if research  In this thesis, we evaluate existing Peer-to-Peer approaches and
 paper is authored solely.} evaluate existing Peer-to-Peer approaches and  
104  choose the best alternative to Fenfire's needs.  choose the best alternative to Fenfire's needs.
105    
106  We start by reviewing existing Peer-to-Peer approaches, algorithms and their key properties.  We start by reviewing existing Peer-to-Peer approaches, algorithms and their key properties.
107  We observe that despite the great amount of proposed Peer-to-Peer systems, all systems fall either to  We observe that despite the great amount of proposed Peer-to-Peer systems, all systems fall either to loosely or
108  loosely structured approach or tightly structured approach. We also discuss open problems in  tightly structured approach. We also discuss open problems in
109  Peer-to-Peer systems and divide problems into three sub-categories: security problems,  Peer-to-Peer systems and divide problems into three sub-categories: security, performance, and miscellaneous
110  performance problems and miscellaneous problems.    problems.  
111    
112  Then, we give an overview of Fenfire project, and evaluate Peer-to-Peer approaches to Fenfire's  Then, we give an overview of the Fenfire project, and evaluate Peer-to-Peer approaches to Fenfire's
113  needs. Finally, we propose simple but yet efficient methods to be used for data lookups in Peer-to-Peer  needs. Finally, we propose simple but yet efficient methods to be used for data lookups in Peer-to-Peer
114  environment.  environment.
115    
# Line 120  Peer-to-Peer domain. However, this thesi Line 118  Peer-to-Peer domain. However, this thesi
118  information can be found from the references.  information can be found from the references.
119    
120  There are three research problems discussed in this thesis. First research problem  There are three research problems discussed in this thesis. First research problem
121  is to find the most efficient way to locate and fetch Fenfire data blocks from a  is finding the most efficient way to locate and fetch Fenfire data blocks from a
122  Peer-to-Peer network, where the block's identifier is given. Second, we want  Peer-to-Peer network, where the block's identifier is given. Second, we want
123  to find the most efficient way to locate and fetch the most recent Fenfire data block from a  to find the most efficient way to locate and fetch the most recent Fenfire data block from a
124  Peer-to-Peer network referred by a pointer block. The third problem  Peer-to-Peer network referred by a pointer block. The third problem
# Line 130  data block, where date and or time range Line 128  data block, where date and or time range
128  This thesis is structured as follows. In the next chapter, we give an overview of  This thesis is structured as follows. In the next chapter, we give an overview of
129  existing Peer-to-Peer approaches, algorithms and key differences between them. In chapter 3, we  existing Peer-to-Peer approaches, algorithms and key differences between them. In chapter 3, we
130  address open problems in Peer-to-Peer domain and divide problems into three  address open problems in Peer-to-Peer domain and divide problems into three
131  sub-categories. Chapter 4 gives an overview of Fenfire system. In chapter  sub-categories. Chapter 4 gives an overview of the Fenfire system. In chapter
132  5, we evaluate existing Peer-to-Peer approaches with regard to Fenfire system.  5, we evaluate existing Peer-to-Peer approaches with regard to the Fenfire system.
133  Finally, in chapter 6 we conclusions and future work.  Finally, in chapter 6 we conclusions and future work.
134    
135    
136  \chapter{Peer-to-Peer architectures}  \chapter{Peer-to-Peer architectures}
137  In this chapter we will give a brief history and overview of Peer-to-Peer networks,  In this chapter we will give a brief history and overview of Peer-to-Peer networks,
138  review most important Peer-to-Peer algorithms and list key differences between  review most the important Peer-to-Peer algorithms and list key differences between the
139  two main approaches.  two main approaches.
140    
141  \section{Brief history and overview}  \section{Brief history and overview}
142    
143  The Internet has been originally established in the late 1960s. The objective  The Internet was originally established in the late 1960s. The objective
144  of the ARPANET-project was to share computers' resources among military computers  of the ARPANET-project was to share computers' resources among military computers
145  around the United States. The most challenging purpose of ARPANET was to integrate  around the United States. The most challenging purpose of ARPANET was to integrate
146  different kinds of existing network technologies with one common network architecture.  different kinds of existing network technologies with one common network architecture.
147  The ARPANET connected the first few hosts together not in client/server relationship,  The ARPANET connected the first few hosts together not in client/server relationship,
148  but rather as equal networking \emph{peers}. This could be seen as starting point  but rather as equal networking \emph{peers}. This could be seen as the starting point
149  both of Peer-to-Peer concept and the Internet \cite{oram01harnessingpower}.  of both the Peer-to-Peer concept and the Internet \cite{oram01harnessingpower}.
150    
151  In subsequent years, the Internet became more restricted to client--server based  In subsequent years, the Internet became more restricted to client--server based
152  applications. In recent years, however, Peer-to-Peer systems have again emerged  applications. In recent years, however, Peer-to-Peer systems have again emerged
153  room in computing world. Indeed, Peer-to-Peer has had significant social and technical  in computing world. Indeed, Peer-to-Peer has had significant social and technical
154  attention in academia \cite{projectirisurl}, industry \cite{p2pworkinggroup},  attention in academia \cite{projectirisurl} and industry \cite{p2pworkinggroup, jxtaurl}.
155  \cite{jxtaurl}. Already deceased Napster \cite{napsterurl},  The deceased Napster \cite{napsterurl},
156  launched in 1999, was a new starting point for modern Peer-to-Peer computing. After  launched in 1999, was a new starting point for modern Peer-to-Peer computing. After
157  Napster, hundreds of Peer-to-Peer systems have been developed and proposed.  Napster, hundreds of Peer-to-Peer systems have been developed and proposed.
158    
159  Modern Peer-to-Peer system is composed of \emph{application} level overlay network.  A modern Peer-to-Peer system is composed of an \emph{application} level overlay network.
160  Figure \ref{fig:application_level} illustrates the analogy of Peer-to-Peer network with  Figure \ref{fig:application_level} illustrates the analogy of Peer-to-Peer network with
161  regard to OSI model. Compared to ARPANET's Peer-to-Peer functionality, modern Peer-to-Peer systems  regard to OSI model. Compared to ARPANET's Peer-to-Peer functionality, modern Peer-to-Peer systems
162  are ad hoc, i.e., peers join and leave the system constantly in a dynamic manner. This  are ad hoc, i.e., peers join and leave the system constantly in a dynamic manner. This
163  fact constitutes challenging requirements for efficient construction and maintenance  fact constitutes challenging requirements for efficient construction and maintenance
164  of the overlay network. Even more demanding tasks are how to perform efficient data  of the overlay network. Even more demanding tasks are performing efficient data
165  lookup and maintain security in a varying distributed environment. The most popular  lookup and maintaining security in a varying distributed environment. The most popular
166  form of modern Peer-to-Peer computing is file-sharing. In this scenario, participants  form of modern Peer-to-Peer computing is file-sharing. In this scenario, participants
167  of Peer-to-Peer network share their file resources to other participants while obtaining  of Peer-to-Peer network share their file resources with other participants.
168  more resources from others. This can be seen as a variant of distributed file system   This can be seen as a variant of distributed file system
169  (e.g., \cite{levy90distributedfilesystems}).  (e.g., \cite{levy90distributedfilesystems}).
170    
171  \begin{figure}  \begin{figure}
# Line 179  more resources from others. This can be Line 177  more resources from others. This can be
177    
178    
179    
180  In a development of modern Peer-to-Peer systems, lot of influences has been attained from  In the development of modern Peer-to-Peer systems, lot of influence has been attained from
181  other research areas than computer science. Research has been conducted regarding  other research areas than computer science. Research has been conducted regarding
182  to self-organizing nature of complex networks \cite{albert-02-statistical}, \cite{albert-00-tolerance}, \cite{watts00dynamics}.  the self-organizing nature of complex networks \cite{albert-02-statistical, albert-00-tolerance, watts00dynamics}.
183  It's interesting to realize that chemical properties of cells, the Internet, ad hoc  It is interesting to realize that chemical properties of biological cells, the Internet, ad hoc
184  Peer-to-Peer systems, and social networks have all in common that they self-organize based on same  Peer-to-Peer systems, and social networks have all in common that they self-organize based on the same
185  principles.  Furthermore, the association between social relationships among people  principles.  Furthermore, the association between social relationships among people
186  and Peer-to-Peer overlay topology has been studied recently  \cite{watts00dynamics},  and Peer-to-Peer overlay topology has been studied recently \cite{watts00dynamics, kleinberg99small, nips02-Kleinberg}.
187  \cite{kleinberg99small}, \cite{nips02-Kleinberg}. This insight is motivated  This insight is motivated by Milgram, who noticed that people are very effective in locating other people in a wide scale
 by Milgram, who noticed that people are very effective to locate other people in a wide scale  
188  based on local knowledge. This phenomenon is called as ''small-world phenomenon''  based on local knowledge. This phenomenon is called as ''small-world phenomenon''
189  \cite{milgram67smallworld}. As a consequence, many modern Peer-to-Peer systems  \cite{milgram67smallworld}. As a consequence, many modern Peer-to-Peer systems
190  have applied techniques outside of computer science when constructing and maintaining  have applied techniques outside of computer science when constructing and maintaining
191  the application level overlay network.  the application level overlay network.
192    
193  In the end, however, there are two main approaches in which all modern Peer-to-Peer  In the end, however, there are two main approaches in which all modern Peer-to-Peer
194  systems fall: loosely structured approach and tightly structured approach. In loosely  systems fall: the loosely structured approach and the tightly structured approach. In the loosely
195  structured approach the construction and the maintenance of the overlay is controlled  structured approach the construction and the maintenance of the overlay is controlled
196  loosely. This approach gives freedom for participating peers  loosely. This approach gives freedom for participating peers
197  to perform certain tasks in Peer-to-Peer network. On the other hand, tightly structured  to perform certain tasks in a Peer-to-Peer network. On the other hand, the tightly structured
198  approach has some rules, which all participating peers have to obey. In the following  approach has some rules, which all participating peers have to obey.
 sections, we will discuss in more detail both approaches and key differences between them.  
199    
200    
201  \section{Centralized}  \section{Centralized}
# Line 208  Napster\footnote{We decided to include N Line 204  Napster\footnote{We decided to include N
204  historical value (see previous section).} \cite{napsterurl}  was designed to allow  historical value (see previous section).} \cite{napsterurl}  was designed to allow
205  people to share music. It was a hybrid Peer-to-Peer file-sharing system, i.e., the search  people to share music. It was a hybrid Peer-to-Peer file-sharing system, i.e., the search
206  index was centralized and the distribution of storage and serving of files was distributed.  index was centralized and the distribution of storage and serving of files was distributed.
207  Peers in the Napster network performed requests to the central directory server to find  Peers in the Napster network made requests to the central directory server to find
208  other peers hosting desirable content. Since service requests were totally based on  other peers hosting desirable content. Since service requests were totally based on a
209  centralized index, Napster didn't scale well because of constantly updated central  centralized index, Napster didn't scale well because of constantly updated central
210  directory, and had a single point of failure.  directory, and had a single point of failure.
211    
# Line 221  other pure Peer-to-Peer networks, no pee Line 217  other pure Peer-to-Peer networks, no pee
217  The construction and maintenance of Gnutella network is extremely ad hoc, since participating  The construction and maintenance of Gnutella network is extremely ad hoc, since participating
218  peers can form the overlay network based on \emph{local} knowledge. Figure \ref{fig:gnutella_overlay}  peers can form the overlay network based on \emph{local} knowledge. Figure \ref{fig:gnutella_overlay}
219  illustrates how peers form an overlay network. Initially, peer 1 creates the overlay, since  illustrates how peers form an overlay network. Initially, peer 1 creates the overlay, since
220  it's the first participating peer. Then, repeatedly new peers join the network and connect to  it is the first participating peer. Then, repeatedly new peers join the network and connect to
221  other peers in a random manner. Thus, Gnutella can be considered as a variation of \emph{scale-free  other peers in a random manner. Thus, Gnutella can be considered as a variation of \emph{scale-free
222  graph}\footnote{In scale-free graphs (also known as power-law graphs) only a few peers have high number of neighbor  graph}\footnote{In scale-free graphs (also known as power-law graphs) only a few peers have high number of neighbor
223  links and major of peers have low number of neighbor links.}.  links and the majority of peers have low number of neighbor links.}.
224    
225  \begin{figure}  \begin{figure}
226  \centering  \centering
# Line 234  links and major of peers have low number Line 230  links and major of peers have low number
230  \end{figure}  \end{figure}
231    
232    
233  In Gnutella, each participating peer maintains local index of its own shared content. Also,  In Gnutella, each participating peer maintains a local index of its own shared content. Also,
234  each peer has a few connections to other peer, i.e., peer's \emph{neighbors}. Basic Gnutella  each peer has a few connections to other peers, i.e., peer's \emph{neighbors}. Basic Gnutella
235  data lookup works as follows: peer broadcasts a query request to its neighbors, which in turn  data lookup works as follows: peer broadcasts a query request to its neighbors, which in turn
236  forwards the query to their neighbors. This leads to a situation where number of messages  forward the query to their neighbors. This leads to a situation where the number of messages
237  in the network can grow with $O(n^{2})$, where $n$ is the number of participating peers in  in the network can grow with $O(n^{2})$, where $n$ is the number of participating peers in
238  Gnutella network. To limit the amount of network traffic, Gnutella uses Time-To-Live-limited  Gnutella network. To limit the amount of network traffic, Gnutella uses Time-To-Live-limited
239  (TTL) flooding to distribute queries. Therefore, Gnutella uses a Breadth-First-Search (BFS) algorithm  (TTL) flooding to distribute queries. Therefore, Gnutella uses a Breadth-First-Search (BFS) algorithm
# Line 254  Figure \ref{fig:gnutella_query} shows th Line 250  Figure \ref{fig:gnutella_query} shows th
250  \label{fig:gnutella_query}  \label{fig:gnutella_query}
251  \end{figure}  \end{figure}
252    
253  According to \cite{lv02searchreplication}, Gnutella's way to perform data lookups, \emph{flooding}, has  According to \cite{lv02searchreplication}, Gnutella's way to perform data lookups, \emph{flooding}, has the
254  following limitations. First, choosing the appropriate TTL in practice is not easy. If the  following limitations. First, choosing the appropriate TTL in practice is not easy. If the
255  TTL is too high, query originator may unnecessarily strain the network. If the TTL is too  TTL is too high, query originator may unnecessarily strain the network. If the TTL is too
256  low, the query originator might not find the desired data even if it's available somewhere  low, the query originator might not find the desired data even if it is available somewhere
257  in the network. Second, there are many duplicate messages generated by flooding, especially  in the network. Second, there are many duplicate messages generated by flooding, especially
258  in high connectivity graphs. It is obvious that with these limitations, flooding creates  in high connectivity graphs. It is obvious that with these limitations, flooding creates
259  significant message processing overhead for each data lookup. Even worse, flooding may increase  significant message processing overhead for each data lookup. Even worse, flooding may increase
260  the load on participating to the point, where it has to leave the network.  the load on participating peer to the point where it has to leave the network.
261    
262  Lately, there has been done lot of research to improve Gnutella's data lookup efficiency  Lately, Gnutella's data lookup efficiency and scalability has been deeply researched.
263  and scalability. Adamic et al. \cite{adamic99small}, \cite{adamic02localsearch},  Adamic et al. \cite{adamic99small, adamic02localsearch, adamic01powerlawsearch}
264  \cite{adamic01powerlawsearch} have studied different data lookup methods in power-law  have studied different data lookup methods in power-law networks and have found that by
265  networks and have found that by  instructing the peers that forward data lookups to select high degree peers, the performance of data lookup
 instructing peers forwarding data lookups to select high degree peers, the performance of data lookup  
266  increases significantly. As a result, some of the most recent loosely  increases significantly. As a result, some of the most recent loosely
267  structured Peer-to-Peer systems have adopted this method with some modifications  structured Peer-to-Peer systems have adopted this method with some modifications
268  \cite{gnutella2url}, \cite{shareazaurl}, \cite{fasttrackurl}, \cite{morpheusurl},  \cite{gnutella2url, shareazaurl, fasttrackurl, morpheusurl, kazaaurl, waterhouse02searchp2p, botros01jxtasearch,
269  \cite{kazaaurl}, \cite{waterhouse02searchp2p}, \cite{botros01jxtasearch},  ganesan02yappers}.
 \cite{ganesan02yappers}.  
270  Figures \ref{fig:gnutella_overlay_supernodes} and \ref{fig:gnutella_overlay_cluster}  Figures \ref{fig:gnutella_overlay_supernodes} and \ref{fig:gnutella_overlay_cluster}
271  illustrates simplified variations of power-law overlay networks. Figure \ref{fig:gnutella_powerlaw}  illustrates simplified variations of power-law overlay networks. Figure \ref{fig:gnutella_powerlaw}
272  presents pure topology of power-law network. All the systems  presents pure topology of power-law network.
273  share the property of that high degree peers maintain index of all other peers  
274  they know about. However, it's not clear whether this algorithm is scalable or not,  It is not clear whether this algorithm is scalable or not,
275  as majority of the query requests are sent only to the high degree peers, making  as the majority of the query requests are sent only to the high degree peers, making
276  them stress the load of entire system.  them stress the load of entire system.
277    
278  \begin{figure}  \begin{figure}
# Line 303  them stress the load of entire system. Line 297  them stress the load of entire system.
297  \end{figure}      \end{figure}    
298    
299  Previously presented improvements are only partial solutions. More advanced techniques  Previously presented improvements are only partial solutions. More advanced techniques
300  to improve data lookup of loosely structured systems are presented in chapter 3.  to improve data lookup of loosely structured systems are discussed in chapter 3.
301    
302    
303  \subsection{Sketch of formal definition}  \subsection{Sketch of a formal definition}
304    
305  In this subsection we formalize loosely structured overlay's main components. This  In this subsection we formalize loosely structured overlay's main components. This
306  model is based on original Gnutella overlay network with power-law improvements.  model is based on original Gnutella overlay network with power-law improvements.
307    
308  Let $S$ be the aggregate of all services $s$ in system. Let $P$ be the aggregate of  Let $S$ be the aggregate of all services $s$ in system. Let $P$ be the aggregate of
309  all peers $p$ in system. Then, $\forall s \in S$, there is a provider of the service,  all peers $p$ in system. Then, $\forall s \in S$, there is a provider of the service,
310  expressed as $p = provider(s)$. Every $p$ has neighbor(s), named as $neighbor$, which  expressed as $p = provider(s)$. Every $p$ has neighbor(s), named as $p_n$, which
311  is $P$ = \{$p \in P: \exists neighbor$, which is randomly chosen from $P$\}.  is $P$ = \{$p \in P: \exists neighbor$, which is randomly chosen from $P$\}.
312  \emph{Super peer} is a peer, which hosts the indices of other peers, $si = summaryindex(provider(s))$  \emph{Super peer} is a peer, which hosts the indices of other peers, $si = summaryindex(provider(s))$
313  and $\forall$ regular peer $p$, there is super peer, which has has a index of regular  and $\forall$ regular peer $p$, there is super peer, which has has a index of regular
# Line 332  Koorde \cite{kaashoek03koorde}, Overlapp Line 326  Koorde \cite{kaashoek03koorde}, Overlapp
326  Skip Graphs \cite{AspnesS2003}, SkipNet \cite{harvey03skipnet2},  Skip Graphs \cite{AspnesS2003}, SkipNet \cite{harvey03skipnet2},
327  Symphony \cite{gurmeet03symphony}, SWAN \cite{bonsma02swan}, Tapestry  Symphony \cite{gurmeet03symphony}, SWAN \cite{bonsma02swan}, Tapestry
328  \cite{zhao01tapestry}, Viceroy \cite{malkhi02viceroy} and others \cite{freedman02trie}.  \cite{zhao01tapestry}, Viceroy \cite{malkhi02viceroy} and others \cite{freedman02trie}.
329  The biggest difference compared to loosely structured approach is that with tightly structured systems,  The biggest difference compared to the loosely structured approach is that with tightly structured systems,
330  it is now feasible to perform \emph{global} data lookups in the overlay.  it is now feasible to perform \emph{global} data lookups in the overlay.
331  While there are significant differences among proposed tighty structured systems, they all have in common  While there are significant differences among proposed tighty structured systems, they all have in common
332  that \emph{peer identifiers} are assigned to participating peers from  that \emph{peer identifiers} are assigned to participating peers from
# Line 342  which are selected from the same identif Line 336  which are selected from the same identif
336  space differs between proposed systems. Circular identifier space (and variants)  space differs between proposed systems. Circular identifier space (and variants)
337  is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde},  is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde},
338  Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry}  Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry}
339  and Viceroy \cite{malkhi02viceroy} use circular identifier space of $n$-bit integers modulo $2^{n}$. The  and Viceroy \cite{malkhi02viceroy} use a circular identifier space of $n$-bit integers modulo $2^{n}$. The
340  value of $n$ varies among systems. Again, CAN \cite{ratnasamy01can} uses a $d$-dimensional Cartesian  value of $n$ varies among systems. Again, CAN \cite{ratnasamy01can} uses a $d$-dimensional Cartesian
341  model to implement identifier space.  model to implement identifier space.
342    
343  To store data into tightly structured overlay, each application-specific  To store data into a tightly structured overlay, each application-specific
344  unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent  unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} uniformly (e.g., using consistent
345  hashing \cite{258660}) by the overlay to an existing peer in the overlay. Thus, tightly  hashing \cite{258660}) by the overlay to an existing peer in the overlay. Thus, tightly
346  structured overlay assigns a subset of all possible keys to every participating peer  structured overlay assigns a subset of all possible keys to every participating peer.
347  \footnote{We say that a peer is \emph{responsible} for the keys which are assigned by the overlay.}.    We say that a peer is \emph{responsible} for the keys which are assigned by the overlay.  
348  Also, each peer in tightly structured overlay maintains a \emph{routing table}, which  Also, each peer in tightly structured overlay maintains a \emph{routing table}, which
349  consists of identifiers and IP addresses of other peers in the overlay. Entries of routing  consists of identifiers and IP addresses of other peers in the overlay. Entries of the routing
350  table are peer's neighbors in the overlay network. Figure \ref{fig:structured_hashing} illustrates the  table represents peer's neighbors in the overlay network. Figure \ref{fig:structured_hashing} illustrates the
351  process of data to key mapping in tightly structured overlays.  process of data to key mapping in a tightly structured overlay.
352    
353  \begin{figure}  \begin{figure}
354  \centering  \centering
# Line 367  All messages are routed across the overl Line 361  All messages are routed across the overl
361  peer identifier is gradually ''closer'' to the key's identifier  peer identifier is gradually ''closer'' to the key's identifier
362  in the identifier space. The distance can be measured by numerical  in the identifier space. The distance can be measured by numerical
363  difference between identifiers (e.g., Chord \cite{stoica01chord}), number of  difference between identifiers (e.g., Chord \cite{stoica01chord}), number of
364  same prefix bits between identifiers (e.g., Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry}) or  same prefix bits between identifiers (e.g., Pastry \cite{rowston01pastry} and Tapestry
365  bit-wise exclusive or (XOR) (e.g., Kademlia \cite{maymounkov02kademlia}).  \cite{zhao01tapestry}) or bit-wise exclusive or (XOR) (e.g., Kademlia \cite{maymounkov02kademlia}).
366  Because of XOR-metric, Kademlia's distance function is both unidirectional  Chord's \cite{stoica01chord} distance function does have the property of unidirection
367  (for a given point $p_i$ in the identifier space and distance $d$ > 0, there  (for a given point $p_i$ in the identifier space and distance $d$ > 0, there
368  is exactly one point $p_j$ in a way that the distance between $p_i$ and $p_j$  is exactly one point $p_j$ in a way that the distance between $p_i$ and $p_j$
369  is $d$) and symmetric (the distance from $p_i$ to $p_j$ is same as the  is $d$), but doesn't have symmetry (the distance from $p_i$ to $p_j$ is same as the
370  distance from $p_j$ to $p_i$) \cite{maymounkov02kademlia}. On the other  distance from $p_j$ to $p_i$). Pastry's \cite{rowston01pastry} distance function supports
371  hand, Chord's \cite{stoica01chord} distance function does have the property  symmetry, but doesn't support unidirection. Because of XOR-metric, Kademlia's distance
372  of unidirection, but doesn't have symmetry. Pastry's \cite{rowston01pastry} distance  function is both unidirectional and symmetric. Moreover, Kademlia's \cite{maymounkov02kademlia}
373  function supports symmetry, but doesn't support unidirection. As a consequence,  XOR-based metric doesn't need stabilization (like in Chord \cite{stoica01chord}) and backup links
 Kademlia's \cite{maymounkov02kademlia} XOR-based metric doesn't need  
 stabilization (like in Chord \cite{stoica01chord}) and backup links  
374  (like in Pastry \cite{rowston01pastry}) \cite{balakrishanarticle03lookupp2p}.  (like in Pastry \cite{rowston01pastry}) \cite{balakrishanarticle03lookupp2p}.
375  However, in all previously schemes each  However, in all previous schemes each hop in the overlay shortens the distance between
376  hop in the overlay shortens the distance between current peer working with the data lookup  current peer working with the data lookup and the key which was looked up in the identifier space.
 and the key which was looked up in the identifier space.  
377    
378  Skip Graphs \cite{AspnesS2003} and SWAN \cite{bonsma02swan} employ a key space very similar to a tightly structured  Skip Graphs \cite{AspnesS2003} and SWAN \cite{bonsma02swan} employ a key space very similar to a tightly structured
379  overlay, but in which queries are routed  to \emph{keys}. In these systems  overlay, but in which queries are routed  to \emph{keys}. In these systems
380  peer occupies several positions in the identifier space, one for each  a peer occupies several positions in the identifier space, one for each
381  application-specific key. The indirection of placing close keys in the  application-specific key. The indirection of placing close keys in the
382  custody of a storing peer\footnote{Storing peer is the peer in the overlay which is responsible for the  custody of a storing peer is removed at the cost of each peer maintaining one
383  assigned keys.} is removed at the cost of each peer maintaining one  ''resource peer'' in the overlay network for each data item it publishes. Provider peer is the peer
384  ''resource peer'' in the overlay network for each data item it publishes.  in the overlay which is responsible for the assigned keys
385    
386  PeerNet \cite{eriksson03peernet} differs from other tightly structured overlays in that it operates  PeerNet \cite{eriksson03peernet} differs from other tightly structured overlays in that it operates
387  at the \emph{network} layer. PeerNet makes an explicit distinction  at the \emph{network} layer. PeerNet makes an explicit distinction
# Line 400  as other tightly structured overlays, i. Line 391  as other tightly structured overlays, i.
391  for maintaining information about other peers in the system and  for maintaining information about other peers in the system and
392  $O(\log{n})$ data lookup efficiency.  $O(\log{n})$ data lookup efficiency.
393    
394  Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} have listed  Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} which have to be
 four requirements for tightly structured overlays, which have to be  
395  addressed in order to perform efficient data lookups in tightly structured overlays.  addressed in order to perform efficient data lookups in tightly structured overlays.
396  First, mapping of keys to peers must be done in a load-balanced  First, mapping of keys to peers must be done in a load-balanced
397  way. Second, the overlay must be able to forward a lookup for a  way. Second, the overlay must be able to forward a lookup for a
398  specific key to an appropriate peer. Third, overlay must have a  specific key to an appropriate peer. Third, overlay must have
399  support for a distance function. Finally,  routing tables for each peer  support for a efficient distance function. Finally,  routing tables for each peer
400  must be constructed and maintained adaptively.  must be constructed and maintained adaptively.
401    
402  Currently, all proposed tightly structured overlays provide at least  Currently, all proposed tightly structured overlays provide at least
# Line 417  data structure which resembles Skip list Line 407  data structure which resembles Skip list
407  In figure \ref{fig:structured_query}, we present an overview of Chord's lookup process.  In figure \ref{fig:structured_query}, we present an overview of Chord's lookup process.
408  On the right side of Chord's lookup process, the same data lookup process  On the right side of Chord's lookup process, the same data lookup process
409  is shown as a binary-tree abstraction.  It can be noticed, that in each step, the distance  is shown as a binary-tree abstraction.  It can be noticed, that in each step, the distance
410  decreases between with a logarithmic efficiency.  decreases with a logarithmic efficiency.
411    
412  Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry  Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry
413  \cite{zhao01tapestry} uses balanced $k$-trees as routing table's data structure. Figure  \cite{zhao01tapestry} uses balanced $k$-trees as routing table's data structure. Figure
414  \ref{fig:kademlia_lookup} shows the process of Kademlia's  \ref{fig:kademlia_lookup} shows the process of Kademlia's
415  data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}),  data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}),
416  which requires only constant number of neighbor peers while providing $O(\log{n})$ data lookup  which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup
417  efficiency. Koorde \cite{kaashoek03koorde}, recent modification of Chord, uses de Bruijn graphs  efficiency. Koorde \cite{kaashoek03koorde}, a recent modification of Chord, uses de Bruijn graphs
418  \cite{debruijn46graph} to maintain local routing tables. Koorde \cite{kaashoek03koorde} requires  \cite{debruijn46graph} to maintain local routing tables. Koorde \cite{kaashoek03koorde} requires
419  each peer to have only about two links to other peers to provide $O(\log{n})$ performance.  each peer to have only about two links to other peers to provide $O(\log{n})$ performance.
420    
# Line 452  implements three operations: \texttt{loo Line 442  implements three operations: \texttt{loo
442  \texttt{insert(key)}. As the name suggests, DHT implements the same functionality  \texttt{insert(key)}. As the name suggests, DHT implements the same functionality
443  as a regular hash table, by storing the mapping between a key and a value. DHT's  as a regular hash table, by storing the mapping between a key and a value. DHT's
444  \emph{interface} is generic; values can be any size and type. Figure \ref{fig:Structured_lookup_using_DHT_model}  \emph{interface} is generic; values can be any size and type. Figure \ref{fig:Structured_lookup_using_DHT_model}
445  shows the DHT abstraction of tightly structured overlay. Second, Decentralized  shows the DHT abstraction of the tightly structured overlay. Second, Decentralized
446  Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is distributed  Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is a distributed
447  directory service. DOLR stores \emph{pointers} to where data items are stored  directory service. DOLR stores \emph{pointers} to data items throughout the overlay. DOLR's main
448  throughout the overlay. DOLR's main operations are \texttt{publish(key)},  operations are \texttt{publish(key)}, \texttt{removePublished(key)} and \texttt{sendToObject(key)}. The key
 \texttt{removePublished(key)} and \texttt{sendToObject(key)}. The key  
449  difference between DHT and DOLR abstraction is that DOLR routes overlay's messages  difference between DHT and DOLR abstraction is that DOLR routes overlay's messages
450  to nearest available peer, hosting a specific data item. This form of locality  to nearest available peer, hosting a specific data item. This form of locality
451  is not supported by DHT. Finally, tightly structured overlay can be used for  is not supported by DHT. Finally, tightly structured overlay can be used for
# Line 485  presents the DOLR abstraction. Line 474  presents the DOLR abstraction.
474  \end{figure}  \end{figure}
475    
476    
477  \subsection{Sketch of formal definition}  \subsection{Sketch of a formal definition}
478    
479  In this subsection we formalize tightly structured overlay's main features, i.e.,  In this subsection, we formalize the main features of tightly structured overlay, i.e.,
480  identifiers, identifier space and mapping function.  identifiers, identifier space and mapping function.
481    
482  Let $S$ be the aggregate of all services $s$ in system. Let $P$ be the aggregate of  Let $S$ be the aggregate of all services $s$ in the system. Let $P$ be the aggregate of
483  all peers $p$ in system. Let $I$ be the aggregate of all identifiers $i$ in system.  all peers $p$ in system. Let $I$ be the aggregate of all identifiers $i$ in system.
484  Let $IS$ be the aggregate of all identifier points $ip$ in system. Then, $\forall s \in S$,  Let $IS$ be the aggregate of all identifier points $ip$ in system. Then, $\forall s \in S$,
485  there is a provider of the service, expressed as $p = provider(s)$. Service's identifier  there is a provider of the service, expressed as $p = provider(s)$. Service's identifier
# Line 506  where $difference(p,neighbor) = ''close' Line 495  where $difference(p,neighbor) = ''close'
495    
496  \section{Summary}  \section{Summary}
497    
498  In this section we compare loosely structured approach and tightly structured approach.  In this section we compare the loosely structured approach and the tightly structured approach.
499  We also summarize proposed Peer-to-Peer algorithms and their key properties with regard  We also summarize proposed Peer-to-Peer algorithms and their key properties with regard
500  to performance and scalability aspects.  to performance and scalability aspects.
501    
502  \subsection{Differences}  \subsection{Differences}
503    
504  Even though loosely structured and tightly structured approach are both Peer-to-Peer schemes, they  Even though the loosely structured and the tightly structured approach are both Peer-to-Peer schemes, they
505  have very little in common. Indeed, the only thing they share is the fact that no other peer is more  have very little in common. Indeed, the only thing they share is the fact that no other peer is more
506  important than an other in the Peer-to-Peer network. Fault tolerance \emph{may}  important than any other in the Peer-to-Peer network. Fault tolerance \emph{may}
507  be an area, in which approaches have similar properties (e.g., no single point of failure).  be an area, in which approaches have similar properties (e.g., no single point of failure).
508  Fault tolerance properties of both approaches are currently only initial calculations, or  Fault tolerance properties of both approaches are currently only initial calculations, or
509  experimented in simulation environments. In real-life, however, measuring fault tolerance is much more  experimented in simulation environments. In real-life, however, measuring fault tolerance is a much more
510  challenging task and requires more research to get reliable answers.  challenging task and requires more research to get reliable answers.
511    
512  The most important difference between approaches is performance and scalability properties. While  The most important difference between approaches is performance and scalability properties. Generally
513  performance of loosely structured approach is not always even linear, generally tightly structured  tightly structured systems can perform all internal operations in poly-logarithmic time\footnote{However, it is unknown
514  approach can perform all internal operations in poly-logarithmic time\footnote{However, it is unknown  whether all proposed algorithms can preserve logarithmic properties in real-life applications or not.}
515  whether all proposed algorithms can preserve logarithmic properties in real-life applications or not.}.  while the performance of loosely structured systems is not always even linear, .
516  Moreover, loosely structured systems scale to millions of peers, whereas tightly structured systems are able  Moreover, loosely structured systems scale to millions of peers, whereas tightly structured systems are able
517  to cope with billions of concurrent peers \cite{osokine02distnetworks}, \cite{kubiatowicz00oceanstore}.  to cope with billions of concurrent peers \cite{osokine02distnetworks}, \cite{kubiatowicz00oceanstore}.
518    
519  To end user, biggest difference between these systems is how data lookups are performed. Loosely  To end user, biggest difference between these systems is how data lookups are performed. Loosely
520  structured systems provide more richer and user friendly way of searching data as they  structured systems provide a more rich and user friendly way of searching data as they
521  have support for keyword search. On the other hand, tightly structured systems support  have support for keyword search than tightly structured systems. On the other hand, tightly structured
522  only exact key lookups as each data item is identified by globally unique keys.  systems support only exact key lookups as each data item is identified by globally unique keys.
523    
524  In the end, both systems have open problems and issues. We will discuss these aspects in more detail in  In the end, both systems have open problems and issues. We will discuss these aspects in more detail in
525  chapter 3. Table \ref{table_comparison_approach} lists the key differences between loosely structured  chapter 3. Table \ref{table_comparison_approach} lists the key differences between the loosely structured
526  approach and tightly structured approach.  approach and the tightly structured approach.
527    
528    
529  \scriptsize  \scriptsize
# Line 625  approach and tightly structured approach Line 614  approach and tightly structured approach
614    
615  Table \ref{table_Peer-to-Peer_algorithms} lists proposed Peer-to-Peer algorithms  Table \ref{table_Peer-to-Peer_algorithms} lists proposed Peer-to-Peer algorithms
616  and their key properties with regard to performance and scalability. List  and their key properties with regard to performance and scalability. List
617  includes algorithms from both loosely and tightly structured approaches. However, majority of the algorithms  includes algorithms from both loosely and tightly structured approaches. The list doesn't
 listed above belongs to tightly structured approach since there has been active  
 research being pursued towards tightly structured approach lately. List doesn't  
618  include \emph{all} proposed Peer-to-Peer algorithms. Only the ones which already have  include \emph{all} proposed Peer-to-Peer algorithms. Only the ones which already have
619  been widely deployed in real life, or the ones which may be promising in the future  been widely deployed in real life, or the ones which may be promising in the future
620  Peer-to-Peer systems are included in this thesis.  Peer-to-Peer systems are included in this thesis.
621    
622  We decided to follow the guidelines from \cite{kaashoek03koorde} when  We decided to follow the guidelines from \cite{kaashoek03koorde} in measuring
623  measuring properties of different Peer-to-Peer systems. However, we dropped  the properties of different Peer-to-Peer systems. However, we dropped
624  out fault tolerance and load balancing properties, since they are hard to measure  out fault tolerance and load balancing properties, since they are hard to measure
625  in face of real life requirements. Additionally, however, we decided to include  in face of real life requirements. Additionally, however, we decided to include
626  the number of \emph{real} network connections for each peer in the overlay.  the number of \emph{real} network connections for each peer in the overlay.
# Line 641  the number of \emph{real} network connec Line 628  the number of \emph{real} network connec
628  Here, we describe the listed properties of Peer-to-Peer algorithms:  Here, we describe the listed properties of Peer-to-Peer algorithms:
629    
630  \begin{itemize}  \begin{itemize}
631  \item \textbf{Lookup}: number of messages required when a data lookup is performed  \item \textbf{Lookup}: the number of messages required when a data lookup is performed
632  \item \textbf{Space}: number of neighbors which peers knows about (neighbors)  \item \textbf{Space}: the number of neighbors which peers knows about (neighbors)
633  \item \textbf{Insert/delete}: number of messages required when a peer joins or leaves the network  \item \textbf{Insert/delete}: the number of messages required when a peer joins or leaves the network
634   \item \textbf{Number of network connections}: number of concurrent network connections required to maintain correct neighbor information   \item \textbf{Number of network connections}: the number of concurrent network connections required to maintain correct neighbor information
635  \end{itemize}  \end{itemize}
636    
637  \scriptsize  \scriptsize
# Line 679  Here, we describe the listed properties Line 666  Here, we describe the listed properties
666  \parbox{37pt}{$O$($d$)} &  \parbox{37pt}{$O$($d$)} &
667  \parbox{37pt}{$O(dn^{\frac{1}{d}})$} &  \parbox{37pt}{$O(dn^{\frac{1}{d}})$} &
668  \parbox{85pt}{2$d$} &  \parbox{85pt}{2$d$} &
669  \parbox{85pt}{The performance of system may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, where $d$ is the dimension of virtual key space}  \parbox{85pt}{System performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, where $d$ is the dimension of virtual key space}
670  \\ \hline  \\ \hline
671    
672  \parbox{37pt}{Chord \cite{stoica01chord}} &  \parbox{37pt}{Chord \cite{stoica01chord}} &
# Line 687  Here, we describe the listed properties Line 674  Here, we describe the listed properties
674  \parbox{37pt}{$O(\log{n}$} &  \parbox{37pt}{$O(\log{n}$} &
675  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
676  \parbox{85pt}{2$(\log{n})$} &  \parbox{85pt}{2$(\log{n})$} &
677  \parbox{85pt}{The performance of system may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner}  \parbox{85pt}{System performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner}
678  \\ \hline  \\ \hline
679    
680    
# Line 723  Here, we describe the listed properties Line 710  Here, we describe the listed properties
710  \parbox{37pt}{$O$($\sqrt{n}$)} &  \parbox{37pt}{$O$($\sqrt{n}$)} &
711  \parbox{37pt}{$O(1)$} &  \parbox{37pt}{$O(1)$} &
712  \parbox{85pt}{$\frac{n}{\sqrt{n}} + c*(\sqrt{n}-1) + \frac{Totalnumber of files}{\sqrt{n}}$, where n is the number of peers and c the number of contacts/foreign affinity group} &  \parbox{85pt}{$\frac{n}{\sqrt{n}} + c*(\sqrt{n}-1) + \frac{Totalnumber of files}{\sqrt{n}}$, where n is the number of peers and c the number of contacts/foreign affinity group} &
713  \parbox{85pt}{Insert/delete overhead is constant and performed in the background, the performance of system may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner}  \parbox{85pt}{Insert/delete overhead is constant and performed in the background, system performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner}
714  \\ \hline  \\ \hline
715    
716  \parbox{37pt}{Koorde \cite{kaashoek03koorde}} &  \parbox{37pt}{Koorde \cite{kaashoek03koorde}} &
# Line 748  Here, we describe the listed properties Line 735  Here, we describe the listed properties
735  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
736  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
737  \parbox{85pt}{$(2^{b - 1})\frac{\log{n}}{b}$, where $b$ is a configurable parameter for tuning digit-fixing properties (routing table)} &  \parbox{85pt}{$(2^{b - 1})\frac{\log{n}}{b}$, where $b$ is a configurable parameter for tuning digit-fixing properties (routing table)} &
738  \parbox{85pt}{The performance of system performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, based on Plaxton's algorithm}  \parbox{85pt}{System performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, based on Plaxton's algorithm}
739  \\ \hline  \\ \hline
740    
741    
# Line 797  Here, we describe the listed properties Line 784  Here, we describe the listed properties
784  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
785  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
786  \parbox{85pt}{$2k+2+f$, where k = long range connections, 2 = peer's neighbors, f = fault tolerance connections)} &  \parbox{85pt}{$2k+2+f$, where k = long range connections, 2 = peer's neighbors, f = fault tolerance connections)} &
787  \parbox{85pt}{Space can be also $O(1)$. Additional space of can be used as a lookahead list for better performance, not necessarily fault-tolerant because of constant degree of neighbors}  \parbox{85pt}{Space can also be $O(1)$. Additional space of can be used as a lookahead list for better performance}
788  \\ \hline  \\ \hline
789    
790  \parbox{37pt}{SWAN \cite{bonsma02swan}} &  \parbox{37pt}{SWAN \cite{bonsma02swan}} &
# Line 814  Here, we describe the listed properties Line 801  Here, we describe the listed properties
801  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
802  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
803  \parbox{85pt}{$(2^{b - 1})\frac{\log{n}}{b}$, where $b$ is a configurable parameter for tuning digit-fixing properties (routing table)} &  \parbox{85pt}{$(2^{b - 1})\frac{\log{n}}{b}$, where $b$ is a configurable parameter for tuning digit-fixing properties (routing table)} &
804  \parbox{85pt}{The system performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, based on Plaxton's algorithm}  \parbox{85pt}{System performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, based on Plaxton's algorithm}
805  \\ \hline  \\ \hline
806    
807  \parbox{37pt}{Viceroy \cite{malkhi02viceroy}} &  \parbox{37pt}{Viceroy \cite{malkhi02viceroy}} &
# Line 822  Here, we describe the listed properties Line 809  Here, we describe the listed properties
809  \parbox{37pt}{$O(1)$} &  \parbox{37pt}{$O(1)$} &
810  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
811  \parbox{85pt}{11} &  \parbox{85pt}{11} &
812  \parbox{85pt}{The system performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, not necessarily fault-tolerant because of constant degree of neighbors}  \parbox{85pt}{System performance may decrease if peers are not homogeneous and peers join and leave the system in a dynamic manner, not necessarily fault-tolerant because of constant degree of neighbors}
813  \\ \hline  \\ \hline
814    
815    
# Line 838  Here, we describe the listed properties Line 825  Here, we describe the listed properties
825  \chapter{Open Problems in Peer-to-Peer}  \chapter{Open Problems in Peer-to-Peer}
826    
827  In this chapter, we discuss open problems in Peer-to-Peer research. We describe  In this chapter, we discuss open problems in Peer-to-Peer research. We describe
828  open problems and their proposed solutions. Then, we list all issues in  the open problems and their proposed solutions. Then, we list all issues in
829  tables; we list description of the problem, solution and comments on that  tables. Note that the open problems list considered here is not meant
 specific open problem. Note that open problems list considered here is not meant  
830  to be an exhaustive survey of \emph{all} open problems in Peer-to-Peer domain;  to be an exhaustive survey of \emph{all} open problems in Peer-to-Peer domain;
831  we focus our attention to security, scalability, usability and performance issues  we focus our attention to security, scalability, usability and performance issues
832  only.  only.
833    
834  \section{Overview}  \section{Overview}
835    
836  Partly due to the non-maturity of modern Peer-to-Peer technology, it has several  Partly due to the non-maturity of modern Peer-to-Peer technology, there are several
837  open problems to be solved. The most severe problems are related to performance, scalability, usability  open problems to be solved. The most severe problems are related to performance, scalability, usability
838  and security. More important, many techniques developed for traditional distributed  and security. Also, many techniques developed for traditional distributed
839  systems may no longer apply with Peer-to-Peer systems. Therefore, new solutions are  systems may no longer apply with Peer-to-Peer systems. Therefore, new solutions are
840  needed to make Peer-to-Peer systems more secure and efficient.  needed to make Peer-to-Peer systems more secure and efficient.
841    
842  Both loosely structured and tightly structured approach have their own specific problems.  Both the loosely structured and the tightly structured approach have their own specific problems.
843  Since Napster \cite{napsterurl} and Gnutella \cite{gnutellaurl} was first introduced  Since Napster \cite{napsterurl} and Gnutella \cite{gnutellaurl} were first introduced
844  to public, researchers' main concern has been the scalability problem of loosely structured  to the public, researchers' main concern has been the scalability problem of the loosely structured
845  approach. However, people often misunderstand the scalability problem of loosely structured  approach. However, people often misunderstand the scalability problem of the loosely structured
846  approach; \emph{network} of loosely structured systems is scalable, but the \emph{data lookup model} is not.  approach; \emph{network} of loosely structured systems is scalable, but the \emph{data lookup model} is not.
847  The main concern of tightly structured system is to make overlay's data lookup process  The main concern of the tightly structured system is to make overlay's data lookup process
848  more fault tolerant against hostile attacks. Other key problems in tightly structured  more fault tolerant against hostile attacks. Other key problems in tightly structured
849  systems are the lack of keyword searches, support for heterogeneous peers and load balancing  systems are the lack of keyword searches, support for heterogeneous peers and load balancing
850  \cite{balakrishanarticle03lookupp2p}.  \cite{balakrishanarticle03lookupp2p}.
851    
852  To make Peer-to-Peer systems even more popular (e.g., in industry), Peer-to-Peer domain  To make Peer-to-Peer systems even more popular (e.g., in industry), Peer-to-Peer domain
853  needs better infrastructures to deal with security issues. There has been done some  needs better infrastructures to deal with security issues. Some research has been done regarding
854  research regarding anonymity, access control, data availability and data integrity but as  anonymity, access control, data availability and data integrity but as
855  we state in the following sections, much more research work is required to solve these issues.  we state in the following sections, much more research work is required to solve these issues.
856    
857  \section{Security problems in Peer-to-Peer}  \section{Security problems in Peer-to-Peer}
# Line 878  There are five known attack models again Line 864  There are five known attack models again
864  Fail-stop attack, Spam attack \cite{naor03simpledht}, Byzantine attack \cite{357176} and \cite{296824}, and  Fail-stop attack, Spam attack \cite{naor03simpledht}, Byzantine attack \cite{357176} and \cite{296824}, and
865  general Distributed Denial of Service attack.  general Distributed Denial of Service attack.
866    
867  In Sybil attack model, hostile entity presents multiple  In Sybil attack model, a hostile entity presents multiple
868  entities. Therefore, one hostile entity can control a large fraction of the Peer-to-Peer system. Optimal  entities. Therefore, one hostile entity can control a large fraction of the Peer-to-Peer system. Possible solution to
869  possible solution to Sybil attack would be that system could \emph{distinct} entities of the system reliably. Unfortunately,  Sybil attack would be that the system could distinguish entities of the system reliably. Unfortunately,
870  currently there are no realizable techniques for this task. Partial solutions for Sybil attack is to replicate  currently there are no realizable techniques for this task. Partial solutions for Sybil attack is to replicate
871  and fragment data randomly among several participating peer. However, both suggestions assume that two different  and fragment data randomly among several participating peers. However, both suggestions assume that two different
872  remote entities are actually different; Sybil attacks are still possible and therefore, would need centralized  remote entities are actually different; Sybil attacks are still possible and therefore would need centralized
873  authority for reliable authentication. As author argues in \cite{douceur02sybil}, without centralized authority,  authority for reliable authentication. As the author argues in \cite{douceur02sybil}, without centralized authority,
874  Sybil attacks are always possible in Peer-to-Peer system except under extreme and unrealistic assumptions of  Sybil attacks are always possible in a Peer-to-Peer system except under extreme and unrealistic assumptions of
875  resource parity and coordination among entities.  resource parity and coordination among entities.
876    
877  In random fail-stop model, cited in \cite{naor03simpledht}, faulty peer is deleted from the Peer-to-Peer system.  In random fail-stop model, cited in \cite{naor03simpledht}, a faulty peer is deleted from the Peer-to-Peer system.
878  The reason for faultiness of peer can be a software failure, a hostile attack, or external threat such as virus or  The reason for the faultiness of a peer can be a software failure, a hostile attack, or an external threat such as virus or
879  trojan. Closely related to fail-stop model is the Byzantine attack model  trojan. The Byzantine attack model \cite{357176} closely related to fail-stop model. Byzantine model can be seen as more
880  \cite{357176}. Byzantine model can be seen as more severe than fail-stop model as there are no restrictions over  severe than fail-stop model as there are no restrictions over the behavior of faulty peers. Practical but partial
881  the behavior of faulty peers. Practical, but partial solution for Byzantine failures has been proposed by Castro et  solution for Byzantine failures has been proposed by Castro et al. \cite{296824}.
 al. \cite{296824}.  
882    
883  Spam generating attack is another known attack model against Peer-to-Peer system. In Spam  Spam generating attack is another known attack model against Peer-to-Peer system. In Spam
884  attack, hostile or faulty peer may produce false information of the data, or refuses/is not able to reply to requests.  attack, a hostile or faulty peer may produce false information of the data, or refuses to (or is not able to) reply to requests.
885  Possible solution against this attack is that peer should not trust to single entity. Instead, peer should get  Possible solution against this attack is that peer should not trust a single entity. Instead, a peer should get
886  information from multiple entities and trust on majority's opinion. This method requires more messages to be  information from multiple entities and trust on the majority's opinion. This method requires more messages to be
887  sent to network while increasing system load. However, if Spam attack is combined with Sybil attack, obviously  sent to the network while increasing the system load. However, if the Spam attack is combined with the Sybil attack, obviously
888  previously mentioned solution doesn't work. Naor et al. \cite{naor03simpledht} have proposed a partial solution against Spam attack  the previously mentioned solution doesn't work. Naor et al. \cite{naor03simpledht} have proposed a partial solution against Spam attack
889  in \emph{faulty} peer environment (not hostile).  in \emph{faulty} peer environment (not hostile).
890    
891  Traditional overloading of targeted peers is best known form of distributed Denial of Service attack (DDoS). For example,  Traditional overloading of targeted peers is the best known form of distributed Denial of Service attack (DDoS). For example,
892  hostile entity can attempt to burden targeted peers with garbage network packets. As an implication, peers may act  a hostile entity can attempt to burden targeted peers with garbage network packets. As an implication, peers may act
893  incorrectly or stop working. DDoS attack may be very severe, especially if rate of replication and caching  incorrectly or stop working. DDoS attack may be very severe, especially if the rate of replication and caching
894  in Peer-to-Peer system is low. This may lead to data loss in the Peer-to-Peer system. Daswani et al.  in the Peer-to-Peer system is low. This may lead to data loss in the Peer-to-Peer system. Daswani et al.
895  \cite{daswani02queryflooddos} have done research regarding to this subject. Authors suggest efficient load balancing  \cite{daswani02queryflooddos} suggest efficient load balancing
896  policies for Peer-to-Peer system in order to prevent massive system failures. Sit et al. \cite{sit02securitycons}  policies for Peer-to-Peer system in order to prevent massive system failures. Sit et al. \cite{sit02securitycons}
897  suggest that identifier assignment algorithm for peers would assign identifier with respect to network topology  suggest that identifier assignment algorithm for peers would assign identifier with respect to network topology
898  and replicas should be located physically to different locations.  and replicas should be located physically to different locations.
899    
900  As stated in \cite{naor03simpledht}, an important aspect is that when it comes to general security aspects and  As stated in \cite{naor03simpledht}, an important aspect is that when it comes to general security aspects and
901  Byzantine faults in any Peer-to-Peer system, there should be a clear distinction between attacks on the  Byzantine faults in any Peer-to-Peer system, there should be a clear distinction between attacks on the
902  algorithms assuming the construction of overlay is correct, and attacks on the construction itself. Clearly, Sybil  algorithms assuming the construction of the overlay is correct, and attacks on the construction itself. Clearly, Sybil
903  and Spam attack belongs to the first category, and rest of the attacks to the latter category.  and Spam attacks belong to the first category, and the rest of the attacks to the latter category.
904    
905  \subsection{Trust, data authenticity and integrity}  \subsection{Trust, data authenticity and integrity}
906    
907  Trust in Peer-to-Peer systems is based on \emph{reputation}. Proposed reputation methods focus either  Trust in Peer-to-Peer systems is based on \emph{reputation}. Proposed reputation methods focus either
908  on the semantic properties, or the data management properties of the trust model. Some research has been  on the semantic properties or the data management properties of the trust model. Some research has been
909  done on reputation models in Peer-to-Peer systems, such as \cite{aberer01trust}, \cite{cornelli02reputableservents}.  done on reputation models in Peer-to-Peer systems, such as \cite{aberer01trust}, \cite{cornelli02reputableservents}.
910  One implementation include Advogato \cite{advogatourl}. None of the current proposals or implementations  One implementation include Advogato \cite{advogatourl}. None of the current proposals or implementations
911  based on reputation address trust in a reliable, practical way.  based on reputation address trust in a reliable, practical way.
912    
913  Optimal solution for trust in Peer-to-Peer systems would be certificate based security models.  Optimal solution for trust in Peer-to-Peer systems would be certificate based security models.
914  Quite recently, widely used Public Key Infrastructure (PKI) has been deployed in distributed  Quite recently, widely used Public Key Infrastructure (PKI) has been deployed in distributed
915  systems \cite{rivest96sdsi}, \cite{spkiworkinggroup}. PKI is reliable technology for securing  systems \cite{rivest96sdsi}, \cite{spkiworkinggroup}. PKI is a reliable technology for securing
916  data in rather \emph{static} computing systems, such as in the Internet. However, in Peer-to-Peer  data in rather \emph{static} computing systems, such as the Internet. However, in Peer-to-Peer
917  network, the problem of key based security mechanism is the maintenance of the keys as participating  networks, the problem of key based security mechanism is the maintenance of the keys as participating
918  peers constantly join and leave the system. Specifically, the distribution of key changes comes an essential  peers constantly join and leave the system. Specifically, the distribution of key changes becomes an essential
919  problem in ad hoc environments. These include revocation of keys and new key distribution in hostile  problem in ad hoc environments. These include revocation of keys and new key distribution in hostile
920  environment.  environment.
921    
922  ConChord \cite{ajmani02conchord} is the first Peer-to-Peer system which has a support for PKI based  ConChord \cite{ajmani02conchord} is the first Peer-to-Peer system which has a support for PKI based
923  security infrastructure. Still, however, ConChord \cite{ajmani02conchord} is in early phase of development and lacks of  security infrastructure. Still, however, ConChord \cite{ajmani02conchord} is in early phase of development and lacks
924  important features of PKI to be fully usable yet. Furthermore, the hierarchy of Simple Distributed Security Infrastructure  important features of PKI to be fully usable yet. Furthermore, the hierarchy of Simple Distributed Security Infrastructure
925  (SDSI) \cite{rivest96sdsi} and Simple Public Key Infrastructure (SPKI) \cite{spkiworkinggroup} may be a problem for  (SDSI) \cite{rivest96sdsi} and Simple Public Key Infrastructure (SPKI) \cite{spkiworkinggroup} may be a problem for
926  Peer-to-Peer systems, in which hierarchy is intentionally missing.  Peer-to-Peer systems, in which hierarchy is intentionally missing.
# Line 944  For data integrity, on the other hand, t Line 929  For data integrity, on the other hand, t
929  \cite{fips-sha-1}, their variations \cite{merkle87hashtree} and implementation techniques \cite{mohr02thex},  \cite{fips-sha-1}, their variations \cite{merkle87hashtree} and implementation techniques \cite{mohr02thex},
930  are efficient and reliable methods for identifying the integrity of data in Peer-to-Peer systems. One  are efficient and reliable methods for identifying the integrity of data in Peer-to-Peer systems. One
931  possible application of cryptographic content hashes may be in peer identifier creation process, in which  possible application of cryptographic content hashes may be in peer identifier creation process, in which
932  IP address of peer can be verified by the other peer. This is one form of \emph{self-certifying data}.  the IP address of a peer can be verified by the other peer. This is one form of \emph{self-certifying data}.
933    
934    
935  \subsection{Anonymity}  \subsection{Anonymity}
936    
937  According to \cite{dingledine00free}, there exists several kinds of anonymity. Author-anonymity is a form  According to \cite{dingledine00free}, there exist several kinds of anonymity. Author-anonymity is a form
938  of anonymity in which no one can link author to a specific document. In publisher-anonymity system,  of anonymity in which no one can link the author to a specific document. In publisher-anonymity system,
939  no one is able to link publisher to a specific document. Reader-anonymity means that a specific  no one is able to link the publisher to a specific document. Reader-anonymity means that a specific
940  document cannot be linked to document's readers. This form of anonymity protects the privacy of  document cannot be linked to the readers of a document. This form of anonymity protects the privacy of
941  users of the system. Furthermore, peer-anonymity means that no peer can be linked to a specific document, i.e.,  the users of the system. Furthermore, peer-anonymity means that no peer can be linked to a specific document, i.e.,
942  no one is able to determine the peer, where the document was originally published. Document-anonymity  no one is able to determine the peer, where the document was originally published. Document-anonymity
943  means that peer doesn't know which data it is currently hosting. Finally, query-anonymity is a form  means that a peer doesn't know which data it is currently hosting. Finally, query-anonymity is a form
944  of document-anonymity; when other peers performs data lookups, peer doesn't know which data it serves  of document-anonymity; when other peers perform data lookups, a peer doesn't know which data it serves
945  to the data lookup originators. As the authors cite, some forms of anonymity may imply each other and  to the data lookup originators. As the authors cite in \cite{dingledine00free}, some forms of anonymity
946  possible issues raised by this property is one area of future work.  may imply each other and possible issues raised by this property is one area of future work.
947    
948  With regard to anonymity in Peer-to-Peer systems, there has been done much research work both at network  With regard to anonymity in Peer-to-Peer systems, much research has been done both at the network
949  level layer \cite{tarzan:ccs9} and at application level layer \cite{reiter98crowds}, \cite{mixminionurl}.  level layer \cite{tarzan:ccs9} and at the application level layer \cite{reiter98crowds}, \cite{mixminionurl}.
950  Research on anonymity outside of Peer-to-Peer context have been done also \cite{352607}, \cite{293447}.  Anonymity outside of Peer-to-Peer context has also been researched \cite{352607}, \cite{293447}.
951    
952  Obviously, providing several types of anonymity, it often conflicts with other key properties of  Obviously, existance of several types of anonymity often conflicts with other key properties of
953  Peer-to-Peer system. Let's consider anonymity and efficient data lookup. In efficient data lookup, we must know  Peer-to-Peer systems. Let us consider anonymity and efficient data lookup. In efficient data lookup, we must know
954  the peers responsible for given data in Peer-to-Peer system. Of course, when we know the peers responsible  the peers responsible for given data. Of course, when we know the peers responsible
955  for the data, the anonymity of peer is lost. Fortunately, there are partial solutions to previously  for the data, the anonymity of peer is lost. Fortunately, there are partial solutions to previously
956  mentioned situations, such as pseudonym which is a partial form of anonymity. For instance, pseudonym can be used for  mentioned situations, such as pseudonymity which is a partial form of anonymity. For instance, pseudonymity can be used for
957  addressing peer-anonymity by providing anonymous-like identifiers to peers (e.g., peer identifiers of tightly  addressing peer-anonymity by providing anonymous-like identifiers to peers (e.g., peer identifiers of a tightly
958  structured system).  structured system).
959    
960  Anonymity is widely used in those Peer-to-Peer system in which data publication and non-censorship are important properties  Anonymity is widely used in Peer-to-Peer system in which data publication and non-censorship are important properties
961  of the system. These include  of the system. These include
962  Freenet \cite{clarke00freenet}, Publius \cite{pub00}, Free haven \cite{dingledine00free}, Crowds \cite{reiter98crowds},  Freenet \cite{clarke00freenet}, Publius \cite{pub00}, Free Haven \cite{dingledine00free}, Crowds \cite{reiter98crowds},
963  Tangler \cite{502002} and upcoming Mnet \cite{mneturl}. Forwarding proxies are used in Freenet, Crowds and  Tangler \cite{502002} and upcoming Mnet \cite{mneturl}. Forwarding proxies are used in Freenet, Crowds and
964  Free Haven in order to provide various types of anonymity. Tangler and Publius uses cryptographic  Free Haven in order to provide various types of anonymity. Tangler and Publius use cryptographic
965  sharing methods to split data into fragments \cite{Shamir1979a}. Mix mailer networks, such as  sharing methods to split data into fragments \cite{Shamir1979a}. Mix mailer networks, such as
966  \cite{mixminionurl}, are commonly used in distributed systems, which are able to provide some level  \cite{mixminionurl}, are commonly used in distributed systems, which are able to provide some level
967  of anonymity.  of anonymity.
968    
969  Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no  Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no
970  such system which is able to provide all kinds of anonymity as listed above. Specifically, the conflicts  such system which is able to provide all kinds of anonymity as listed above. Specifically, the conflicts
971  between anonymity and other Peer-to-Peer system properties requires more research work.  between anonymity and other Peer-to-Peer system properties require more research work.
972    
973    
974  \subsection{Access control}  \subsection{Access control}
975    
976  Any distributed computing system must support different levels of access control. For instance, in Peer-to-Peer  Any distributed computing system must support different levels of access control. For instance, in a Peer-to-Peer
977  system, we may want to restrict the accessibility of data to only limited amount of participating peers. Yet, Peer-to-Peer  system, we may want to restrict the accessibility of data to only limited amount of participating peers. Yet, Peer-to-Peer
978  systems do not have working and distributed access control scheme. Moreover,  systems do not have a working and distributed access control scheme. Moreover,
979  there has been a lot of violation of copyright laws by users of Peer-to-Peer file sharing systems. As a  there has been a lot of violations of copyright laws by users of Peer-to-Peer file sharing systems. As a
980  consequence, some law suits have been created against the companies who have build popular file-sharing programs.  consequence, some law suits have been filed against the companies who have build popular file-sharing programs.
981    
982  To our knowledge, Nejdl et al. \cite{nejdl03accesscontrol} have proposed very recently first practical solution to access  To our knowledge, Nejdl et al. \cite{nejdl03accesscontrol} have very recently proposed the first practical solution to access
983  control problem in Peer-to-Peer systems. They use Resource Description Framework (RDF) \cite{w3rdfurl} based  control problem in Peer-to-Peer systems. They use Resource Description Framework (RDF) \cite{w3rdfurl} based
984  schema policies to restrict access to certain data. Unfortunately, their current early prototype version works only in  schema policies to restrict access to certain data. Unfortunately, their current early prototype version only works in
985  loosely structured systems.  loosely structured systems.
986    
987    
988  \subsection{Hostile entities}  \subsection{Hostile entities}
989    
990  One serious problem in Peer-to-Peer system is lack of ability to identify hostile entities trustworthy.  One serious problem in Peer-to-Peer systems is the inability to identify hostile entities as trustworthy.
991  Possible solutions include self-monitoring systems \cite{zhang03somo}, maintaining system invariants as  Possible solutions include self-monitoring systems \cite{zhang03somo}, maintaining system invariants as
992  proposed in \cite{sit02securitycons}, distributed and secure peer identifier assignment  proposed in \cite{sit02securitycons}, distributed and secure peer identifier assignment
993  \cite{castro02securerouting}, \cite{clarke00freenet} and self-certifying data using cryptographic  \cite{castro02securerouting}, \cite{clarke00freenet} and self-certifying data using cryptographic
994  content hashes (e.g., SHA-1 \cite{fips-sha-1}). Identification of hostile entities is essential in tightly structured  content hashes (e.g., SHA-1 \cite{fips-sha-1}). Identification of hostile entities is essential in the tightly structured
995  approach, in which fundamental (and implicit) assumption is that there is a random, uniform distribution  approach, in which the fundamental (and implicit) assumption is that there is a random, uniform distribution
996  of peer identifiers that cannot be controlled by hostile entity.  of peer identifiers that cannot be controlled by a hostile entity.
997    
998  Of course centralized authorities could be used for assignment of peer identifiers, but they may not be suitable  Naturally, centralized authorities could be used for the assignment of peer identifiers, but they may not be suitable
999  for ad hoc Peer-to-Peer environrment and have property of single point of failure. Moreover, distributed peer  for ad hoc Peer-to-Peer environrment and have property of single point of failure. Moreover, distributed peer
1000  identification assignment can be problematic as long as Sybil attack \cite{douceur02sybil} remains unsolved.  identification assignment can be problematic as long as Sybil attack \cite{douceur02sybil} remains unsolved.
1001  However, there are some partial solutions for controlling the rate at which hostile entity is able to obtain peer  However, there are some partial solutions for controlling the rate at which hostile entity is able to obtain peer
1002  identifier, such as crypto-based puzzles \cite{juels99clientpuzzles}.  identifier, such as crypto-based puzzles \cite{juels99clientpuzzles}.
1003    
1004  In the end, none of the previously mentioned solutions are able to identify hostile entities safely.  In the end, none of these problems solutions are able to identify hostile entities safely.
1005    
1006    
1007  \subsection{Secure query routing}  \subsection{Secure query routing}
# Line 1024  In the end, none of the previously menti Line 1009  In the end, none of the previously menti
1009  Much work has been done on secure routing, especially related to tightly structured systems. In  Much work has been done on secure routing, especially related to tightly structured systems. In
1010  \cite{castro02securitystructured} and \cite{castro02securerouting}, authors suggest the usage  \cite{castro02securitystructured} and \cite{castro02securerouting}, authors suggest the usage
1011  of constrained routing tables and diverse routes, and detection of faults during query routing.  of constrained routing tables and diverse routes, and detection of faults during query routing.
1012  Additionally, authors present in \cite{castro02securerouting} an important aspect of tightly structured approach with regard  Additionally, authors present in \cite{castro02securerouting} an important aspect of the tightly structured approach with regard
1013  to fault-tolerant query routing: the probability of routing successfully between to arbitrary  to fault-tolerant query routing: the probability of routing successfully between to arbitrary
1014  correct peers, when a fraction $f$ of the other peers are faulty or hostile, is only $(1-f)^{h-1}$, where  correct peers, when a fraction $f$ of the other peers are faulty or hostile, is only $(1-f)^{h-1}$, where
1015  $h$ is the number of hops in the overlay.  $h$ is the number of hops in the overlay.
# Line 1045  prove the lower and upper bounds for spa Line 1030  prove the lower and upper bounds for spa
1030  Peer-to-Peer system. They show that to provide high degree of fault tolerance and efficiency, each  Peer-to-Peer system. They show that to provide high degree of fault tolerance and efficiency, each
1031  participating peer must maintain average of $O(\log{n})$ neighbors.  participating peer must maintain average of $O(\log{n})$ neighbors.
1032    
1033  Fiat et al. in \cite{fiat02censorship}, \cite{saia02dynamicfaultcontentnetwork} and Datar in \cite{datar02butterflies}    Fiat et al. in \cite{fiat02censorship, saia02dynamicfaultcontentnetwork} and Datar in \cite{datar02butterflies}  
1034  describe tightly structured overlay with analytical results in the presence of hostile entities. However,  describe tightly structured overlay with analytical results in the presence of hostile entities. However,
1035  none of these proposals address an efficient, dynamic tightly structured overlay and multiple rounds  none of these proposals address an efficient, dynamic tightly structured overlay and multiple rounds
1036  of hostile attack. Also, above mentioned proposals are not very efficient. In \cite{fiat02censorship}, each peer  of hostile attack. Also, above mentioned proposals are not very efficient. In \cite{fiat02censorship}, each peer
1037  must maintain information of $O(\log^3{n})$ other peers, and in \cite{datar02butterflies}, $O(\log^2{n})$ is required.  must maintain information of $O(\log^3{n})$ other peers, and in \cite{datar02butterflies}, $O(\log^2{n})$ is required.
1038    
1039  Finally, Ratnasamy and Gavoille \cite{ratnasamy02routing}, \cite{gavoille01routing} list several open problems  Finally, Ratnasamy and Gavoille \cite{ratnasamy02routing, gavoille01routing} list several open problems
1040  regarding routing in distributed networks. Obviously, more research is required in order to provide secure  regarding routing in distributed networks. Obviously, more research is required in order to provide secure
1041  data lookup routing possible in Peer-to-Peer networks.  data lookup routing possible in Peer-to-Peer networks.
1042    
# Line 1062  Ross Lee Graham lists several external t Line 1047  Ross Lee Graham lists several external t
1047  the list includes viruses and trojans. Currently, there are not even partial solutions  the list includes viruses and trojans. Currently, there are not even partial solutions
1048  to the problems mentioned above. General robustness properties of Peer-to-Peer system is able to  to the problems mentioned above. General robustness properties of Peer-to-Peer system is able to
1049  deal with software failures and hostile attack, but fault tolerance against external threats is unknown.  deal with software failures and hostile attack, but fault tolerance against external threats is unknown.
1050  The reason for this is that there are no experiences on these kinds of attacks. Possible solution  The reason for this is that there are no experience on these kinds of attacks. Possible solution
1051  would be distributed anti-virus software, but much more intensive research is required until  would be distributed anti-virus software, but much more intensive research is required until
1052  this kind of solution would be applicable.  this kind of solution would be applicable.
1053    
# Line 1074  In this section, we discuss performance Line 1059  In this section, we discuss performance
1059  \subsection{Efficient data lookup}  \subsection{Efficient data lookup}
1060    
1061  The most intensive research in Peer-to-Peer domain has been focused on efficient data lookup methods,  The most intensive research in Peer-to-Peer domain has been focused on efficient data lookup methods,
1062  especially with loosely structured approach. In addition to ''super-peer'' method presented in chapter  especially with the loosely structured approach. In addition to ''super-peer'' method presented in chapter
1063  2, there has been other improvements also.  2, there has been other improvements also.
1064  In iterative deepening  In iterative deepening
1065  \cite{yang02improvingsearch}, multiple BFS searches are initiated  \cite{yang02improvingsearch}, multiple BFS searches are initiated
# Line 1116  it adapts well with varying usage patter Line 1101  it adapts well with varying usage patter
1101    
1102  Since tightly structured systems have efficient data lookup at the application level overlay,  Since tightly structured systems have efficient data lookup at the application level overlay,
1103  current research efforts are focused on proximity based data lookup. In proximity based data lookup,  current research efforts are focused on proximity based data lookup. In proximity based data lookup,
1104  peers try to choose routing-tables entries referring to other peers that are \emph{nearby} in the  peers try to choose entries of routing-tables referring to other peers that are \emph{nearby} in the
1105  underlying network. In this way, tightly structured systems are able to decrease actual  underlying network. In this way, tightly structured systems are able to decrease actual
1106  lookup \emph{latency}. CAN \cite{ratnasamy01can}, Kademlia \cite{maymounkov02kademlia},  lookup \emph{latency}. CAN \cite{ratnasamy01can}, Kademlia \cite{maymounkov02kademlia},
1107  Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry} have advanced heuristics for  Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry} have advanced heuristics for
1108  proximity based routing. Additionally, most recent version of Chord uses proximity based  proximity based routing. Additionally, most recent version of Chord uses proximity based
1109  routing inspired by Karger and Ruhl \cite{karger02findingnearest}. SkipNet \cite{harvey03skipnet1}  routing inspired by Karger and Ruhl \cite{karger02findingnearest}. SkipNet \cite{harvey03skipnet1}
1110  uses combination of proximity and application level overlay routing when performing data  uses a combination of proximity and application level overlay routing when performing data
1111  lookups. Authors call this feature \emph{constrained load balancing}.  lookups. Authors call this feature \emph{constrained load balancing}.
1112    
1113  Additional research related to proximity based routing include \cite{karger02findingnearest},  Additional research related to proximity based routing include \cite{karger02findingnearest, hildrum02distributedobject,
1114  \cite{hildrum02distributedobject}, \cite{brinkmann02compactplacement}, \cite{rhea02probabilistic},  brinkmann02compactplacement, rhea02probabilistic, castro02networkproximity, ng02predicting, pias03lighthouse}.
 \cite{castro02networkproximity}, \cite{ng02predicting} and \cite{pias03lighthouse}.  
1115    
1116  \subsection{Fast and usable search}  \subsection{Fast and usable search}
1117    
1118  To make Peer-to-Peer systems even more popular (and usable), these systems have to support flexible, efficient  To make Peer-to-Peer systems even more popular (and usable), these systems have to support flexible, efficient
1119  and easy to use search methods. For instance, Internet's perhaps the most important feature  and easy search methods. For instance, Internet's perhaps the most important feature
1120  is the ability to perform keyword searches (e.g., Google \cite{googleurl}). Currently, only loosely  is the ability to perform keyword searches (e.g., Google \cite{googleurl}). Currently, only loosely
1121  structured systems are able to carry out this requirement. Unfortunately, as discussed in this text,  structured systems are able to carry out this requirement. Unfortunately, as discussed in this text,
1122  the data lookup model of loosely structured approach doesn't scale. Thus, research efforts have  the data lookup model of the loosely structured approach doesn't scale. Thus, research efforts have
1123  been focused on tightly structured systems. The main problem with tightly structured systems is the  been focused on tightly structured systems. The main problem with tightly structured systems is the
1124  fact that tightly structured algorithms perform data lookups based on a globally unique identifier (key).  fact that tightly structured algorithms perform data lookups based on a globally unique identifier (key).
1125    
# Line 1143  Recent study has been focused on the fea Line 1127  Recent study has been focused on the fea
1127  \cite{li03feasibility} on top of tightly structured overlays. Authors argue, that it is possible to implement  \cite{li03feasibility} on top of tightly structured overlays. Authors argue, that it is possible to implement
1128  Peer-to-Peer Web-like search with certain compromises. First, Peer-to-Peer search engine may need to  Peer-to-Peer Web-like search with certain compromises. First, Peer-to-Peer search engine may need to
1129  decrease result quality in order to make searching more efficient. Second, Peer-to-Peer systems must  decrease result quality in order to make searching more efficient. Second, Peer-to-Peer systems must
1130  observe better the properties of underlying network for better performance.  observe the properties of underlying network for better performance.
1131    
1132  Some studies have been concentrated on SQL-like queries \cite{harren02complex}  Some studies have been concentrated on SQL-like queries \cite{harren02complex}
1133  in tightly structured overlays. Other approaches include adaption of data lookup model of loosely  in tightly structured overlays. Other approaches include adaption of data lookup model of the loosely
1134  structured approach into tightly structured systems \cite{ansaryefficientbroadcast03}, \cite{chord:om_p-meng}.  structured approach into tightly structured systems \cite{ansaryefficientbroadcast03, chord:om_p-meng}.
1135  Some studies include additional layer upon overlay network \cite{kronfol02fasdsearch},  Some studies include additional layer upon overlay network \cite{kronfol02fasdsearch, joseph02p2players}
1136  \cite{joseph02p2players} and range queries \cite{andrzejak02rangequeries}.  and range queries \cite{andrzejak02rangequeries}.
1137    
1138  Many techniques have been developed in order to provide more efficient search indexing. As  Many techniques have been developed in order to provide more efficient search indexing. As
1139  several studies show, the popularity of queries in the Internet follow Zipf-like  several studies show, the popularity of queries in the Internet follow Zipf-like
# Line 1172  more research is required to make indexi Line 1156  more research is required to make indexi
1156    
1157  Adaptive system management and self-organization are essential properties  Adaptive system management and self-organization are essential properties
1158  of any Peer-to-Peer system, since centralized control over the system is missing. Loosely structured  of any Peer-to-Peer system, since centralized control over the system is missing. Loosely structured
1159  systems require less system management properties than tightly structured systems; in loosely  systems require less system management properties than tightly structured systems; in a loosely
1160  structured system, peers join and leave the system constantly without any restrictions. On the  structured system, peers join and leave the system constantly without any restrictions. On the
1161  other hand, however, peers in tightly structured system join and leave the system but have less freedom,  other hand, however, peers in tightly structured system join and leave the system but have less freedom,
1162  i.e. overlay chooses peer's neighbors on behalf of peer itself and maps data items randomly  i.e. overlay chooses peer's neighbors on behalf of peer itself and maps data items randomly
1163  throughout the overlay network.  throughout the overlay network.
1164    
1165  Current research has been focused on system management of tightly structured systems, and all presented  Current research has been focused on system management of tightly structured systems, and all presented
1166  algorithms of tightly structured approach have been analyzed under static simulation environments. Furthermore, proposed  algorithms of the tightly structured approach have been analyzed under static simulation environments. Furthermore, proposed
1167  tightly structured overlays are configured statically to achieve the desired reliability even in uncommon and adverse environment  tightly structured overlays are configured statically to achieve the desired reliability even in uncommon and adverse environment
1168  \cite{rowston03controlloingreliability}. The most important factor for  \cite{rowston03controlloingreliability}. The most important factor for
1169  future research is to get real-life experiences from tightly structured systems, when there are frequent  future research is to get real-life experiences from tightly structured systems, when there are frequent
# Line 1206  by Freedman et al. tries to reduce hot s Line 1190  by Freedman et al. tries to reduce hot s
1190  \cite{sloppy:iptps03}. Another key feature of their work is that peers self-organize into clusters,  \cite{sloppy:iptps03}. Another key feature of their work is that peers self-organize into clusters,
1191  therefore enabling peers to find nearby data without looking up data from distant peers.  therefore enabling peers to find nearby data without looking up data from distant peers.
1192    
1193  As mentioned before, an implicit assumption of almost every tightly structured system is that there is random, uniform  As mentioned before, an implicit assumption of almost every tightly structured system is that there is a random, uniform
1194  distribution of peer and key identifiers. Even if participating peers are extremely heterogeneous, e.g., in  distribution of peer and key identifiers. Even if participating peers are extremely heterogeneous, e.g., in
1195  face of computing power or network bandwidth, all data items are distributed uniformly. Clearly, this is  face of computing power or network bandwidth, all data items are distributed uniformly. Clearly, this is
1196  a serious problem of tightly structured overlays in face of performance and load balancing. Measurement study  a serious problem of tightly structured overlays in face of performance and load balancing. Measurement study
# Line 1217  to support heterogeneity better \cite{zh Line 1201  to support heterogeneity better \cite{zh
1201    
1202  Research has been done on self-organization. Ledlie et al. propose techniques for forming and maintaining  Research has been done on self-organization. Ledlie et al. propose techniques for forming and maintaining
1203  groups in highly dynamic environment \cite{ledlie02selfp2p}. Unfortunately their work relies on idea that  groups in highly dynamic environment \cite{ledlie02selfp2p}. Unfortunately their work relies on idea that
1204  participating peers would create multiple hierarchical groups. It's not clear whether this approach  participating peers would create multiple hierarchical groups. It is not clear whether this approach
1205  is fault-tolerant and suitable to Peer-to-Peer environment. More promising work has been done by Rowston et al.  is fault-tolerant and suitable to Peer-to-Peer environment. More promising work has been done by Rowston et al.
1206  in \cite{rowston03controlloingreliability}. Authors propose techniques for self-tuning, dealing with  in \cite{rowston03controlloingreliability}. Authors propose techniques for self-tuning, dealing with
1207  uncommon conditions (e.g., network partition and high failure rates). Moreover, authors argue that  uncommon conditions (e.g., network partition and high failure rates). Moreover, authors argue that
# Line 1243  components. More important, all existing Line 1227  components. More important, all existing
1227  of the most important area of future research is to create common programming abstractions, i.e.,  of the most important area of future research is to create common programming abstractions, i.e.,
1228  interfaces, design patters and frameworks. Also, equal benchmarks are needed for comparing  interfaces, design patters and frameworks. Also, equal benchmarks are needed for comparing
1229  different algorithms. Recently, there have been few proposals towards common programming  different algorithms. Recently, there have been few proposals towards common programming
1230  guidelines. This list includes \cite{zhao03api}, \cite{frise02p2pframework}, \cite{babaoglu02anthill}.  guidelines. This list includes \cite{zhao03api, frise02p2pframework, babaoglu02anthill}.
1231  Early experiments with Peer-to-Peer benchmarking include \cite{ratnasamy02routing} and \cite{rhea03benchmarks}.  Early experiments with Peer-to-Peer benchmarking include \cite{ratnasamy02routing, rhea03benchmarks}.
1232    
1233  \subsection{Social behavior}  \subsection{Social behavior}
1234    
1235  Frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate. Another belief  Frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate. Another belief
1236  is that all peers would behave equally, i.e., all peers both consume and contribute resources.  is that all peers would behave equally, i.e., all peers both consume and contribute resources.
1237  However, these assumptions are not true as several studies show. Peers rather consume than contribute and  However, these assumptions are not true as several studies show. Peers rather consume than contribute and
1238  peers are unwilling to cooperate \cite{saroiu02measurementstudyp2p}, \cite{oram01harnessingpower},  peers are unwilling to cooperate \cite{saroiu02measurementstudyp2p, oram01harnessingpower, hearn02mojonation}.
 \cite{hearn02mojonation}.  
1239    
1240  Somewhat surprisingly little research has been done in this area, especially when considering  Somewhat surprisingly little research has been done in this area, especially when considering
1241  the possible impact of \emph{unwanted social behavior} to performance of Peer-to-Peer  the possible impact of \emph{unwanted social behavior} to performance of Peer-to-Peer
# Line 1271  Peer-to-Peer system, which uses empirica Line 1254  Peer-to-Peer system, which uses empirica
1254  Very little research has been done on simulating a Peer-to-Peer system. Presumably, this  Very little research has been done on simulating a Peer-to-Peer system. Presumably, this
1255  is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very  is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very
1256  difficult. Floyd et al. has been studying the simulation of the Internet in \cite{504642}. Authors  difficult. Floyd et al. has been studying the simulation of the Internet in \cite{504642}. Authors
1257  state that simulating the Internet is very challenging task, because of Internet's heterogeneity  state that simulating the Internet is very challenging task, because of its heterogeneity
1258  and rapid change. Obviously, these factors exist also in Peer-to-Peer systems even with higher  and rapid change. Obviously, these factors exist also in Peer-to-Peer systems even with higher
1259  rates.  rates.
1260    
# Line 1317  miscellaneous open problems are listed. Line 1300  miscellaneous open problems are listed.
1300    
1301    
1302    
1303  \parbox{90pt}{Query routing \cite{sit02securitycons}, \cite{aspnes02faultrouting}, \cite{castro02securerouting}, \cite{ratnasamy02routing}, \cite{gavoille01routing},  \parbox{90pt}{Query routing \cite{sit02securitycons, aspnes02faultrouting, castro02securerouting, ratnasamy02routing, gavoille01routing,
1304  \cite{lynch02atomicdataaccess}, \cite{fiat02censorship}, \cite{saia02dynamicfaultcontentnetwork}, \cite{datar02butterflies}} &                    lynch02atomicdataaccess, fiat02censorship, saia02dynamicfaultcontentnetwork, datar02butterflies}} &                    
1305  \parbox{110pt}{Incorrect forwarding (hostile), incorrect routing (hostile)} &  \parbox{110pt}{Incorrect forwarding (hostile), incorrect routing (hostile)} &
1306  \parbox{110pt}{Query monitoring, cross check routing tables, verify routing tables, create routing table invariants} &  \parbox{110pt}{Query monitoring, cross check routing tables, verify routing tables, create routing table invariants} &
1307  \parbox{110pt}{Increases system complexity}  \parbox{110pt}{Increases system complexity}
1308  \\ \hline  \\ \hline
1309    
1310    
1311  \parbox{90pt}{DoS attack \cite{sit02securitycons}, \cite{saia02dynamicfaultcontentnetwork}, \cite{datar02butterflies}, \cite{daswani02queryflooddos}, \cite{juels99clientpuzzles}} &  \parbox{90pt}{DoS attack \cite{sit02securitycons, saia02dynamicfaultcontentnetwork, datar02butterflies, daswani02queryflooddos, juels99clientpuzzles}} &
1312  \parbox{110pt}{Distributed, controlled burden against specific computer(s)} &  \parbox{110pt}{Distributed, controlled burden against specific computer(s)} &
1313  \parbox{110pt}{Client puzzles, load balancing, traffic measurements, traffic models, replication} &  \parbox{110pt}{Client puzzles, load balancing, traffic measurements, traffic models, replication} &
1314  \parbox{110pt}{Only partial solutions, traffic models most effective}  \parbox{110pt}{Only partial solutions, traffic models most effective}
1315  \\ \hline  \\ \hline
1316    
1317    
1318  \parbox{90pt}{Sybil attack \cite{douceur02sybil}, \cite{castro02securerouting}} &  \parbox{90pt}{Sybil attack \cite{douceur02sybil, castro02securerouting}} &
1319  \parbox{110pt}{Single hostile entity presents multiple entities} &  \parbox{110pt}{Single hostile entity presents multiple entities} &
1320  \parbox{110pt}{Identify all peers simultaneously across the system, collect pool of peers which are validated, distributed peer ID creation} &  \parbox{110pt}{Identify all peers simultaneously across the system, collect pool of peers which are validated, distributed peer ID creation} &
1321  \parbox{110pt}{Not practically realizable, research focused on persistence, not on identity distinction}  \parbox{110pt}{Not practically realizable, research focused on persistence, not on identity distinction}
# Line 1360  miscellaneous open problems are listed. Line 1343  miscellaneous open problems are listed.
1343  \\ \hline  \\ \hline
1344    
1345    
1346  \parbox{90pt}{Anonymity \cite{dingledine00free}, \cite{tarzan:ccs9}, \cite{pub00}, \cite{clarke00freenet}, \cite{reiter98crowds}, \cite{352607},\cite{502002}} &  \parbox{90pt}{Anonymity \cite{dingledine00free, tarzan:ccs9, pub00, clarke00freenet, reiter98crowds, 352607, 502002}} &
1347  \parbox{110pt}{Anonymity cannot be provided in all cases} &  \parbox{110pt}{Anonymity cannot be provided in all cases} &
1348  \parbox{110pt}{Remailers, pre-routing} &  \parbox{110pt}{Remailers, pre-routing} &
1349  \parbox{110pt}{Total anonymity cannot be provided yet}  \parbox{110pt}{Total anonymity cannot be provided yet}
1350  \\ \hline  \\ \hline
1351    
1352    
1353  \parbox{90pt}{Malicious peers \cite{sit02securitycons}, \cite{castro02securerouting}} &  \parbox{90pt}{Malicious peers \cite{sit02securitycons, castro02securerouting}} &
1354  \parbox{110pt}{How to identify malicious peers in the system ?} &  \parbox{110pt}{How to identify malicious peers in the system ?} &
1355  \parbox{110pt}{Create invariants for peer behavior, verify invariants, self-certifying data} &  \parbox{110pt}{Create invariants for peer behavior, verify invariants, self-certifying data} &
1356  \parbox{110pt}{Partial solutions, self-certifying data most reliable}  \parbox{110pt}{Partial solutions, self-certifying data most reliable}
1357  \\ \hline  \\ \hline
1358    
1359    
1360  \parbox{90pt}{Access Control \cite{nejdl03accesscontrol}, \cite{daswani03openproblems}} &  \parbox{90pt}{Access Control \cite{nejdl03accesscontrol, daswani03openproblems}} &
1361  \parbox{110pt}{Can we define access control levels in Peer-to-Peer network ?} &  \parbox{110pt}{Can we define access control levels in Peer-to-Peer network ?} &
1362  \parbox{110pt}{Schema-based rules} &  \parbox{110pt}{Schema-based rules} &
1363  \parbox{110pt}{Some initial experiences, need more research}  \parbox{110pt}{Some initial experiences, need more research}
# Line 1432  miscellaneous open problems are listed. Line 1415  miscellaneous open problems are listed.
1415    
1416  \endfoot  \endfoot
1417                                    
1418  \parbox{90pt}{Web indexing and searching \cite{li03feasibility}, \cite{Bhattacharjee03resultcache}, \cite{362692}, \cite{CuencaAcuna2002DSIWorkshop},  \parbox{90pt}{Web indexing and searching \cite{li03feasibility, Bhattacharjee03resultcache, 362692, CuencaAcuna2002DSIWorkshop,
1419  \cite{rhea02probabilistic}, \cite{joseph02neurogrid}, \cite{crespo02semanticoverlay}, \cite{joseph02p2players}, \cite{chord:om_p-meng},  rhea02probabilistic, joseph02neurogrid, crespo02semanticoverlay, joseph02p2players, chord:om_p-meng, wittengigabytes, 338634}} &
 \cite{wittengigabytes}, \cite{338634}} &  
1420  \parbox{110pt}{Perform Web like searches in Peer-to-Peer network} &  \parbox{110pt}{Perform Web like searches in Peer-to-Peer network} &
1421  \parbox{110pt}{Data compression, view trees, bloom filters and its variations, gap compression, index intersection optimizations, clustering} &  \parbox{110pt}{Data compression, view trees, bloom filters and its variations, gap compression, index intersection optimizations, clustering} &
1422  \parbox{110pt}{Effective but complex solutions, some compromises have to be done (decrease result quality, modify overlay's structure), more research needed}  \parbox{110pt}{Effective but complex solutions, some compromises have to be done (decrease result quality, modify overlay's structure), more research needed}
1423  \\ \hline  \\ \hline
1424    
1425    
1426  \parbox{90pt}{Efficient and scalable data discovery \cite{lv02searchreplication}, \cite{osokine02distnetworks}, \cite{yang02improvingsearch}, \cite{lv02gnutellascalable},  \parbox{90pt}{Efficient and scalable data discovery \cite{lv02searchreplication, osokine02distnetworks, yang02improvingsearch, lv02gnutellascalable,
1427  \cite{ganesan02yappers}, \cite{adamic02localsearch}, \cite{adamic01powerlawsearch}, \cite{ripeanu02mappinggnutella}, \cite{milgram67smallworld}, \cite{adamic99small},  ganesan02yappers, adamic02localsearch, adamic01powerlawsearch, ripeanu02mappinggnutella, milgram67smallworld, adamic99small,
1428  \cite{ramanathan02goodpeers}, \cite{kleinberg99small}, \cite{nips02-Kleinberg}, \cite{zhang02using}, \cite{watts00dynamics}, \cite{karger02findingnearest},  ramanathan02goodpeers, kleinberg99small, nips02-Kleinberg, zhang02using, watts00dynamics, karger02findingnearest,
1429  \cite{brinkmann02compactplacement}, \cite{rhea02probabilistic}, \cite{castro02networkproximity}, \cite{ng02predicting}, \cite{pias03lighthouse}} &  brinkmann02compactplacement, rhea02probabilistic, castro02networkproximity, ng02predicting, pias03lighthouse}} &
1430  \parbox{110pt}{Find resources efficiently, if resource exists (loosely structured)} &  \parbox{110pt}{Find resources efficiently, if resource exists (loosely structured)} &
1431  \parbox{110pt}{Super peers, peer clusters, caching techniques} &  \parbox{110pt}{Super peers, peer clusters, caching techniques} &
1432  \parbox{110pt}{More efficient, less network traffic, not comparable to the efficiency of tightly structured systems}  \parbox{110pt}{More efficient, less network traffic, not comparable to the efficiency of tightly structured systems}
1433  \\ \hline  \\ \hline
1434    
1435    
1436  \parbox{90pt}{Richness of queries \cite{harren02complex}, \cite{ansaryefficientbroadcast03}, \cite{andrzejak02rangequeries}} &  \parbox{90pt}{Richness of queries \cite{harren02complex, ansaryefficientbroadcast03, andrzejak02rangequeries}} &
1437  \parbox{110pt}{Query languages should be more powerful in tightly structured overlays} &  \parbox{110pt}{Query languages should be more powerful in tightly structured overlays} &
1438  \parbox{110pt}{SQL-like queries} &  \parbox{110pt}{SQL-like queries} &
1439  \parbox{110pt}{Hard to implement, increases system complexity, not much research has been done}  \parbox{110pt}{Hard to implement, increases system complexity, not much research has been done}
1440  \\ \hline  \\ \hline
1441    
1442    
1443  \parbox{90pt}{Robustness \cite{datar02butterflies}, \cite{saia02dynamicfaultcontentnetwork}, \cite{fiat02censorship}, \cite{aspnes02faultrouting}, \cite{albert-00-tolerance}, \cite{libennowell01observations}} &  \parbox{90pt}{Robustness \cite{datar02butterflies, saia02dynamicfaultcontentnetwork, fiat02censorship, aspnes02faultrouting, albert-00-tolerance, libennowell01observations}} &
1444  \parbox{110pt}{How well system performs under hostile attacks/in the case of severe failure ?} &  \parbox{110pt}{How well system performs under hostile attacks/in the case of severe failure ?} &
1445  \parbox{110pt}{Self-tuning, backup links, use diverse routing paths, power-law networks/properties} &  \parbox{110pt}{Self-tuning, backup links, use diverse routing paths, power-law networks/properties} &
1446  \parbox{110pt}{Working solutions}  \parbox{110pt}{Working solutions}
# Line 1479  miscellaneous open problems are listed. Line 1461  miscellaneous open problems are listed.
1461  \\ \hline  \\ \hline
1462    
1463    
1464  \parbox{90pt}{Network proximity \cite{pias03lighthouse}, \cite{ng02predicting}, \cite{ratnasamy02ght}, \cite{eriksson03peernet}, \cite{castro02networkproximity}} &  \parbox{90pt}{Network proximity \cite{pias03lighthouse, ng02predicting, ratnasamy02ght, eriksson03peernet, castro02networkproximity}} &
1465  \parbox{110pt}{Can we take into account the underlying network's properties better when forming overlay network (network-awareness for performance) ?} &  \parbox{110pt}{Can we take into account the underlying network's properties better when forming overlay network (network-awareness for performance) ?} &
1466  \parbox{110pt}{Global network positioning, lighthouse technique, triangulated heuristics} &  \parbox{110pt}{Global network positioning, lighthouse technique, triangulated heuristics} &
1467  \parbox{110pt}{Increases system complexity, no real world experience in a wide scale, proposed solutions are susceptible to single point of failure}  \parbox{110pt}{Increases system complexity, no real world experience in a wide scale, proposed solutions are susceptible to single point of failure}
1468  \\ \hline  \\ \hline
1469    
1470    
1471  \parbox{90pt}{Locality \cite{keleher-02-p2p}, \cite{hildrum02distributedobject}, \cite{freedman02trie}, \cite{sloppy:iptps03}, \cite{plaxton97accessingnearby}, \cite{karger02findingnearest}} &  \parbox{90pt}{Locality \cite{keleher-02-p2p, hildrum02distributedobject, freedman02trie, sloppy:iptps03, plaxton97accessingnearby, karger02findingnearest}} &
1472  \parbox{110pt}{Could tightly structured systems exploit locality properties better ?} &  \parbox{110pt}{Could tightly structured systems exploit locality properties better ?} &
1473  \parbox{110pt}{Constrained Load Balancing, using network properties for nearest neighbor selection, self-organizing clusters} &  \parbox{110pt}{Constrained Load Balancing, using network properties for nearest neighbor selection, self-organizing clusters} &
1474  \parbox{110pt}{Working solutions}  \parbox{110pt}{Working solutions}
1475  \\ \hline  \\ \hline
1476    
1477    
1478  \parbox{90pt}{Hot spots \cite{258660}, \cite{sloppy:iptps03}, \cite{maymounkov03ratelesscodes}} &  \parbox{90pt}{Hot spots \cite{258660, sloppy:iptps03, maymounkov03ratelesscodes}} &
1479  \parbox{110pt}{What will happen if some resource is extremely popular and only one peer is hosting it ?} &  \parbox{110pt}{What will happen if some resource is extremely popular and only one peer is hosting it ?} &
1480  \parbox{110pt}{Caching, multisource downloads, replication, load balancing, sloppy hashing} &  \parbox{110pt}{Caching, multisource downloads, replication, load balancing, sloppy hashing} &
1481  \parbox{110pt}{For query hot spots, caching and multisource downloads efficiently reduce hot spots, for routing hot spots, benefits are smaller}  \parbox{110pt}{For query hot spots, caching and multisource downloads efficiently reduce hot spots, for routing hot spots, benefits are smaller}
1482  \\ \hline  \\ \hline
1483    
1484    
1485  \parbox{90pt}{Load balancing \cite{rao03loadbalancing}, \cite{ledlie02selfp2p}, \cite{byers03dhtbalancing}} &  \parbox{90pt}{Load balancing \cite{rao03loadbalancing, ledlie02selfp2p, byers03dhtbalancing}} &
1486  \parbox{110pt}{Random (but uniformly distributed) identifier selection could cause system inbalance among participants with different capabilities} &  \parbox{110pt}{Random (but uniformly distributed) identifier selection could cause system inbalance among participants with different capabilities} &
1487  \parbox{110pt}{Caching, virtual server transfers} &  \parbox{110pt}{Caching, virtual server transfers} &
1488  \parbox{110pt}{Effective, more research required in fully dynamic environment}  \parbox{110pt}{Effective, more research required in fully dynamic environment}
1489  \\ \hline  \\ \hline
1490    
1491  \parbox{90pt}{System in flux \cite{libennowell01observations}, \cite{571863}, \cite{ledlie02selfp2p}, \cite{albert-02-statistical}} &  \parbox{90pt}{System in flux \cite{libennowell01observations, 571863, ledlie02selfp2p, albert-02-statistical}} &
1492  \parbox{110pt}{Peers join and leave system constantly. What about load balancing and performance ?} &  \parbox{110pt}{Peers join and leave system constantly. What about load balancing and performance ?} &
1493  \parbox{110pt}{Half-life phenomenon (for analysis), simple overlay maintenance and construction algorithm} &  \parbox{110pt}{Half-life phenomenon (for analysis), simple overlay maintenance and construction algorithm} &
1494  \parbox{110pt}{Initial theoretical analysis have been created, but not comprehensive model for analyzing different system states and its variations (e.g. complex usage patterns)}  \parbox{110pt}{Initial theoretical analysis have been created, but not comprehensive model for analyzing different system states and its variations (e.g. complex usage patterns)}
1495  \\ \hline  \\ \hline
1496    
1497  \parbox{90pt}{Sudden network partition \cite{harvey03skipnet1}, \cite{harvey03skipnet2}, \cite{rowston03controlloingreliability}} &  \parbox{90pt}{Sudden network partition \cite{harvey03skipnet1, harvey03skipnet2, rowston03controlloingreliability}} &
1498  \parbox{110pt}{Sub network is isolated from other network because of network disconnection} &  \parbox{110pt}{Sub network is isolated from other network because of network disconnection} &
1499  \parbox{110pt}{Self-tuning, environment observation, localized network connection for minimum latency (backup connections)} &  \parbox{110pt}{Self-tuning, environment observation, localized network connection for minimum latency (backup connections)} &
1500  \parbox{110pt}{Creates more overhead/space requirements per peer}  \parbox{110pt}{Creates more overhead/space requirements per peer}
1501  \\ \hline  \\ \hline
1502    
1503  \parbox{90pt}{Fail Stop \cite{rowston03controlloingreliability}, \cite{zhang03somo}} &  \parbox{90pt}{Fail Stop \cite{rowston03controlloingreliability, zhang03somo}} &
1504  \parbox{110pt}{A faulty peer stops working} &  \parbox{110pt}{A faulty peer stops working} &
1505  \parbox{110pt}{Failure detectors, informing algorithms} &  \parbox{110pt}{Failure detectors, informing algorithms} &
1506  \parbox{110pt}{Creates more network traffic, peer's information can be outdated, failure detectors not reliable}  \parbox{110pt}{Creates more network traffic, peer's information can be outdated, failure detectors not reliable}
# Line 1563  miscellaneous open problems are listed. Line 1545  miscellaneous open problems are listed.
1545  \endfoot  \endfoot
1546    
1547    
1548  \parbox{90pt}{Mutual distrust \cite{cornelli02reputableservents}, \cite{aberer01trust}} &  \parbox{90pt}{Mutual distrust \cite{cornelli02reputableservents, aberer01trust}} &
1549  \parbox{110pt}{Nobody trusts anybody} &  \parbox{110pt}{Nobody trusts anybody} &
1550  \parbox{110pt}{Reputation methods, key infrastructures} &  \parbox{110pt}{Reputation methods, key infrastructures} &
1551  \parbox{110pt}{Resource demanding, not practical to implement/not working solutions, no real world experience in a wide scale}  \parbox{110pt}{Resource demanding, not practical to implement/not working solutions, no real world experience in a wide scale}
1552  \\ \hline  \\ \hline
1553    
1554    
1555  \parbox{90pt}{Lack of motivation to cooperate \cite{golle01incentivesp2p}, \cite{ngan03enforcefile}, \cite{shneidman03rationality}} &  \parbox{90pt}{Lack of motivation to cooperate \cite{golle01incentivesp2p, ngan03enforcefile, shneidman03rationality}} &
1556  \parbox{110pt}{All participants do not behave like they should be, instead they go for own profit} &  \parbox{110pt}{All participants do not behave like they should be, instead they go for own profit} &
1557  \parbox{110pt}{Different reputation methods} &  \parbox{110pt}{Different reputation methods} &
1558  \parbox{110pt}{No real world experience in a wide scale}  \parbox{110pt}{No real world experience in a wide scale}
1559  \\ \hline  \\ \hline
1560    
1561    
1562  \parbox{90pt}{Heterogeneity \cite{saroiu02measurementstudyp2p}, \cite{brinkmann02compactplacement}, \cite{zhao02brocade},\cite{gurmeet03symphony}, \cite{rowston03controlloingreliability}} &  \parbox{90pt}{Heterogeneity \cite{saroiu02measurementstudyp2p, brinkmann02compactplacement, zhao02brocade, gurmeet03symphony, rowston03controlloingreliability}} &
1563  \parbox{110pt}{There are different kind of peers in the system, in light of bandwidth and computing power} &  \parbox{110pt}{There are different kind of peers in the system, in light of bandwidth and computing power} &
1564  \parbox{110pt}{Super peers (loosely structured), clusters (loosely structured) additional layer upon tighty structured systems, structure itself is simple (tighty structured)} &  \parbox{110pt}{Super peers (loosely structured), clusters (loosely structured) additional layer upon tighty structured systems, structure itself is simple (tighty structured)} &
1565  \parbox{110pt}{Working solutions, increases system complexity (additional layer)}  \parbox{110pt}{Working solutions, increases system complexity (additional layer)}
1566  \\ \hline  \\ \hline
1567    
1568    
1569  \parbox{90pt}{Programming guidelines \cite{zhao03api}, \cite{frise02p2pframework}, \cite{babaoglu02anthill}, \cite{rhea03benchmarks}, \cite{garciamolina03sil}, \cite{balakrishnan03semanticfree}} &  \parbox{90pt}{Programming guidelines \cite{zhao03api, frise02p2pframework, babaoglu02anthill, rhea03benchmarks, garciamolina03sil, balakrishnan03semanticfree}} &
1570  \parbox{110pt}{Set of programming guidelines/frameworks is needed for better interoperability between different systems} &  \parbox{110pt}{Set of programming guidelines/frameworks is needed for better interoperability between different systems} &
1571  \parbox{110pt}{Common frameworks and APIs} &  \parbox{110pt}{Common frameworks and APIs} &
1572  \parbox{110pt}{Common framework/API is still missing, a few proposals have been made (tightly structured)}  \parbox{110pt}{Common framework/API is still missing, a few proposals have been made (tightly structured)}
# Line 1599  miscellaneous open problems are listed. Line 1581  miscellaneous open problems are listed.
1581    
1582    
1583  \parbox{90pt}{Overlay management and health monitoring \cite{zhang03somo}} &  \parbox{90pt}{Overlay management and health monitoring \cite{zhang03somo}} &
1584  \parbox{110pt}{System is self-capable to monitor it's status and health for better performance} &  \parbox{110pt}{System is self-capable to monitor it is status and health for better performance} &
1585  \parbox{110pt}{Build a meta data overlay atop of structured overlay (such as SOMO for structured overlays), make local decisions about overlay (loosely structured)} &  \parbox{110pt}{Build a meta data overlay atop of structured overlay (such as SOMO for structured overlays), make local decisions about overlay (loosely structured)} &
1586  \parbox{110pt}{For tightly structured overlays, efficient and simple to implement, fault tolerance unknown, for loosely structured not necessarily efficient because decisions are based on local knowledge}  \parbox{110pt}{For tightly structured overlays, efficient and simple to implement, fault tolerance unknown, for the loosely structured approach not necessarily efficient because decisions are based on local knowledge}
1587  \\ \hline  \\ \hline
1588    
1589  \parbox{90pt}{Locating Peer-to-Peer network} &  \parbox{90pt}{Locating Peer-to-Peer network} &
# Line 1620  networks (MANETs) can be only connected Line 1602  networks (MANETs) can be only connected
1602    
1603  \chapter{Fenfire hypermedia system}  \chapter{Fenfire hypermedia system}
1604    
1605  In this chapter we give an overview of Fenfire system. We also  In this chapter we give an overview of the Fenfire system. We also
1606  describe briefly xanalogical storage model. At the end of this chapter we study Storm,  describe briefly xanalogical storage model. At the end of this chapter we study Storm,
1607  Fenfire's software module, which is an essential part of Fenfire's Peer-to-Peer  Fenfire's software module, which is an essential part of Fenfire's Peer-to-Peer
1608  functionality.  functionality.
1609    
1610  \section{Overview}  \section{Overview}
1611    
1612  Fenfire project \cite{fenfireurl} is an effort to build a location transparent, hyperstructured desktop  The Fenfire project \cite{fenfireurl} is an effort to build a location transparent, hyperstructured desktop
1613  environment. Fenfire's uses xanalogical storage model \cite{ted-xu-model} as a basis for hyperstructured  environment. Fenfire uses xanalogical storage model \cite{ted-xu-model} as a basis for hyperstructured
1614  media. Fenfire uses innovative user interfaces for displaying data to the end users. All data in Fenfire  media. Fenfire deploys innovative user interfaces for displaying data to the end users. All data in the Fenfire
1615  is stored in same format, i.e., blocks. This should allow making references between data easier and more  is stored in a unified format, i.e., blocks. This should allow making references between data easier and more
1616  seamlessly interoperating than in other systems. For location transparency in a distributed system, Fenfire  seamlessly interoperating than in other systems. For location transparency in a distributed system, Fenfire
1617  uses Peer-to-Peer network for locating and fetching blocks.  uses Peer-to-Peer network for locating and fetching blocks.
1618    
1619  Fenfire is free software and it is licensed under GNU L-GPL.  Fenfire was formerly also a partial implementation  Fenfire is free software and it is licensed under GNU LGPL.  Fenfire was formerly also a partial implementation
1620  of the ZigZag\texttrademark --structure, which was originally invented  of the ZigZag\texttrademark --structure, which was originally invented
1621  by Ted Nelson. Now, however, Fenfire uses Resource Description Framework (RDF) \cite{w3rdfurl}  by Ted Nelson. Now, however, Fenfire uses Resource Description Framework (RDF) \cite{w3rdfurl}
1622  for representing internal data structures and their relationships.  for representing internal data structures and their relationships.
1623    
1624  Fenfire is high modular software system. It consists of several independent software modules:  Fenfire is a high modular software system. It consists of several independent software modules:
1625    
1626  \begin{itemize}  \begin{itemize}
1627  \item \textbf{Storm}: distributed storage module for storing arbitrary data items  \item \textbf{Storm}: distributed storage module for storing arbitrary data items
# Line 1652  Fenfire is high modular software system. Line 1634  Fenfire is high modular software system.
1634    
1635  In this thesis, we focus on Storm and Alph modules, since they are the foundation of Fenfire's  In this thesis, we focus on Storm and Alph modules, since they are the foundation of Fenfire's
1636  Peer-to-Peer functionality. If not otherwise mentioned, we use term 'Storm' when referring to both  Peer-to-Peer functionality. If not otherwise mentioned, we use term 'Storm' when referring to both
1637  Storm and Alph software modules. For location transparency in Fenfire system, Storm software module  Storm and Alph software modules. For location transparency in the Fenfire system, Storm software module
1638  must have support for Peer-to-Peer functionality as it provides low-level data storage operations  must have support for Peer-to-Peer functionality as it provides low-level data storage operations
1639  in Fenfire system.  in the Fenfire system.
1640    
1641    
1642  \section{Xanalogical storage model}  \section{Xanalogical storage model}
# Line 1683  of fluid media contents. In xanalogical Line 1665  of fluid media contents. In xanalogical
1665  and bidirectional. Xanadu link is an \emph{association} of two enfilades, such as an  and bidirectional. Xanadu link is an \emph{association} of two enfilades, such as an
1666  annotation to a specific part of a another document. \emph{Transclusion} is an inclusion in  annotation to a specific part of a another document. \emph{Transclusion} is an inclusion in
1667  enfilade of contents already used in another enfilade, i.e., current fluid media is copied into  enfilade of contents already used in another enfilade, i.e., current fluid media is copied into
1668  different data contents. By using this mechanism, system implementing xanalogical storage model  different data contents. By using this mechanism, a system implementing xanalogical storage model
1669  is able to show all data content that share same fluid media with current data content  is able to show all data content that share same fluid media with current data content
1670  (e.g., all documents containing current document's text). Figure \ref{fig:xanalogical_model}  (e.g., all documents containing current document's text). Figure \ref{fig:xanalogical_model}
1671  illustrates xanalogical storage model with documents, text and characters.  illustrates xanalogical storage model with documents, text and characters.
# Line 1703  In this section, we will give a brief ov Line 1685  In this section, we will give a brief ov
1685  from recent publications: for general discussion about Fenfire in Peer-to-Peer environment,  from recent publications: for general discussion about Fenfire in Peer-to-Peer environment,
1686  see \cite{lukka02freenetguids}, and for detailed Storm design, see \cite{fallenstein03storm}.  see \cite{lukka02freenetguids}, and for detailed Storm design, see \cite{fallenstein03storm}.
1687    
1688  Storm (for \emph{STORage Module}) is a software module, which is used in Fenfire for  Storm (for \emph{STORage Module}) is a software module, which is used in the Fenfire for
1689  data storage operations. Storm stores all data as \emph{blocks}, which  data storage operations. Storm stores all data as \emph{blocks}, which
1690  are immutable byte sequences. SHA-1\footnote{SHA-1 is considered as a collision free  are immutable byte sequences. SHA-1\footnote{SHA-1 is considered as a collision free
1691  hash function. Therefore, it is very unlikely that two different Storm data blocks  hash function. Therefore, it is very unlikely that two different Storm data blocks
# Line 1713  SHA-1 \cite{fips-sha-1} is used for veri Line 1695  SHA-1 \cite{fips-sha-1} is used for veri
1695  blocks have much in common with regular files, except that Storm blocks are \emph{immutable} as  blocks have much in common with regular files, except that Storm blocks are \emph{immutable} as
1696  any change to the byte sequence would change block's hash value, i.e., globally unique  any change to the byte sequence would change block's hash value, i.e., globally unique
1697  identifier. This mechanism creates a basis for implementing xanalogical storage model in  identifier. This mechanism creates a basis for implementing xanalogical storage model in
1698  Fenfire system. Figure \ref{fig:storm_model} illustrates simplified Storm storage model.  in Fenfire system. Figure \ref{fig:storm_model} illustrates simplified Storm storage model.
1699    
1700  \begin{figure}  \begin{figure}
1701  \centering  \centering
# Line 1730  More information about diffs can be foun Line 1712  More information about diffs can be foun
1712    
1713  Pointer \cite{benja02urn5} is a semantic-free, updatable reference to  Pointer \cite{benja02urn5} is a semantic-free, updatable reference to
1714  Storm data block, i.e., Storm scroll block.  Storm data block, i.e., Storm scroll block.
1715  In practice, pointer is random string, which resembles Universal Resource Names  In practice, pointer is a random string, which resembles Universal Resource Names
1716  (URN) \cite{rfc2396}. Pointer itself doesn't contain any data, it's rather a \emph{concept} of  (URN) \cite{rfc2396}. Pointer itself doesn't contain any data, it is rather a \emph{concept} of
1717  data. Pointers are created automatically by Storm and each pointer is  data. Pointers are created automatically by Storm and each pointer is
1718  associated with a collection of \emph{pointer blocks}. Pointer block has a single  associated with a collection of \emph{pointer blocks}. Pointer block has a single
1719  target for the pointer. In figure \ref{fig:storm_model}, we present overall  target for the pointer. In figure \ref{fig:storm_model}, we present overall
# Line 1755  the most recent pointer's block target i Line 1737  the most recent pointer's block target i
1737  In this chapter we evaluate Fenfire in Peer-to-Peer environment.  In this chapter we evaluate Fenfire in Peer-to-Peer environment.
1738  We start by giving a problem overview when considering Fenfire in Peer-to-Peer  We start by giving a problem overview when considering Fenfire in Peer-to-Peer
1739  environment. We define Fenfire's special needs and evaluate existing  environment. We define Fenfire's special needs and evaluate existing
1740  Peer-to-Peer approaches in light of these requirements. After that, we propose system  Peer-to-Peer approaches in light of these requirements. After that, we propose a system
1741  model for Fenfire in Peer-to-Peer environment and present simple methods to perform data  model for Fenfire in Peer-to-Peer environment and present simple methods to perform data
1742  lookups in Peer-to-Peer environment. In the end of this chapter, we discuss possible problems of using Fenfire  lookups in Peer-to-Peer environment. In the end of this chapter, we discuss possible problems of using Fenfire
1743  in Peer-to-Peer environment  in Peer-to-Peer environment.
1744    
1745    
1746  \section{Problem overview}  \section{Problem overview}
# Line 1770  support global data lookups efficiently Line 1752  support global data lookups efficiently
1752  from fragments of data.  from fragments of data.
1753    
1754  In the xanalogical storage model, each fragment of data is identified by a globally  In the xanalogical storage model, each fragment of data is identified by a globally
1755  unique identifier. In Fenfire, data fragments are scroll blocks, generated by Storm storage module.  unique identifier. In the Fenfire system, data fragments are scroll blocks, generated by Storm storage module.
1756  As we discussed already in chapter 4, Fenfire's Storm design  As we discussed already in chapter 4, Fenfire's Storm design
1757  uses SHA-1 \cite{fips-sha-1} hash over the contents of a scroll block for creating globally unique  uses SHA-1 \cite{fips-sha-1} hash over the contents of a scroll block for creating globally unique
1758  identifiers for each scroll block.  In our scenario, fragments of data is distributed  identifiers for each scroll block.  In our scenario, fragments of data is distributed
# Line 1808  publishing or not. Line 1790  publishing or not.
1790  \section{Evaluation of Peer-to-Peer approaches with regard to Fenfire}  \section{Evaluation of Peer-to-Peer approaches with regard to Fenfire}
1791    
1792  In chapter 2, we discussed main differences between loosely and tightly structured  In chapter 2, we discussed main differences between loosely and tightly structured
1793  approaches. As stated, the most significant difference is that tightly structured  approaches. As stated, the most significant difference is that the tightly structured
1794  approach has logarithmical properties in all internal operations, while loosely  approach has logarithmical properties in all internal operations, while the loosely
1795  structured approach doesn't always have even linear properties. Furthermore, the  structured approach doesn't always have even linear properties. Furthermore, the
1796  data lookup model of tightly structured overlay scales much better than loosely  data lookup model of tightly structured overlay scales much better than loosely
1797  structured overlays; tightly structured overlay supports global data lookups  structured overlays; tightly structured overlay supports global data lookups
1798  in the overlay, whereas the data lookup model of loosely structured approach  in the overlay, whereas the data lookup model of the loosely structured approach
1799  is limited to certain area of overlay\footnote{The area depends on where the query  is limited to certain area of overlay\footnote{The area depends on where the query
1800  originator is located in the overlay.}.  originator is located in the overlay.}.
1801    
1802  For Fenfire's special needs for \emph{locating} data, an important advantage of  For Fenfire's special needs for \emph{locating} data, an important advantage of the
1803  tightly structured approach over loosely structured approach is that tightly  tightly structured approach over the loosely structured approach is that tightly
1804  structured systems use location-independent, globally unique identifiers for  structured systems use location-independent, globally unique identifiers for
1805  identifying data in the system. Indeed, this  identifying data in the system. Indeed, this
1806  feature is almost analogical to Fenfire's (and xanalogical storage model's) way of  feature is almost analogical to Fenfire's (and xanalogical storage model's) way of
# Line 1827  to provide general purpose \emph{interfa Line 1809  to provide general purpose \emph{interfa
1809  Domain Name System (DNS) \cite{rfc1101} is widely used RRS system in the Internet.}  Domain Name System (DNS) \cite{rfc1101} is widely used RRS system in the Internet.}
1810   \cite{balakrishnan03semanticfree}. Authors argue that next generation RRS must be   \cite{balakrishnan03semanticfree}. Authors argue that next generation RRS must be
1811  application-independent and references itself should be \emph{unstructured} and  application-independent and references itself should be \emph{unstructured} and
1812  \emph{semantic free}. Finally, as said, with tightly structured systems, it is feasible to  \emph{semantically free}. Finally, as said, with tightly structured systems, it is feasible to
1813  perform \emph{global} data lookups in the overlay. To summarize, these aspects may be the most important features  perform \emph{global} data lookups in the overlay. To summarize, these aspects may be the most important features
1814  of Peer-to-Peer infrastructure with regard to Fenfire as a distributed, location transparent hypermedia system.  of Peer-to-Peer infrastructure with regard to Fenfire as a distributed, location transparent hypermedia system.
1815  Thus, we see the tightly structured approach as the best alternative to locate data in Peer-to-Peer  Thus, we see the tightly structured approach as the best alternative to locate data in Peer-to-Peer
1816  environment.  environment.
1817    
1818  Once located, for \emph{fetching} Fenfire related data from the overlay, we can use regular  Once located, for \emph{fetching} Storm blocks from the overlay, we can use regular
1819  TCP/IP-protocols, such as Hypertext Transfer protocol (HTTP) \cite{rfc2068}. However, HTTP-protocol may  TCP/IP-protocols, such as Hypertext Transfer protocol (HTTP) \cite{rfc2068}. However, HTTP-protocol may
1820  not be optimal when obtaining large amounts of data from the Peer-to-Peer network (e.g.,  not be optimal when obtaining large amounts of data from the Peer-to-Peer network (e.g.,
1821  videos, images or music). In this case, multisource downloads can be very useful  videos, images or music). In this case, multisource downloads can be very useful
1822  for better efficiency \cite{maymounkov03ratelesscodes}, \cite{bittorrenturl}. Furthermore,  for better efficiency \cite{maymounkov03ratelesscodes, bittorrenturl}. Furthermore,
1823  multisource downloads can be used for decreasing load of certain peer, thus avoiding query  multisource downloads can be used for decreasing load of certain peer, thus avoiding query
1824  hot spots in the system \cite{ratnasamy02routing}. Current implementation of Fenfire uses  hot spots in the system \cite{ratnasamy02routing}. Current implementation of Fenfire uses
1825  standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content  standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content
# Line 1846  for a scroll block. In face of multisour Line 1828  for a scroll block. In face of multisour
1828  tree-based hash\footnote{With multisource downloads, tree based hash functions can be used  tree-based hash\footnote{With multisource downloads, tree based hash functions can be used
1829  to verify fixed length segments of data. If hash value of data segment is incorrect,  to verify fixed length segments of data. If hash value of data segment is incorrect,
1830  we need only to fetch \emph{segment} of data (instead of whole data, e.g., a file) from  we need only to fetch \emph{segment} of data (instead of whole data, e.g., a file) from
1831  other source.}, such as \cite{merkle87hashtree} and \cite{mohr02thex} for reliable and efficient  other source.}, such as \cite{merkle87hashtree, mohr02thex} for reliable and efficient
1832  data validation.  data validation.
1833    
1834  Again, there are research challenges with tightly structured systems which have to be  Again, there are research challenges with tightly structured systems which have to be
# Line 1854  addressed, as described in chapter 3. Th Line 1836  addressed, as described in chapter 3. Th
1836  tolerance when system in presence of system flux, non-optimal distance functions in identifier space,  tolerance when system in presence of system flux, non-optimal distance functions in identifier space,
1837  proximity routing, hostile entities and flexible search \cite{balakrishanarticle03lookupp2p}.  proximity routing, hostile entities and flexible search \cite{balakrishanarticle03lookupp2p}.
1838  Additionally, there is only little real world experiments yet with tightly structured systems  Additionally, there is only little real world experiments yet with tightly structured systems
1839  (e.g., \cite{overneturl}, \cite{edonkey2kurl}). Therefore, we can't say for sure, how well these  (e.g., \cite{overneturl, edonkey2kurl}). Therefore, we can't say for sure, how well these
1840  systems would perform in real Peer-to-Peer environment. However, we believe that issues are  systems would perform in real Peer-to-Peer environment. However, we believe that issues are
1841  solved, since there is a strong and wide research community towards to tightly structured  solved, since there is a strong and wide research community towards to tightly structured
1842  overlays \cite{projectirisurl}.  overlays \cite{projectirisurl}.
# Line 1872  Currently, we see Kademlia \cite{maymoun Line 1854  Currently, we see Kademlia \cite{maymoun
1854  locating data efficiently in the Peer-to-Peer overlay. There are two main  locating data efficiently in the Peer-to-Peer overlay. There are two main
1855  reasons for this. First, Kademlia's XOR-based distance function is superior  reasons for this. First, Kademlia's XOR-based distance function is superior
1856  over the distance functions of other systems (see section 2.4). Second, there exist already    over the distance functions of other systems (see section 2.4). Second, there exist already  
1857  deployed real-life systems using Kademlia (e.g., \cite{overneturl}, \cite{edonkey2kurl}, \cite{kashmirurl},  deployed real-life systems using Kademlia (e.g., \cite{overneturl, edonkey2kurl, kashmirurl,
1858  \cite{kato02gisp}), which means that Kademlia's algorithm is simple and easy to implement.  kato02gisp}), which means that Kademlia's algorithm is simple and easy to implement.
1859    
1860  On top of Kademlia, we propose the usage of Sloppy hashing \cite{sloppy:iptps03} which  On top of Kademlia, we propose the usage of Sloppy hashing \cite{sloppy:iptps03} which
1861  is optimized for the DOLR abstraction of tightly structured overlays. With the Sloppy hashing,  is optimized for the DOLR abstraction of tightly structured overlays. With the Sloppy hashing,
# Line 1895  on rateless erasure codes \cite{maymounk Line 1877  on rateless erasure codes \cite{maymounk
1877    
1878  \subsection{Methods}  \subsection{Methods}
1879    
1880  We use the DOLR abstraction of tightly structured approach, i.e., each participating peer hosts  We use the DOLR abstraction of the tightly structured approach, i.e., each participating peer hosts
1881  the data and overlay maintains only the \emph{pointers} to the data. We decided to use the DOLR  the data and overlay maintains only the \emph{pointers} to the data. We decided to use the DOLR
1882  abstraction in our model, since DOLR systems locate data without specifying a storage policy explicitly \cite{rhea03benchmarks}.  abstraction in our model, since DOLR systems locate data without specifying a storage policy explicitly \cite{rhea03benchmarks}.
1883  DHT based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have  DHT based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have
1884  critical problems with load balancing in highly heterogeneous environment. This problem is caused by peers  critical problems with load balancing in highly heterogeneous environment. This problem is caused by peers
1885  which may not be able to store relatively large amount of data with key/value pair, assigned randomly by  which may not be able to store relatively large amount of data with key/value pair, assigned randomly by
1886  mapping function of the overlay. These systems wastes both storage and bandwidth, and  mapping function of the overlay. These systems waste both storage and bandwidth, and
1887  are sensitive to certain attacks (e.g., DDoS attack). Additionally, we emphasize that we prefer \emph{abstraction}  are sensitive to certain attacks (e.g., DDoS attack). Additionally, we emphasize that we prefer \emph{abstraction}
1888  level analysis as very recently better and better tightly structured algorihtms have been proposed.  level analysis as very recently better and better tightly structured algorithms have been proposed.
1889  Thus, we don't want to bind our system proposal to a specific algorithm definitively as we expect  Thus, we don't want to bind our system proposal to a specific algorithm definitively as we expect
1890  that this development continues.  that this development continues.
1891    
# Line 1934  Finally, we assume that all local operat Line 1916  Finally, we assume that all local operat
1916  \end{itemize}  \end{itemize}
1917    
1918  Figure \ref{fig:storm_query_blockid} illustrates how Storm scroll block is located  Figure \ref{fig:storm_query_blockid} illustrates how Storm scroll block is located
1919  in a tightly structured overlay using the DOLR abstraction, where identifier of Storm scroll  in tightly structured overlay using the DOLR abstraction, where identifier of Storm scroll
1920  block is known.  block is known.
1921    
1922    
# Line 1959  block is known. Line 1941  block is known.
1941  \end{itemize}  \end{itemize}
1942    
1943  Figure \ref{fig:storm_query_urn5} illustrates how Storm scroll block is located  Figure \ref{fig:storm_query_urn5} illustrates how Storm scroll block is located
1944  in a tightly structured overlay using the DOLR abstraction, where pointer random string is known.  in tightly structured overlay using the DOLR abstraction, where pointer random string is known.
1945    
1946  Each of these algorithms can locate Fenfire related data in $O(\log{n})$ time at application level overlay:  Each of these algorithms can locate Fenfire data in $O(\log{n})$ time at application level overlay:
1947  $O(\log{n})$ time for query routing to pointer peer and constant time for  $O(\log{n})$ time for query routing to pointer peer and constant time for
1948  locating hosting peer with a given reference link.  locating hosting peer with a given reference link.
1949    
# Line 1987  Perhaps the most biggest issue in Peer-t Line 1969  Perhaps the most biggest issue in Peer-t
1969  security technologies. For instance, online entities cannot be identified  security technologies. For instance, online entities cannot be identified
1970  safely (e.g., the Sybil attack \cite{douceur02sybil}). For Fenfire, one  safely (e.g., the Sybil attack \cite{douceur02sybil}). For Fenfire, one
1971  security related problem occurs when user wants to perform global data lookup with a given  security related problem occurs when user wants to perform global data lookup with a given
1972  pointer random string; how the user is able to verify the correctness  pointer random string; how can the user verify the correctness
1973  of the search results ? Specifically, how she/he knows which one is the  of the search results ? Specifically, how she or he knows which one is the
1974  correct Storm scroll block ? Spam attack \cite{naor03simpledht} is a variation of previously  correct Storm scroll block ? Spam attack \cite{naor03simpledht} is a variation of previously
1975  mentioned problem; data lookup is performed by a user, but there is no reply  mentioned problem; data lookup is performed by a user, but there is no reply
1976  from the system. How we are able to know if this was a spam attack, or the  from the system. How are we able to know if this was a spam attack, or the
1977  data really doesn't exist in the system ? Another problem related to Fenfire's  data really doesn't exist in the system ? Another problem related to the Fenfire's
1978  security is that if a user downloads data from the network to local computer  security is that if a user downloads data from the network to local computer
1979  and after network disconnection, user wants to verify \emph{off line} the  and after network disconnection, user wants to verify \emph{off line} the
1980  authenticity of data. Obviously, optimal solution to all security issues would  authenticity of data. Obviously, optimal solution to all security issues would
1981  be that digital signatures are included to every message sent to the system therefore  be that digital signatures are included to every message sent to the system therefore
1982  enabling peers to authenticate other peers safely. However, these problems are not  enabling peers to authenticate other peers safely. However, these problems are not
1983  only limited to Fenfire as it concerns all Peer-to-Peer based computer systems.  only limited to the Fenfire as it concerns all Peer-to-Peer based computer systems.
1984    
1985  As security technologies come more mature, we wish to apply these  As security technologies come more mature, we wish to apply these
1986  technologies with Fenfire, if applicable.  technologies with Fenfire, if applicable.
# Line 2013  sub-categories have number of open probl Line 1995  sub-categories have number of open probl
1995  yet, or solutions are only partial. We point out that much research work is required to  yet, or solutions are only partial. We point out that much research work is required to
1996  solve these problems.  solve these problems.
1997    
1998  Then, we focused our attention to Fenfire system. First, we gave a brief  Then, we focused our attention to the Fenfire system. First, we gave a brief
1999  overview of Fenfire and xanalogical model. We also described Storm software module.  overview of Fenfire and xanalogical model. We also described Storm software module.
2000    
2001  In the last chapter, we evaluated existing Peer-to-Peer approaches with regard  In the last chapter, we evaluated existing Peer-to-Peer approaches with regard
2002  to Fenfire's needs. We proposed that tightly structured approach is the  to Fenfire's needs. We proposed that the tightly structured approach is the
2003  best alternative to Fenfire's needs for the following reasons. First, Storm, xanalogical  best alternative to Fenfire's needs for the following reasons. First, Storm, xanalogical
2004  model and tightly structured systems use global unique identifiers  model and tightly structured systems use global unique identifiers
2005  for identifying data. Second, our Storm design uses \emph{semantic-free references}  for identifying data. Second, our Storm design uses \emph{semantic-free references}
# Line 2027  hash \cite{fips-sha-1}. As the authors o Line 2009  hash \cite{fips-sha-1}. As the authors o
2009  we also agree that tightly structured overlays provide general purpose  we also agree that tightly structured overlays provide general purpose
2010  interface to next-generation reference resolution services. Third, by using  interface to next-generation reference resolution services. Third, by using
2011  the DOLR abstraction of tightly structured overlay, we can minimize the lack  the DOLR abstraction of tightly structured overlay, we can minimize the lack
2012  of locality in tightly structured approach. Finally, we believe that issues  of locality in the tightly structured approach. Finally, we believe that issues
2013  related to tightly structured overlays are solved in near future, because of  related to tightly structured overlays are solved in near future, because of
2014  wide and intensive co-operation among research groups.  wide and intensive co-operation among research groups.
2015    

Legend:
Removed from v.1.152  
changed lines
  Added in v.1.153

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26