/[gzz]/gzz/Documentation/misc/hemppah-progradu/masterthesis.tex
ViewVC logotype

Diff of /gzz/Documentation/misc/hemppah-progradu/masterthesis.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.128 by hemppah, Fri Mar 7 13:23:37 2003 UTC revision 1.129 by hemppah, Wed Mar 12 07:47:26 2003 UTC
# Line 94  properties of Peer-to-Peer network, whil Line 94  properties of Peer-to-Peer network, whil
94  ''The user's machine is a client and a server'' describes best Peer-to-Peer  ''The user's machine is a client and a server'' describes best Peer-to-Peer
95  systems. To summarize, Peer-to-Peer systems can be characterized as distributed  systems. To summarize, Peer-to-Peer systems can be characterized as distributed
96  systems in which all communication is symmetric and all participants have identical  systems in which all communication is symmetric and all participants have identical
97  capabilities and responsabilities. Each \emph{peer} may contribute data or  capabilities and responsibilities. Each \emph{peer} may contribute data or
98  computing resources (e.g., unused storage) to the overall system and the welfare  computing resources (e.g., unused storage) to the overall system and the welfare
99  of the community can scale with the number of participants. Thus, each participant  of the community can scale with the number of participants. Thus, each participant
100  rely on one another's services and resources, rather than solely relying on dedicated  rely on one another's services and resources, rather than solely relying on dedicated
101  and centralized infracstructure.  and centralized infrastructure.
102    
103  One of the most important properties of any distributed computing system are efficient  One of the most important properties of any distributed computing system are efficient
104  data lookup and security. In this thesis, we\footnote{Use of the plural is customary even if research paper is authored solely.}  data lookup and security. In this thesis, we\footnote{Use of the plural is customary even if research paper is authored solely.}
105  focus on these aspects in Peer-to-Peer domain.  focus on these aspects in Peer-to-Peer domain.
106  Specifically, we review existing Peer-to-Peer approaches, algorithms and their key properties. We observe  Specifically, we review existing Peer-to-Peer approaches, algorithms and their key properties. We observe
107  that despite of greate amount of proposed Peer-to-Peer systems, all systems fall either  that despite of create amount of proposed Peer-to-Peer systems, all systems fall either
108  loosely structured approach or tightly structured approach. We also discuss open problems in  loosely structured approach or tightly structured approach. We also discuss open problems in
109  Peer-to-Peer systems and divide problems into three sub-categories: security related problems,  Peer-to-Peer systems and divide problems into three sub-categories: security related problems,
110  performance related problems and miscellaneous problems. In the end, we summarize all  performance related problems and miscellaneous problems. In the end, we summarize all
# Line 116  Peer-to-Peer functionality. We evaluate Line 116  Peer-to-Peer functionality. We evaluate
116  choose the best alternative to Fenfire's needs. We discover that Fenfire, xanalogical model and  choose the best alternative to Fenfire's needs. We discover that Fenfire, xanalogical model and
117  tightly structured Peer-to-Peer approach all have similar method to deal with data,  tightly structured Peer-to-Peer approach all have similar method to deal with data,
118  i.e., globally unique identifiers. Finally, we propose system model for Fenfire in Peer-to-Peer  i.e., globally unique identifiers. Finally, we propose system model for Fenfire in Peer-to-Peer
119  environment and present yet simple but efficient algortihms to be used for data lookups in  environment and present yet simple but efficient algorithms to be used for data lookups in
120  Peer-to-Peer environment.  Peer-to-Peer environment.
121    
122  To our knowledge, this thesis is the most comprehensive work with regard to summarizing  To our knowledge, this thesis is the most comprehensive work with regard to summarizing
# Line 178  regard to OSI model. Line 178  regard to OSI model.
178  \label{fig:application_level}  \label{fig:application_level}
179  \end{figure}  \end{figure}
180    
181  Compared to ARPANET's Peer-to-Peer functionality, modern Peer-to-Peer systems  Compared to ARPA Net's Peer-to-Peer functionality, modern Peer-to-Peer systems
182  are ad-hoc, i.e., peers join and leave the system constantly in a dynamic manner. This  are ad-hoc, i.e., peers join and leave the system constantly in a dynamic manner. This
183  fact constitutes challenging requirements for efficient construction and maintenance  fact constitutes challenging requirements for efficient construction and maintenance
184  of the overlay network. Even more demanding tasks are how to perform efficient data  of the overlay network. Even more demanding tasks are how to perform efficient data
185  lookup and maintain security in a varying distributed environment. The most popular  lookup and maintain security in a varying distributed environment. The most popular
186  form of modern Peer-to-Peer computing is file-sharing. In this scenario, participants  form of modern Peer-to-Peer computing is file-sharing. In this scenario, participants
187  of Peer-to-Peer network share their resources to other participants while obtaining  of Peer-to-Peer network share their resources to other participants while obtaining
188  more resources from others. This can been seen as a variant of distributed filesystem  more resources from others. This can been seen as a variant of distributed file system
189  (e.g., \cite{levy90distributedfilesystems}).  (e.g., \cite{levy90distributedfilesystems}).
190    
191  In a development of modern Peer-to-Peer systems, lot of influences has been attained from  In a development of modern Peer-to-Peer systems, lot of influences has been attained from
# Line 193  other research areas than computer scien Line 193  other research areas than computer scien
193  to ad-hoc nature of complex networks \cite{albert-02-statistical}, \cite{albert-00-tolerance}, \cite{watts00dynamics}.  to ad-hoc nature of complex networks \cite{albert-02-statistical}, \cite{albert-00-tolerance}, \cite{watts00dynamics}.
194  It's interesting to realize that chemical properties of cells, the Internet, ad-hoc  It's interesting to realize that chemical properties of cells, the Internet, ad-hoc
195  Peer-to-Peer systems, have all in common that they self-organize based on same  Peer-to-Peer systems, have all in common that they self-organize based on same
196  principles.  Furthermore, the assocation between social connections among people  principles.  Furthermore, the association between social connections among people
197  and Peer-to-Peer overlay topology has been studied recently  \cite{watts00dynamics},  and Peer-to-Peer overlay topology has been studied recently  \cite{watts00dynamics},
198  \cite{kleinberg99small}, \cite{nips02-Kleinberg}. This insight is motivated  \cite{kleinberg99small}, \cite{nips02-Kleinberg}. This insight is motivated
199  by Milgram, how noticed that people are very effective to locate other people in a wide scale  by Milgram, how noticed that people are very effective to locate other people in a wide scale
# Line 206  In the end, however, there are two main Line 206  In the end, however, there are two main
206  systems fall: loosely structured approach and tightly structured approach. In loosely  systems fall: loosely structured approach and tightly structured approach. In loosely
207  structured approach the construction and the maintenance of the overlay-- as the name  structured approach the construction and the maintenance of the overlay-- as the name
208  suggests- is controlled loosely. This approach gives freedom for participating peers  suggests- is controlled loosely. This approach gives freedom for participating peers
209  to perform certains tasks in Peer-to-Peer network. On the other hand, tightly structured  to perform certain tasks in Peer-to-Peer network. On the other hand, tightly structured
210  approach has some rules, which all participating peers have to obey. In the following  approach has some rules, which all participating peers have to obey. In the following
211  sections, we discuss in more detail both approaches, they disadvantages and advantages, and  sections, we discuss in more detail both approaches, they disadvantages and advantages, and
212  key differences.  key differences.
# Line 244  other nodes in a random manner. Thus, gn Line 244  other nodes in a random manner. Thus, gn
244    
245  In Gnutella, each participating peer maintains local index of its own shared content. Also,  In Gnutella, each participating peer maintains local index of its own shared content. Also,
246  each peer has a few connections to other peer, i.e., peer's \emph{neighbors}. Basic gnutella  each peer has a few connections to other peer, i.e., peer's \emph{neighbors}. Basic gnutella
247  data lookup works as follows: peer broadcasts a query request to its neighors, which in turn  data lookup works as follows: peer broadcasts a query request to its neighbors, which in turn
248  forwards the query to their neighbors. This leads in the situation where number of messages  forwards the query to their neighbors. This leads in the situation where number of messages
249  in the network can grow with $O(n^{2})$, where $n$ is the number of participating peers in the  in the network can grow with $O(n^{2})$, where $n$ is the number of participating peers in the
250  Gnutella network. To limit the amount of network traffic, Gnutella uses Time-To-Live-limited  Gnutella network. To limit the amount of network traffic, Gnutella uses Time-To-Live-limited
251  (TTL) flooding to distributed queries. Gnutella uses a Breadt-First-Search (BFS) with depth limit  (TTL) flooding to distributed queries. Gnutella uses a Breadth-First-Search (BFS) with depth limit
252  $T$ (e.g., 7), where $T$ is the system-wide maximum TTL of a message in hops. Therefore, only peers that  $T$ (e.g., 7), where $T$ is the system-wide maximum TTL of a message in hops. Therefore, only peers that
253  are TTL hops away from the query originator will forward the query or respond to the query.  are TTL hops away from the query originator will forward the query or respond to the query.
254  In Gnutella network, search results are fast, because BFS sends queries to  In Gnutella network, search results are fast, because BFS sends queries to
# Line 263  Figure \ref{fig:gnutella_query} shows th Line 263  Figure \ref{fig:gnutella_query} shows th
263  \end{figure}  \end{figure}
264    
265  According to \cite{lv02searchreplication}, Gnutella's way to perform data lookups, \emph{flooding}, has  According to \cite{lv02searchreplication}, Gnutella's way to perform data lookups, \emph{flooding}, has
266  following limitations. First, choosing the approriate TTL in practice is not easy. If the  following limitations. First, choosing the appropriate TTL in practice is not easy. If the
267  TTL is too high, query originator may unnecessarily strain the network. If the TTL is too  TTL is too high, query originator may unnecessarily strain the network. If the TTL is too
268  low, the query originator might not find the desired data even it's available somewhere  low, the query originator might not find the desired data even it's available somewhere
269  in the network. Second, there are many duplicate messages generated by flooding, especially  in the network. Second, there are many duplicate messages generated by flooding, especially
# Line 277  and scalability. Adamic et al. \cite{ada Line 277  and scalability. Adamic et al. \cite{ada
277  networks\footnote{In power-law networks only a few peers have high number of neighbor  networks\footnote{In power-law networks only a few peers have high number of neighbor
278  links and major of peers have low number of neighbor links.} and have found that by  links and major of peers have low number of neighbor links.} and have found that by
279  instructing peers forwarding data lookups to select high degree peers, the performance of data lookup  instructing peers forwarding data lookups to select high degree peers, the performance of data lookup
280  increases signficantly. As a result, some of the most recent loosely  increases significantly. As a result, some of the most recent loosely
281  structured Peer-to-Peer systems have adopted this method with some modifications  structured Peer-to-Peer systems have adopted this method with some modifications
282  \cite{gnutella2url}, \cite{shareazaurl}, \cite{fasttrackurl}, \cite{morpheusurl},  \cite{gnutella2url}, \cite{shareazaurl}, \cite{fasttrackurl}, \cite{morpheusurl},
283  \cite{kazaaurl}, \cite{jxtaurl}, \cite{jxtaoverview}, \cite{botros01jxtasearch},  \cite{kazaaurl}, \cite{jxtaurl}, \cite{jxtaoverview}, \cite{botros01jxtasearch},
# Line 311  loosely structured systems are presented Line 311  loosely structured systems are presented
311    
312  \subsection{Sketch of formal definition}  \subsection{Sketch of formal definition}
313    
314  In this subsection we formalize loosely strucured overlay's main components. This  In this subsection we formalize loosely structured overlay's main components. This
315  model is based on original Gnutella overlay network with power-law improvements.  model is based on original Gnutella overlay network with power-law improvements.
316    
317  Let $S$ be the aggregate of all services $s$ in system. Let $P$ be the aggregate of  Let $S$ be the aggregate of all services $s$ in system. Let $P$ be the aggregate of
# Line 319  all peers $p$ in system. Then, $\forall Line 319  all peers $p$ in system. Then, $\forall
319  expressed as $p = provider(s)$. Every $p$ has neighbor(s), named as $neighbor$, which  expressed as $p = provider(s)$. Every $p$ has neighbor(s), named as $neighbor$, which
320  is $P$ = \{$p \in P: \exists neighbor$, which is randomly chosen from $P$\}.  is $P$ = \{$p \in P: \exists neighbor$, which is randomly chosen from $P$\}.
321  \emph{Super peer} is a peer, which hosts the indices of other peers, $sp = summaryindex(provider(s))$.  \emph{Super peer} is a peer, which hosts the indices of other peers, $sp = summaryindex(provider(s))$.
322  Moreover, $\forall$ reqular peer $p$, there is super peer, which has has a index of regular  Moreover, $\forall$ regular peer $p$, there is super peer, which has has a index of regular
323  peer's content, specifically $ps$, $P$ = \{$p \in P: \exists ps$,  peer's content, specifically $ps$, $P$ = \{$p \in P: \exists ps$,
324  where $ps$ = $summaryindex(provider(s)) \wedge (p = provider(s))$\}  where $ps$ = $summaryindex(provider(s)) \wedge (p = provider(s))$\}
325    
# Line 337  Symphony \cite{gurmeet03symphony}, SWAN Line 337  Symphony \cite{gurmeet03symphony}, SWAN
337  \cite{zhao01tapestry}, Viceroy \cite{malkhi02viceroy} and others \cite{freedman02trie}.  \cite{zhao01tapestry}, Viceroy \cite{malkhi02viceroy} and others \cite{freedman02trie}.
338  The biggest difference compared to loosely structured approach is that with tightly structured systems,  The biggest difference compared to loosely structured approach is that with tightly structured systems,
339  it is now feasible to perform \emph{global} data lookups in the overlay.  it is now feasible to perform \emph{global} data lookups in the overlay.
340  While there are significat differences among proposed systems, they all have in common  While there are significant differences among proposed systems, they all have in common
341  that \emph{peer identifiers} is assigned to participating peers from  that \emph{peer identifiers} is assigned to participating peers from
342  a large \emph{identifier space} by the overlay. Furthermore, application-specific  a large \emph{identifier space} by the overlay. Furthermore, application-specific
343  data items are also assigned globally unique identifiers, \emph{keys},  data items are also assigned globally unique identifiers, \emph{keys},
# Line 346  space differs between proposed systems. Line 346  space differs between proposed systems.
346  is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde},  is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde},
347  Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry}  Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry}
348  and Viceroy \cite{malkhi02viceroy} use circular identifier space of $n$-bit integers modulo $2^{n}$. The  and Viceroy \cite{malkhi02viceroy} use circular identifier space of $n$-bit integers modulo $2^{n}$. The
349  value of $n$ varies among systems. Again, CAN \cite{ratnasamy01can} uses a $d$-dimensional cartesian  value of $n$ varies among systems. Again, CAN \cite{ratnasamy01can} uses a $d$-dimensional Cartesian
350  model to implement identifier space.  model to implement identifier space.
351    
352  To store data into tightly structured overlay, each application-specific  To store data into tightly structured overlay, each application-specific
# Line 356  structured overlay assigns a subset of a Line 356  structured overlay assigns a subset of a
356  Also, each peer in tightly structured overlay maintains a \emph{routing table}, which  Also, each peer in tightly structured overlay maintains a \emph{routing table}, which
357  consists of identifiers and IP addresses of other peers in the overlay. Entries of routing  consists of identifiers and IP addresses of other peers in the overlay. Entries of routing
358  table are peer's neighbors in the overlay network. Figure \ref{fig:structured_hashing} illustrates the  table are peer's neighbors in the overlay network. Figure \ref{fig:structured_hashing} illustrates the
359  process of data to key mapping in tightly strucuted overlays.  process of data to key mapping in tightly structured overlays.
360    
361  \begin{figure}  \begin{figure}
362  \centering  \centering
# Line 378  is $d$) and symmetric (the distance from Line 378  is $d$) and symmetric (the distance from
378  distance from $p_j$ to $p_i$) \cite{maymounkov02kademlia}. On the other  distance from $p_j$ to $p_i$) \cite{maymounkov02kademlia}. On the other
379  hand, Chord's \cite{stoica01chord} distance function does have the property  hand, Chord's \cite{stoica01chord} distance function does have the property
380  of unidirection, but doesn't have symmetry. Pastry's \cite{rowston01pastry} distance  of unidirection, but doesn't have symmetry. Pastry's \cite{rowston01pastry} distance
381  function supports symmetry, but doesn't support unidirection. As a concequence,  function supports symmetry, but doesn't support unidirection. As a consequence,
382  Kademlia's \cite{maymounkov02kademlia} XOR-based metric doesn't need  Kademlia's \cite{maymounkov02kademlia} XOR-based metric doesn't need
383  stabilization (like in Chord \cite{stoica01chord}) and backup links  stabilization (like in Chord \cite{stoica01chord}) and backup links
384  (like in Pastry \cite{rowston01pastry}) \cite{balakrishanarticle03lookupp2p}.  (like in Pastry \cite{rowston01pastry}) \cite{balakrishanarticle03lookupp2p}.
# Line 407  four requirements for tightly structured Line 407  four requirements for tightly structured
407  addressed in order to perform efficient data lookups in tightly structured overlays.  addressed in order to perform efficient data lookups in tightly structured overlays.
408  First, mapping of keys to peers must be done in a load-balanced  First, mapping of keys to peers must be done in a load-balanced
409  way. Second, the overlay must be able to forward a lookup for a  way. Second, the overlay must be able to forward a lookup for a
410  specific key to an approriate peer. Third, overlay must have a  specific key to an appropriate peer. Third, overlay must have a
411  support for a distance function. Finally,  routing tables for each peer  support for a distance function. Finally,  routing tables for each peer
412  must be constructed and maintained adaptively.  must be constructed and maintained adaptively.
413    
414  Currently, all proposed tightly structured overlays provide at least  Currently, all proposed tightly structured overlays provide at least
415  poly--logaritmical data lookup operations. However, there are some key  poly--logarithmical data lookup operations. However, there are some key
416  differences in the data structure that they use as a routing table. For example, Chord  differences in the data structure that they use as a routing table. For example, Chord
417  \cite{stoica01chord}, Skip graphs \cite{AspnesS2003} and Skipnet \cite{harvey03skipnet2} maintain a local  \cite{stoica01chord}, Skip graphs \cite{AspnesS2003} and Skipnet \cite{harvey03skipnet2} maintain a local
418  data structure which resembles Skip lists \cite{78977}.  data structure which resembles Skip lists \cite{78977}.
419  In figure \ref{fig:structured_query}, we present overview of Chord's lookup process.  In figure \ref{fig:structured_query}, we present overview of Chord's lookup process.
420  On the left side of Chord's lookup process, we show the same data lookup process  On the left side of Chord's lookup process, we show the same data lookup process
421  as binary-tree abstraction.  We can notice, that in each step, the distance between  as binary-tree abstraction.  We can notice, that in each step, the distance between
422  locarithmic efficiency.  logarithmic efficiency.
423    
424  Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry  Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry
425  \cite{zhao01tapestry} uses balanced $k$-trees as routing table's data structure. Figure  \cite{zhao01tapestry} uses balanced $k$-trees as routing table's data structure. Figure
# Line 447  about two links to other peers to to pro Line 447  about two links to other peers to to pro
447    
448    
449  There are three higher level abstractions which tightly structured overlays provide  There are three higher level abstractions which tightly structured overlays provide
450  \cite{zhao03api}. Each of these abstractions fulfil a storage layer in an overlay, but  \cite{zhao03api}. Each of these abstractions fulfill a storage layer in an overlay, but
451  they have semantical differences in the \emph{usage} of overlay. First, Distributed Hash  they have semantical differences in the \emph{usage} of overlay. First, Distributed Hash
452  Table (DHT) (see e.g., \cite{dabek01widearea}, \cite{rowstron01storage}),    Table (DHT) (see e.g., \cite{dabek01widearea}, \cite{rowstron01storage}),  
453  implements three operations: \texttt{lookup(key)}, \texttt{remove(key)} and  implements three operations: \texttt{lookup(key)}, \texttt{remove(key)} and
454  \texttt{insert(key)}. As the name suggests, DHT implements the same functionality  \texttt{insert(key)}. As the name suggests, DHT implements the same functionality
455  as a regular hashtable, by storing the mapping between a key and a value. DHT's  as a regular hash table, by storing the mapping between a key and a value. DHT's
456  \emph{interface} is generic; values can be any size and type. Figure \ref{fig:Structured_lookup_using_DHT_model}  \emph{interface} is generic; values can be any size and type. Figure \ref{fig:Structured_lookup_using_DHT_model}
457  shows the DHT abstraction of tightly structured overlay. Second, Decentralized  shows the DHT abstraction of tightly structured overlay. Second, Decentralized
458  Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is distributed  Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is distributed
# Line 462  throughout the overlay. DOLR's main oper Line 462  throughout the overlay. DOLR's main oper
462  difference between DHT and DOLR abstraction is that DOLR routes overlay's messages  difference between DHT and DOLR abstraction is that DOLR routes overlay's messages
463  to nearest available peer, hosting a specific data item. This form of locality  to nearest available peer, hosting a specific data item. This form of locality
464  is not supported by DHT. Finally, tightly structured overlay can be used for  is not supported by DHT. Finally, tightly structured overlay can be used for
465  scalable group multicast/anycast operations (CAST) (see e.g., \cite{zhuang01bayeux}).  scalable group multicast/any cast operations (CAST) (see e.g., \cite{zhuang01bayeux}).
466  The basic operations are \texttt{join(groupIdentifier)}, \texttt{leave(groupIdentifier)},  The basic operations are \texttt{join(groupIdentifier)}, \texttt{leave(groupIdentifier)},
467  \texttt{multicast(message, groupIdentifier)},  \texttt{anycast(message, groupIdentifier)}.  \texttt{multicast(message, groupIdentifier)},  \texttt{anycast(message, groupIdentifier)}.
468  Participating peers may join and leave the group and send multicast messages to  Participating peers may join and leave the group and send multicast messages to
469  the group, or anycast message to a specific member of the group. DOLR and CAST abstraction  the group, or any-cast message to a specific member of the group. DOLR and CAST abstraction
470  have much in common. For instance, they both use network proximity techniques  have much in common. For instance, they both use network proximity techniques
471  to optimize their operation in the overlay. Figure \ref{fig:Strucutred_lookup_using_DOLR_model}  to optimize their operation in the overlay. Figure \ref{fig:Strucutred_lookup_using_DOLR_model}
472   presents basic operation of DOLR abstraction.   presents basic operation of DOLR abstraction.
# Line 482  to optimize their operation in the overl Line 482  to optimize their operation in the overl
482  \begin{figure}  \begin{figure}
483  \centering  \centering
484  \includegraphics[width=10cm, height=8cm]{DOLR_lookup.eps}  \includegraphics[width=10cm, height=8cm]{DOLR_lookup.eps}
485  \caption{Decentralized Object Lcation (DOLR) abstraction of tightly structured overlay.}  \caption{Decentralized Object Location (DOLR) abstraction of tightly structured overlay.}
486  \label{fig:Strucutred_lookup_using_DOLR_model}  \label{fig:Strucutred_lookup_using_DOLR_model}
487  \end{figure}  \end{figure}
488    
489    
490  \subsection{Sketch of formal definition}  \subsection{Sketch of formal definition}
491    
492  In this subsection we formalize tightly strucured overlay's main features. The model  In this subsection we formalize tightly structured overlay's main features. The model
493  describes basic features of tightly structured overlay, i.e., identifiers, identifier  describes basic features of tightly structured overlay, i.e., identifiers, identifier
494  space and mapping function.  space and mapping function.
495    
# Line 535  structured approach has certain features Line 535  structured approach has certain features
535  (such as mapping of data items).  (such as mapping of data items).
536    
537  To end user, biggest difference between these systems is how data lookups are performed. Loosely  To end user, biggest difference between these systems is how data lookups are performed. Loosely
538  structured systems provide much more richier and user friendly way of searching data as they  structured systems provide much more richer and user friendly way of searching data as they
539  have support for keyword search and fuzzy search. On the other hand, tightly structured systems support  have support for keyword search and fuzzy search. On the other hand, tightly structured systems support
540  only exact key lookups as each data item is identified by globally unique keys.  only exact key lookups as each data item is identified by globally unique keys.
541    
# Line 620  approach and tightly structured approach Line 620  approach and tightly structured approach
620  \parbox{100pt}{Partial}  \parbox{100pt}{Partial}
621  \\ \hline  \\ \hline
622                        
623  \parbox{90pt}{Possibility for routing hotspots} &  \parbox{90pt}{Possibility for routing hot spots} &
624  \parbox{100pt}{No} &  \parbox{100pt}{No} &
625  \parbox{100pt}{Yes}  \parbox{100pt}{Yes}
626  \\ \hline  \\ \hline
# Line 636  approach and tightly structured approach Line 636  approach and tightly structured approach
636  \\ \hline  \\ \hline
637    
638    
639  \caption{Comparison of loosely structured and tighly structured approaches}  \caption{Comparison of loosely structured and tightly structured approaches}
640  \label{table_comparison_approach}  \label{table_comparison_approach}
641    
642    
# Line 720  Here, we describe the listed properties Line 720  Here, we describe the listed properties
720  \parbox{37pt}{$O(1)$} &  \parbox{37pt}{$O(1)$} &
721  \parbox{37pt}{$O(n)$} &  \parbox{37pt}{$O(n)$} &
722  \parbox{85pt}{Typical configuration e.g., {4--150}} &  \parbox{85pt}{Typical configuration e.g., {4--150}} &
723  \parbox{85pt}{Average lookup performance is $O(\log{n})$ with tens of thousands concurrent users, beyond that, the performace is $O(n)$}  \parbox{85pt}{Average lookup performance is $O(\log{n})$ with tens of thousands concurrent users, beyond that, the performance is $O(n)$}
724  \\ \hline  \\ \hline
725    
726    
# Line 763  Here, we describe the listed properties Line 763  Here, we describe the listed properties
763  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
764  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
765  \parbox{85pt}{$2(\log{n})$} &  \parbox{85pt}{$2(\log{n})$} &
766  \parbox{85pt}{There are two lookup algorithms. The other is $O(\log{n})$, which is robus under random deletion. The second is $O(\log^2{n})$, which is also robust under spam generating model}  \parbox{85pt}{There are two lookup algorithms. The other is $O(\log{n})$, which is robust under random deletion. The second is $O(\log^2{n})$, which is also robust under spam generating model}
767  \\ \hline  \\ \hline
768    
769    
# Line 789  Here, we describe the listed properties Line 789  Here, we describe the listed properties
789  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
790  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
791  \parbox{85pt}{$O(\log{n})$} &  \parbox{85pt}{$O(\log{n})$} &
792  \parbox{85pt}{Plaxton's algortihm is designed to operate in static environment (e.g., web cache)}  \parbox{85pt}{Plaxton's algorithm is designed to operate in static environment (e.g., web cache)}
793  \\ \hline  \\ \hline
794    
795  \parbox{37pt}{Skip Graphs \cite{AspnesS2003}} &  \parbox{37pt}{Skip Graphs \cite{AspnesS2003}} &
# Line 797  Here, we describe the listed properties Line 797  Here, we describe the listed properties
797  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
798  \parbox{37pt}{$O(\log{n})$} &  \parbox{37pt}{$O(\log{n})$} &
799  \parbox{85pt}{$4r(\log{n}) + (\log{n})$, where r=number of resources provided)} &  \parbox{85pt}{$4r(\log{n}) + (\log{n})$, where r=number of resources provided)} &
800  \parbox{85pt}{In this approach, node is treated as 'named resource'; in this approach, \emph{resources} self-organise (opposite to DHTs)}  \parbox{85pt}{In this approach, node is treated as 'named resource'; in this approach, \emph{resources} self-organize (opposite to DHTs)}
801  \\ \hline  \\ \hline
802    
803  \parbox{37pt}{SkipNet \cite{harvey03skipnet2}} &  \parbox{37pt}{SkipNet \cite{harvey03skipnet2}} &
# Line 812  Here, we describe the listed properties Line 812  Here, we describe the listed properties
812  \parbox{37pt}{$O(1)$} &  \parbox{37pt}{$O(1)$} &
813  \parbox{37pt}{$O(1)$} &  \parbox{37pt}{$O(1)$} &
814  \parbox{37pt}{$O(n)$} &  \parbox{37pt}{$O(n)$} &
815  \parbox{85pt}{Can be 1-10000 connections (aka social connections, connections are permament)} &  \parbox{85pt}{Can be 1-10000 connections (aka social connections, connections are permanent)} &
816  \parbox{85pt}{Connection number depends on node's memory/network capabilities}  \parbox{85pt}{Connection number depends on node's memory/network capabilities}
817  \\ \hline  \\ \hline
818    
# Line 829  Here, we describe the listed properties Line 829  Here, we describe the listed properties
829  \parbox{37pt}{$O(1)$} &  \parbox{37pt}{$O(1)$} &
830  \parbox{37pt}{$O(\log^2{n})$} &  \parbox{37pt}{$O(\log^2{n})$} &
831  \parbox{85pt}{$r(2b+2s+2l)$ (where r=number of resources provided, b=boot connections, s=short range connections, l=long range connections), typical connection configuration: 2*(6+7+8)=36} &  \parbox{85pt}{$r(2b+2s+2l)$ (where r=number of resources provided, b=boot connections, s=short range connections, l=long range connections), typical connection configuration: 2*(6+7+8)=36} &
832  \parbox{85pt}{In this approach, node is treated as 'named resource'; in this approach, \emph{resources} self-organise (opposite to DHTs)}  \parbox{85pt}{In this approach, node is treated as 'named resource'; in this approach, \emph{resources} self-organize (opposite to DHTs)}
833  \\ \hline  \\ \hline
834    
835    
# Line 883  to public, researchers' main concern has Line 883  to public, researchers' main concern has
883  approach. However, people often misunderstand the scalability problem of loosely structured  approach. However, people often misunderstand the scalability problem of loosely structured
884  approach; \emph{network} of loosely structured systems is scalable, but the \emph{data lookup model} is not.  approach; \emph{network} of loosely structured systems is scalable, but the \emph{data lookup model} is not.
885  The main concern of tightly structured system is to make overlay's data lookup  The main concern of tightly structured system is to make overlay's data lookup
886  routing more flexible againts hostile attacks. Another key problems in tightly structured  routing more flexible against hostile attacks. Another key problems in tightly structured
887  systems are the lack of keyword searches, support for heterogeneous peers and load balancing  systems are the lack of keyword searches, support for heterogeneous peers and load balancing
888  \cite{balakrishanarticle03lookupp2p}.  \cite{balakrishanarticle03lookupp2p}.
889    
# Line 898  In this section we discuss security prob Line 898  In this section we discuss security prob
898    
899  \subsection{Attacks}  \subsection{Attacks}
900    
901  There are five well known attack models againts Peer-to-Peer systems: Sybil attack \cite{douceur02sybil},  There are five well known attack models against Peer-to-Peer systems: Sybil attack \cite{douceur02sybil},
902  Fail-stop attack, Spam attack \cite{naor03simpledht}, Byzantine problem \cite{357176} and \cite{296824}, and  Fail-stop attack, Spam attack \cite{naor03simpledht}, Byzantine problem \cite{357176} and \cite{296824}, and
903  general Distrubuted Denial of Service attack.  general Distributed Denial of Service attack.
904    
905  In Sybil attack model, hostile entity presents multiple  In Sybil attack model, hostile entity presents multiple
906  entities. Therefore, one hostile entity can control a large fraction of the Peer-to-Peer system. Optimal  entities. Therefore, one hostile entity can control a large fraction of the Peer-to-Peer system. Optimal
907  possible solution to Sybil attack would be that system could \emph{distinct} entities of the system reliably. Unfortunately,  possible solution to Sybil attack would be that system could \emph{distinct} entities of the system reliably. Unfortunately,
908  currently there no realizable techiques for this task. Partial solutions for Sybil attack is to replicate  currently there no realizable techniques for this task. Partial solutions for Sybil attack is to replicate
909  and fragment data randomly among several participating peer. However, both suggestions assume that two different  and fragment data randomly among several participating peer. However, both suggestions assume that two different
910  remote entities are actually different; Sybil attacks are still possible and therefore, would need centralized  remote entities are actually different; Sybil attacks are still possible and therefore, would need centralized
911  authority for reliable authentication. As author arques in \cite{douceur02sybil}, without centralized authority,  authority for reliable authentication. As author argues in \cite{douceur02sybil}, without centralized authority,
912  Sybil attacks are always possible in Peer-to-Peer system except under extreme and unrealistic assumptions of  Sybil attacks are always possible in Peer-to-Peer system except under extreme and unrealistic assumptions of
913  resource parity and coordination among entities.  resource parity and coordination among entities.
914    
915  In random fail-stop model, cited in \cite{naor03simpledht}, faulty peer is deleted from the Peer-to-Peer system.  In random fail-stop model, cited in \cite{naor03simpledht}, faulty peer is deleted from the Peer-to-Peer system.
916  The reason for faultyness of peer can be a software failure, a hostile attack, or external threat such as virus or  The reason for faultiness of peer can be a software failure, a hostile attack, or external threat such as virus or
917  troijan. Closely related to fail-stop model is the Byzantine attack model  Trojan. Closely related to fail-stop model is the Byzantine attack model
918  \cite{357176}. Byzantine model can been seen more seveve than fail-stop model as there are no restrictions over  \cite{357176}. Byzantine model can been seen more severe than fail-stop model as there are no restrictions over
919  the behaviour of faulty peers. Practical, but partial solution for byzantine failures has been proposed by Castro et  the behavior of faulty peers. Practical, but partial solution for Byzantine failures has been proposed by Castro et
920  al \cite{296824}.  al \cite{296824}.
921    
922  Spam generating attack is another known attack model againts Peer-to-Peer system. In Spam  Spam generating attack is another known attack model against Peer-to-Peer system. In Spam
923  attack, hostile or faulty peer may produce false information of the data, or refuses/is not able to reply to requests.  attack, hostile or faulty peer may produce false information of the data, or refuses/is not able to reply to requests.
924  Possible solution againts this attack is that peer should not trust to single entity. Instead, peer should get  Possible solution against this attack is that peer should not trust to single entity. Instead, peer should get
925  information from multiple entities and trust on majority's opinion. This methods requires more messages to be  information from multiple entities and trust on majority's opinion. This methods requires more messages to be
926  sent to network while increasing the load of system. However, if Spam attack is combined with Sybil attack, obviously  sent to network while increasing the load of system. However, if Spam attack is combined with Sybil attack, obviously
927  previously mentioned solution doesn't work. Again, more research is required to solve this attack model  previously mentioned solution doesn't work. Again, more research is required to solve this attack model
928  safely. Naor et al. \cite{naor03simpledht} has proposed a partial solution againts Spam attack with  safely. Naor et al. \cite{naor03simpledht} has proposed a partial solution against Spam attack with
929  \emph{faulty} peers (not hostile).  \emph{faulty} peers (not hostile).
930    
931  Traditional overload of targeted peers is best known form of distrubuted Denial of Service attack (DDoS). For example,  Traditional overload of targeted peers is best known form of distributed Denial of Service attack (DDoS). For example,
932  hostile entity can attempt to burden targetted peers with garbage network packets. As a implication, peers may act  hostile entity can attempt to burden targeted peers with garbage network packets. As a implication, peers may act
933  incorrectly or stop working. DDoS attack may be very severe, especially if rate of replication and caching  incorrectly or stop working. DDoS attack may be very severe, especially if rate of replication and caching
934  in Peer-to-Peer system is low. This may lead to data loss in the Peer-to-Peer system. Daswani et al.  in Peer-to-Peer system is low. This may lead to data loss in the Peer-to-Peer system. Daswani et al.
935  \cite{daswani02queryflooddos} has done research regarding to this subject. Authors suggest efficient load balancing  \cite{daswani02queryflooddos} has done research regarding to this subject. Authors suggest efficient load balancing
# Line 938  suggests that identifier assignment algo Line 938  suggests that identifier assignment algo
938  and replicas should be located physically to different locations.  and replicas should be located physically to different locations.
939    
940  As stated in \cite{naor03simpledht}, an important aspect is that when it comes to general security aspects and  As stated in \cite{naor03simpledht}, an important aspect is that when it comes to general security aspects and
941  byzantine faults in any Peer-to-Peer system, there should be a clear distinction between attacks on the  Byzantine faults in any Peer-to-Peer system, there should be a clear distinction between attacks on the
942  algorihms assuming the construction of overlay is correct, and attacks on the construction itself. Clearly, Sybil  algorithms assuming the construction of overlay is correct, and attacks on the construction itself. Clearly, Sybil
943  and Spam attack belongs to the first category, and rest of the attacks to the latter category.  and Spam attack belongs to the first category, and rest of the attacks to the latter category.
944    
945  \subsection{Trust, data authenticity and integrity}  \subsection{Trust, data authenticity and integrity}
946    
947  Trust in Peer-to-Peer systems is based on \emph{reputation}. Proposed repuation methods focus either  Trust in Peer-to-Peer systems is based on \emph{reputation}. Proposed reputation methods focus either
948  on the semantic properties, or data management properties of the trust model. Some research has been  on the semantic properties, or data management properties of the trust model. Some research has been
949  done on reputation models in Peer-to-Peer systems, such as \cite{aberer01trust}, \cite{cornelli02reputableservents}.  done on reputation models in Peer-to-Peer systems, such as \cite{aberer01trust}, \cite{cornelli02reputableservents}.
950  Implementations include Advogato \cite{advogatourl}. None of the current proposals or implementations  Implementations include Advogato \cite{advogatourl}. None of the current proposals or implementations
# Line 956  systems \cite{rivest96sdsi}, \cite{spkiw Line 956  systems \cite{rivest96sdsi}, \cite{spkiw
956  data in rather \emph{static} computing systems, such as in the Internet. However, in Peer-to-Peer  data in rather \emph{static} computing systems, such as in the Internet. However, in Peer-to-Peer
957  network, the problem of key based security mechanism is the maintenance of the keys as participating  network, the problem of key based security mechanism is the maintenance of the keys as participating
958  peer constantly join and leave the system. Specifically, the distribution of key changes comes an essential  peer constantly join and leave the system. Specifically, the distribution of key changes comes an essential
959  problem in ad hoc enviroments. These include revokation of keys and new key distribution in hostile  problem in ad hoc environments. These include revocation of keys and new key distribution in hostile
960  environment.  environment.
961    
962  ConChord \cite{ajmani02conchord} is the first Peer-to-Peer system which has a support for PKI based  ConChord \cite{ajmani02conchord} is the first Peer-to-Peer system which has a support for PKI based
# Line 1001  of the system. These include Line 1001  of the system. These include
1001  Freenet \cite{clarke00freenet}, Publius \cite{pub00}, Free haven \cite{dingledine00free}, Crowds \cite{reiter98crowds},  Freenet \cite{clarke00freenet}, Publius \cite{pub00}, Free haven \cite{dingledine00free}, Crowds \cite{reiter98crowds},
1002  Tangler \cite{502002} and upcoming Mnet \cite{mneturl}. Forwarding proxies are used in Freenet, Crowds and  Tangler \cite{502002} and upcoming Mnet \cite{mneturl}. Forwarding proxies are used in Freenet, Crowds and
1003  Free Haven in order to provide various types of anonymity. Tangler and Publius uses cryptographic  Free Haven in order to provide various types of anonymity. Tangler and Publius uses cryptographic
1004  sharing methods to split a data into data fragments \cite{Shamir1979a}. Mixmailer networks, such as  sharing methods to split a data into data fragments \cite{Shamir1979a}. Mix mailer networks, such as
1005  \cite{mixminionurl}, are commonly used in distributed systems, which are able to provide some level  \cite{mixminionurl}, are commonly used in distributed systems, which are able to provide some level
1006  of anonymity  of anonymity
1007    
# Line 1015  between anonymity and other Peer-to-Peer Line 1015  between anonymity and other Peer-to-Peer
1015  Any distributed computing system must support different levels of access control. For instance, in Peer-to-Peer  Any distributed computing system must support different levels of access control. For instance, in Peer-to-Peer
1016  system, we may want to restrict the accessibility of data to only limited amount of participating peers. Yet, Peer-to-Peer  system, we may want to restrict the accessibility of data to only limited amount of participating peers. Yet, Peer-to-Peer
1017  systems doesn't have working and distributed access control scheme. Moreover,  systems doesn't have working and distributed access control scheme. Moreover,
1018  there has been a lot of violation of copyright laws by users of Peer-to-Peer filesharing systems. As a  there has been a lot of violation of copyright laws by users of Peer-to-Peer file sharing systems. As a
1019  consequence, some lawsuits has been created againts the companies how have build popular file-sharing programs.  consequence, some lawsuits has been created against the companies how have build popular file-sharing programs.
1020    
1021  To our knowledge, Nejdl et al. \cite{nejdl03accesscontrol} have proposed very recently first practical solution to access  To our knowledge, Nejdl et al. \cite{nejdl03accesscontrol} have proposed very recently first practical solution to access
1022  control problem in Peer-to-Peer systems. They use RDF-based schema policies to restrict access to certain  control problem in Peer-to-Peer systems. They use RDF-based schema policies to restrict access to certain
# Line 1049  Much work has been done on secure routin Line 1049  Much work has been done on secure routin
1049  \cite{castro02securitystructured} and \cite{castro02securerouting}, authors suggests the usage  \cite{castro02securitystructured} and \cite{castro02securerouting}, authors suggests the usage
1050  of constrained routing tables and diverse routes, and detection of faults during query routing.  of constrained routing tables and diverse routes, and detection of faults during query routing.
1051  Additionally, authors present a important aspect of tightly structured approach with regard  Additionally, authors present a important aspect of tightly structured approach with regard
1052  to fault-tolerant query routing: the probability of routing succesfully between to arbitrary,  to fault-tolerant query routing: the probability of routing successfully between to arbitrary,
1053  correct peers, when a fraction $f$ of the other peers are faulty or hostile, is only $(1-f)^{h-1}$.  correct peers, when a fraction $f$ of the other peers are faulty or hostile, is only $(1-f)^{h-1}$.
1054    
1055  Sit and Morris \cite{sit02securitycons} discuss the possibility of allowing query originator  Sit and Morris \cite{sit02securitycons} discuss the possibility of allowing query originator
# Line 1061  Additionally, Lynch et al. \cite{lynch02 Line 1061  Additionally, Lynch et al. \cite{lynch02
1061  maintenance, but their solution seems to have to major problems \cite{castro02securitystructured}. First,  maintenance, but their solution seems to have to major problems \cite{castro02securitystructured}. First,
1062  the solution is very expensive even without faulty or hostile entities. Second, each group of replicas  the solution is very expensive even without faulty or hostile entities. Second, each group of replicas
1063  in their solution must have less than 1/3 of its peer faulty. Thus, this feature results in a low  in their solution must have less than 1/3 of its peer faulty. Thus, this feature results in a low
1064  probability of succesful routing.  probability of successful routing.
1065    
1066  Aspnes et al. in \cite{aspnes02faultrouting} and Kaashoek et al. in \cite{kaashoek03koorde} formally  Aspnes et al. in \cite{aspnes02faultrouting} and Kaashoek et al. in \cite{kaashoek03koorde} formally
1067  prove the lower and upper bounds for space requirements of locating a specific data item in  prove the lower and upper bounds for space requirements of locating a specific data item in
# Line 1081  data lookup routing possible in Peer-to- Line 1081  data lookup routing possible in Peer-to-
1081    
1082  \subsection{Other security threats}  \subsection{Other security threats}
1083    
1084  Ross Lee graham lists several external threats againts Peer-to-Peer networks \cite{grahamp2psecurity}. Most important,  Ross Lee graham lists several external threats against Peer-to-Peer networks \cite{grahamp2psecurity}. Most important,
1085  the list includes viruses and trojans. Currently, there are not even partial solutions  the list includes viruses and Trojan. Currently, there are not even partial solutions
1086  to the problems mentioned above. General robustness properties of Peer-to-Peer system is able to  to the problems mentioned above. General robustness properties of Peer-to-Peer system is able to
1087  deal with software failures and hostile attack, but fault tolerance againts external threats is unknown.  deal with software failures and hostile attack, but fault tolerance against external threats is unknown.
1088  The reason for this is that there are no experiences on these kinds of attacks. Possible solution  The reason for this is that there are no experiences on these kinds of attacks. Possible solution
1089  would be distributed anti-virus software, but much more intensive research is required until  would be distributed anti-virus software, but much more intensive research is required until
1090  this kind of solution would be applicable.  this kind of solution would be applicable.
# Line 1094  this kind of solution would be applicabl Line 1094  this kind of solution would be applicabl
1094    
1095  \section{Performance and usability problems in Peer-to-Peer}  \section{Performance and usability problems in Peer-to-Peer}
1096    
1097  In this section, we discuss performance related issues regardin Peer-to-Peer systems.  In this section, we discuss performance related issues regarding Peer-to-Peer systems.
1098    
1099  \subsection{Efficient data lookup}  \subsection{Efficient data lookup}
1100    
# Line 1104  especially with loosely structured appro Line 1104  especially with loosely structured appro
1104  In iterative deepening  In iterative deepening
1105  \cite{yang02improvingsearch}, multiple BFS searches are initiated  \cite{yang02improvingsearch}, multiple BFS searches are initiated
1106  with successively larger TTL depth limits, until either the query is satisfied,  with successively larger TTL depth limits, until either the query is satisfied,
1107  or the maximumum depth $D$ has been reached. To perform a data lookup, query  or the maximum depth $D$ has been reached. To perform a data lookup, query
1108  originator starts a data lookup with small TTL value. If the search is not succesful,  originator starts a data lookup with small TTL value. If the search is not successful,
1109  the query originator increases the TTL value and performs another data lookup. This  the query originator increases the TTL value and performs another data lookup. This
1110  process is repeated until the desired data is found or maximumum depth $D$  process is repeated until the desired data is found or maximum depth $D$
1111  has been reached. Expanding ring, proposed by Shenker et al., \cite{lv02searchreplication},  has been reached. Expanding ring, proposed by Shenker et al., \cite{lv02searchreplication},
1112  is similar to iterative deepening techique. With these techniques, search  is similar to iterative deepening technique. With these techniques, search
1113  may not be fast when desired data item requires many consecutive flooding rounds.  may not be fast when desired data item requires many consecutive flooding rounds.
1114    
1115  Directed BFS \cite{yang02improvingsearch} optimizes the original  Directed BFS \cite{yang02improvingsearch} optimizes the original
# Line 1123  In this scheme, each peer maintains an i Line 1123  In this scheme, each peer maintains an i
1123  $h$ hops of itself, where $h$ is a system-wide variable, called radius of the  $h$ hops of itself, where $h$ is a system-wide variable, called radius of the
1124  index\footnote{In normal BFS case, the value of $h$ is 0, as peer only has index  index\footnote{In normal BFS case, the value of $h$ is 0, as peer only has index
1125  over its local content.}. Mutual index caching architecture, as proposed in  over its local content.}. Mutual index caching architecture, as proposed in
1126  \cite{osokine02distnetworks}, is one variation of local indices techique.  \cite{osokine02distnetworks}, is one variation of local indices technique.
1127    
1128  In random walk approach \cite{lv02searchreplication}, peer forwards query to  In random walk approach \cite{lv02searchreplication}, peer forwards query to
1129  randomly selected neighbor. The basic random walk approach decreases the  randomly selected neighbor. The basic random walk approach decreases the
# Line 1133  random walk approach can be done more ef Line 1133  random walk approach can be done more ef
1133  multiple ''walkers''. Freenet \cite{clarke00freenet} Peer-to-Peer system uses  multiple ''walkers''. Freenet \cite{clarke00freenet} Peer-to-Peer system uses
1134  random walk searches in query lookups. Indeed, Freenet's query resembles  random walk searches in query lookups. Indeed, Freenet's query resembles
1135  Depth-First-Search (DFS) and peers' routing tables are dynamically built  Depth-First-Search (DFS) and peers' routing tables are dynamically built
1136  using caching. This is an outcome of Freenet's main design priciples,  using caching. This is an outcome of Freenet's main design principles,
1137  i.e., anonymity. Another property of Freenet's data lookup model is that  i.e., anonymity. Another property of Freenet's data lookup model is that
1138  it adapts well with varying usage patterns. Improvements to Freenet's data lookup using  it adapts well with varying usage patterns. Improvements to Freenet's data lookup using
1139  ''small-world phenomenon'' has been proposed by Zhang et al.. \cite{zhang02using}.  ''small-world phenomenon'' has been proposed by Zhang et al.. \cite{zhang02using}.
# Line 1141  it adapts well with varying usage patter Line 1141  it adapts well with varying usage patter
1141    
1142  Since tightly structured systems have efficient data lookup at the application level overlay,  Since tightly structured systems have efficient data lookup at the application level overlay,
1143  current research efforts are focused on proximity based data lookup. In proximity based data lookup,  current research efforts are focused on proximity based data lookup. In proximity based data lookup,
1144  peers try to choose routing-tables entries refering to other peers that are \emph{nearby} in the  peers try to choose routing-tables entries referring to other peers that are \emph{nearby} in the
1145  underlying network. In this way, tightly structured systems are able to decrease actual  underlying network. In this way, tightly structured systems are able to decrease actual
1146  lookup \emph{latency}. CAN \cite{ratnasamy01can}, Kademlia \cite{maymounkov02kademlia},  lookup \emph{latency}. CAN \cite{ratnasamy01can}, Kademlia \cite{maymounkov02kademlia},
1147  Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry} have advanced heuristics for  Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry} have advanced heuristics for
# Line 1164  is the ability to perform keyword or fuz Line 1164  is the ability to perform keyword or fuz
1164  structured systems are able carry out this requirement. Unfortunately, as discussed in this text,  structured systems are able carry out this requirement. Unfortunately, as discussed in this text,
1165  the data lookup model of loosely structured approach is not scalable. Thus, research efforts have  the data lookup model of loosely structured approach is not scalable. Thus, research efforts have
1166  been focused on tightly structured approach.  been focused on tightly structured approach.
1167  The main problem with tightly structured approach is the fact that tightly structured algorihms  The main problem with tightly structured approach is the fact that tightly structured algorithms
1168  performs data lookups based on a globally unique identifier (key). Quite recent study has been focused  performs data lookups based on a globally unique identifier (key). Quite recent study has been focused
1169  on the feasibility of Peer-to-Peer Web-like indexing and searching \cite{li03feasibility} on top of  on the feasibility of Peer-to-Peer Web-like indexing and searching \cite{li03feasibility} on top of
1170  tightly structured overlays. Authors argue, that it is possible to implement Peer-to-Peer Web-like search with certain radical compromises.  tightly structured overlays. Authors argue, that it is possible to implement Peer-to-Peer Web-like search with certain radical compromises.
# Line 1183  several studies show, the popularity of Line 1183  several studies show, the popularity of
1183  distributions\footnote{Zipf distribution is a variant of power-law function.  distributions\footnote{Zipf distribution is a variant of power-law function.
1184  Zipf-distribution can be used in observation of frequency of occurrence event $E$, as a function of the rank  Zipf-distribution can be used in observation of frequency of occurrence event $E$, as a function of the rank
1185  $i$ when the rank is determined by the frequency of occurrence, $E_i \sim \frac{1}{i^{a}}$, where the exponent  $i$ when the rank is determined by the frequency of occurrence, $E_i \sim \frac{1}{i^{a}}$, where the exponent
1186  $a$ is close to unity.} (e.g., \cite{breslau98implications}). Therefore, caching and precomputation  $a$ is close to unity.} (e.g., \cite{breslau98implications}). Therefore, caching and pre-computation
1187  can be done for optimizing search indices \cite{li03feasibility}. Regular compression algorithms,  can be done for optimizing search indices \cite{li03feasibility}. Regular compression algorithms,
1188  Bloom filters \cite{362692}, vector space models \cite{CuencaAcuna2002DSIWorkshop} and view  Bloom filters \cite{362692}, vector space models \cite{CuencaAcuna2002DSIWorkshop} and view
1189  trees \cite{Bhattacharjee03resultcache} can be used for even better optimizations. Authors  trees \cite{Bhattacharjee03resultcache} can be used for even better optimizations. Authors
# Line 1218  $N$ new additional nodes arrive into the Line 1218  $N$ new additional nodes arrive into the
1218  requires for half of the living nodes at time $t$ to leave the system. The half-life from  requires for half of the living nodes at time $t$ to leave the system. The half-life from
1219  time $t$ is smaller of the properties stated above. The half-life of the entire system is the  time $t$ is smaller of the properties stated above. The half-life of the entire system is the
1220  minimum half-life over all times $t$. Concept of half-time can be used as basic for developing  minimum half-life over all times $t$. Concept of half-time can be used as basic for developing
1221  more efficient analytical tools for modelling complex Peer-to-Peer system.  more efficient analytical tools for modeling complex Peer-to-Peer system.
1222    
1223  Some research has been done with regard to load balancing properties of tightly structured  Some research has been done with regard to load balancing properties of tightly structured
1224  overlays. Byers et al. suggest "power of two choices" whereby data item is stored at the less loaded  overlays. Byers et al. suggest "power of two choices" whereby data item is stored at the less loaded
# Line 1226  of two (or more) random peer alternative Line 1226  of two (or more) random peer alternative
1226  to control load balance in Peer-to-Peer systems \cite{rao03loadbalancing}. Their work rests on  to control load balance in Peer-to-Peer systems \cite{rao03loadbalancing}. Their work rests on
1227  idea which was originally introduced by Chord \cite{stoica01chord} system.  idea which was originally introduced by Chord \cite{stoica01chord} system.
1228    
1229  Also, query and routing hotspots may be an issue in tightly structured overlays \cite{ratnasamy02routing}.  Also, query and routing hot spots may be an issue in tightly structured overlays \cite{ratnasamy02routing}.
1230  Hotspots happen, when specific key is being requested extremely often in tightly structured overlays. Recent study  Hot spots happen, when specific key is being requested extremely often in tightly structured overlays. Recent study
1231  by Freedman et al. tries to reduce hot spots in the system by performing \emph{sloppy} hashing  by Freedman et al. tries to reduce hot spots in the system by performing \emph{sloppy} hashing
1232  \cite{sloppy:iptps03}. Another key feature of their work is that peers self-organize into clusters,  \cite{sloppy:iptps03}. Another key feature of their work is that peers self-organize into clusters,
1233  therefore enabling peers to find nearby data without looking up data from distant peers.  therefore enabling peers to find nearby data without looking up data from distant peers.
# Line 1238  face of computing power or network bandw Line 1238  face of computing power or network bandw
1238  a serious problem of tightly structured overlays in face of performance and load balancing. Measurement study  a serious problem of tightly structured overlays in face of performance and load balancing. Measurement study
1239  by Saroiu et al. shows that there is extreme heterogeneity among participating peers in already deployed Peer-to-Peer  by Saroiu et al. shows that there is extreme heterogeneity among participating peers in already deployed Peer-to-Peer
1240  systems \cite{saroiu02measurementstudyp2p}. Symphony seems to be the first tightly structured overlay system  systems \cite{saroiu02measurementstudyp2p}. Symphony seems to be the first tightly structured overlay system
1241  which support hetergeneity. Zhao et al. have proposed a secondary layer a top of structured overlay  which support heterogeneity. Zhao et al. have proposed a secondary layer a top of structured overlay
1242  to support hetergeneity better \cite{zhao02brocade}.  to support heterogeneity better \cite{zhao02brocade}.
1243    
1244  Research has been done on self-organization. Ledlie et al. propose techniques for forming and maintaining  Research has been done on self-organization. Ledlie et al. propose techniques for forming and maintaining
1245  groups in highly dynamic environment \cite{ledlie02selfp2p}. Unfortunately their work relies on idea that  groups in highly dynamic environment \cite{ledlie02selfp2p}. Unfortunately their work relies on idea that
1246  participating peers would create multiple hierarchical groups; it's not clear whether this approach  participating peers would create multiple hierarchical groups; it's not clear whether this approach
1247  is fault-tolerant and suitable to Peer-to-Peer environment. More promising work has been done by Rowston et al.  is fault-tolerant and suitable to Peer-to-Peer environment. More promising work has been done by Rowston et al.
1248  in \cite{rowston03controlloingreliability}. Authors propose techiques for self-tuning, dealing with  in \cite{rowston03controlloingreliability}. Authors propose techniques for self-tuning, dealing with
1249  uncommon conditions (e.g., network partition and high failure rates). Moreover, authors arque that  uncommon conditions (e.g., network partition and high failure rates). Moreover, authors argue that
1250  with these techniques, the concerns over the tightly structured overlay maintenance costs are no more  with these techniques, the concerns over the tightly structured overlay maintenance costs are no more
1251  an open issue.  an open issue.
1252    
1253  Finally, little research has been done regarding self-monitoring and data availability. Zhang et al.  Finally, little research has been done regarding self-monitoring and data availability. Zhang et al.
1254  describe a arbitrary data structure on top of tightly structured overlay \cite{zhang03somo}. They  describe a arbitrary data structure on top of tightly structured overlay \cite{zhang03somo}. They
1255  call their proposal as \emph{data overlay}, since it supports several fundamental data structures.  call their proposal as \emph{data overlay}, since it supports several fundamental data structures.
1256  Authors use this data overlay to build Self-Organized Metadata Overlay (SOMO), which can be used  Authors use this data overlay to build Self-Organized Meta data Overlay (SOMO), which can be used
1257  for monitoring health of tightly structured overlay. Fault tolerance of SOMO itself is currently  for monitoring health of tightly structured overlay. Fault tolerance of SOMO itself is currently
1258  unknown.  unknown.
1259    
# Line 1272  different algorithms. Recently, there ha Line 1272  different algorithms. Recently, there ha
1272  guidelines. This list includes \cite{zhao03api}, \cite{frise02p2pframework}, \cite{babaoglu02anthill}.  guidelines. This list includes \cite{zhao03api}, \cite{frise02p2pframework}, \cite{babaoglu02anthill}.
1273  Early experiments with Peer-to-Peer benchmarking include \cite{ratnasamy02routing} and \cite{rhea03benchmarks}.  Early experiments with Peer-to-Peer benchmarking include \cite{ratnasamy02routing} and \cite{rhea03benchmarks}.
1274    
1275  \subsection{Social behaviour}  \subsection{Social behavior}
1276    
1277  Frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate. Another belief  Frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate. Another belief
1278  is that all peers would behave equally, i.e., all peers both consume resources and contributes resources.  is that all peers would behave equally, i.e., all peers both consume resources and contributes resources.
# Line 1281  and peers are unwilling to cooperate \ci Line 1281  and peers are unwilling to cooperate \ci
1281  \cite{hearn02mojonation}.  \cite{hearn02mojonation}.
1282    
1283  Somewhat surprisingly little research has been in this area, especially when considering  Somewhat surprisingly little research has been in this area, especially when considering
1284  the possible impact of this \emph{unwanted socical behaviour} to performance of Peer-to-Peer  the possible impact of this \emph{unwanted social behavior} to performance of Peer-to-Peer
1285  system. Problem is addressed by Golle et al. \cite{golle01incentivesp2p}. Some  system. Problem is addressed by Golle et al. \cite{golle01incentivesp2p}. Some
1286  research has been focused on semantic properties of the overlay in order to increase  research has been focused on semantic properties of the overlay in order to increase
1287  cooperation among participating peers \cite{crespo02semanticoverlay}. Ramanathan et al.  cooperation among participating peers \cite{crespo02semanticoverlay}. Ramanathan et al.
# Line 1295  Peer-to-Peer system, which uses empirica Line 1295  Peer-to-Peer system, which uses empirica
1295    
1296  Very little research has been done on simulating the \emph{global} Peer-to-Peer system. Presumably, this  Very little research has been done on simulating the \emph{global} Peer-to-Peer system. Presumably, this
1297  is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very  is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very
1298  diffucult. Floyd et al. has been studying the simulation of the Internet in \cite{504642}. Authors  difficult. Floyd et al. has been studying the simulation of the Internet in \cite{504642}. Authors
1299  state that simulating the Internet is very challenging task, because of Internet's heterogeneity  state that simulating the Internet is very challenging task, because of Internet's heterogeneity
1300  and rapid change. Obviously, these factors exist also in Peer-to-Peer system even with higher  and rapid change. Obviously, these factors exist also in Peer-to-Peer system even with higher
1301  rates.  rates.
# Line 1350  we list miscellaneous open problems. Line 1350  we list miscellaneous open problems.
1350    
1351    
1352  \parbox{90pt}{DoS attack \cite{sit02securitycons}, \cite{saia02dynamicfaultcontentnetwork}, \cite{datar02butterflies}, \cite{daswani02queryflooddos}, \cite{juels99clientpuzzles}} &  \parbox{90pt}{DoS attack \cite{sit02securitycons}, \cite{saia02dynamicfaultcontentnetwork}, \cite{datar02butterflies}, \cite{daswani02queryflooddos}, \cite{juels99clientpuzzles}} &
1353  \parbox{110pt}{Distributed, controlled burden againts specific computer(s)} &  \parbox{110pt}{Distributed, controlled burden against specific computer(s)} &
1354  \parbox{110pt}{Client puzzles, load balancing, traffic measurements, traffic models, replication} &  \parbox{110pt}{Client puzzles, load balancing, traffic measurements, traffic models, replication} &
1355  \parbox{110pt}{Only partial solutions, traffic models most effective}  \parbox{110pt}{Only partial solutions, traffic models most effective}
1356  \\ \hline  \\ \hline
# Line 1400  we list miscellaneous open problems. Line 1400  we list miscellaneous open problems.
1400    
1401  \parbox{90pt}{Malicious nodes \cite{sit02securitycons}, \cite{castro02securerouting}} &  \parbox{90pt}{Malicious nodes \cite{sit02securitycons}, \cite{castro02securerouting}} &
1402  \parbox{110pt}{How to identify malicious nodes in the system} &  \parbox{110pt}{How to identify malicious nodes in the system} &
1403  \parbox{110pt}{Create invariants for node behaviour, verify invariants, self-certifying data} &  \parbox{110pt}{Create invariants for node behavior, verify invariants, self-certifying data} &
1404  \parbox{110pt}{Partial solutions, self-certifying data most realiable}  \parbox{110pt}{Partial solutions, self-certifying data most reliable}
1405  \\ \hline  \\ \hline
1406    
1407    
# Line 1412  we list miscellaneous open problems. Line 1412  we list miscellaneous open problems.
1412  \\ \hline  \\ \hline
1413    
1414    
1415  \parbox{90pt}{Inconsistent behaviour \cite{sit02securitycons}} &  \parbox{90pt}{Inconsistent behavior \cite{sit02securitycons}} &
1416  \parbox{110pt}{Hostile node could act correctly with its neighbors, but incorrectly with others} &  \parbox{110pt}{Hostile node could act correctly with its neighbors, but incorrectly with others} &
1417  \parbox{110pt}{Public keys, digital signatures} &  \parbox{110pt}{Public keys, digital signatures} &
1418  \parbox{110pt}{Not practical approach/working proposal created yet}  \parbox{110pt}{Not practical approach/working proposal created yet}
# Line 1422  we list miscellaneous open problems. Line 1422  we list miscellaneous open problems.
1422  \parbox{90pt}{Hostile groups \cite{castro02securerouting}} &  \parbox{90pt}{Hostile groups \cite{castro02securerouting}} &
1423  \parbox{110pt}{Joining node may join parallel network, formed a group of hostile nodes, hostile node(s) controls the construction of the network} &  \parbox{110pt}{Joining node may join parallel network, formed a group of hostile nodes, hostile node(s) controls the construction of the network} &
1424  \parbox{110pt}{Use trusted nodes, based on history information, Cryptography, key infrastructure} &  \parbox{110pt}{Use trusted nodes, based on history information, Cryptography, key infrastructure} &
1425  \parbox{110pt}{Not 100\% sure if Centreal Authority (CA) is missing, not practical approach/working proposal created yet}  \parbox{110pt}{Not 100\% sure if Central Authority (CA) is missing, not practical approach/working proposal created yet}
1426  \\ \hline  \\ \hline
1427    
1428    
1429  \parbox{90pt}{External security threats} &  \parbox{90pt}{External security threats} &
1430  \parbox{110pt}{Viruses, trojans, sniffers} &  \parbox{110pt}{Viruses, Trojan, sniffers} &
1431  \parbox{110pt}{Data integrity/authenticity, distributed antivirus software} &  \parbox{110pt}{Data integrity/authenticity, distributed anti virus software} &
1432  \parbox{110pt}{Not much research has been done on this}  \parbox{110pt}{Not much research has been done on this}
1433  \\ \hline  \\ \hline
1434    
# Line 1472  we list miscellaneous open problems. Line 1472  we list miscellaneous open problems.
1472    
1473  \parbox{90pt}{Efficient and scalable data discovery \cite{lv02searchreplication}, \cite{osokine02distnetworks}, \cite{yang02improvingsearch}, \cite{lv02gnutellascalable}, \cite{ganesan02yappers}, \cite{adamic02localsearch}, \cite{adamic01powerlawsearch}, \cite{ripeanu02mappinggnutella}, \cite{milgram67smallworld}, \cite{adamic99small}, \cite{sterling95beowulf}, \cite{ramanathan02goodpeers}, \cite{kleinberg99small}, \cite{nips02-Kleinberg}, \cite{zhang02using}, \cite{watts00dynamics}} &  \parbox{90pt}{Efficient and scalable data discovery \cite{lv02searchreplication}, \cite{osokine02distnetworks}, \cite{yang02improvingsearch}, \cite{lv02gnutellascalable}, \cite{ganesan02yappers}, \cite{adamic02localsearch}, \cite{adamic01powerlawsearch}, \cite{ripeanu02mappinggnutella}, \cite{milgram67smallworld}, \cite{adamic99small}, \cite{sterling95beowulf}, \cite{ramanathan02goodpeers}, \cite{kleinberg99small}, \cite{nips02-Kleinberg}, \cite{zhang02using}, \cite{watts00dynamics}} &
1474  \parbox{110pt}{Find resources efficiently, if resource exists (loosely structured)} &  \parbox{110pt}{Find resources efficiently, if resource exists (loosely structured)} &
1475  \parbox{110pt}{Super nodes, node clusters, caching techiques} &  \parbox{110pt}{Super nodes, node clusters, caching techniques} &
1476  \parbox{110pt}{More efficient, less network traffic, not comparable to DHT's efficiency}  \parbox{110pt}{More efficient, less network traffic, not comparable to DHT's efficiency}
1477  \\ \hline  \\ \hline
1478    
# Line 1498  we list miscellaneous open problems. Line 1498  we list miscellaneous open problems.
1498  \\ \hline  \\ \hline
1499    
1500    
1501  \parbox{90pt}{Data availability/persistency \cite{bhagwan03availability}} &  \parbox{90pt}{Data availability/persistence \cite{bhagwan03availability}} &
1502  \parbox{110pt}{Data might be temporarily unavailable, or lost permanently} &  \parbox{110pt}{Data might be temporarily unavailable, or lost permanently} &
1503  \parbox{110pt}{Data caching, data replication} &  \parbox{110pt}{Data caching, data replication} &
1504  \parbox{110pt}{Working solutions, but creates more traffic and overhead per node}  \parbox{110pt}{Working solutions, but creates more traffic and overhead per node}
# Line 1507  we list miscellaneous open problems. Line 1507  we list miscellaneous open problems.
1507    
1508  \parbox{90pt}{Network proximity \cite{pias03lighthouse}, \cite{ng02predicting}, \cite{ratnasamy02ght}, \cite{eriksson03peernet}, \cite{castro02networkproximity}} &  \parbox{90pt}{Network proximity \cite{pias03lighthouse}, \cite{ng02predicting}, \cite{ratnasamy02ght}, \cite{eriksson03peernet}, \cite{castro02networkproximity}} &
1509  \parbox{110pt}{Can we take account the underlying network's properties better when forming overlay network (network-awareness for performance) ?} &  \parbox{110pt}{Can we take account the underlying network's properties better when forming overlay network (network-awareness for performance) ?} &
1510  \parbox{110pt}{Global Network Positioning, Lighthouse technique, trianqulated heuristics} &  \parbox{110pt}{Global Network Positioning, Lighthouse technique, triangulated heuristics} &
1511  \parbox{110pt}{Increases system complexity, no real world experience in a wide scale, proposed solutions are susceptible to single point of failure}  \parbox{110pt}{Increases system complexity, no real world experience in a wide scale, proposed solutions are susceptible to single point of failure}
1512  \\ \hline  \\ \hline
1513    
# Line 1519  we list miscellaneous open problems. Line 1519  we list miscellaneous open problems.
1519  \\ \hline  \\ \hline
1520    
1521    
1522  \parbox{90pt}{Hotspots \cite{258660}, \cite{sloppy:iptps03}, \cite{maymounkov03ratelesscodes}} &  \parbox{90pt}{Hot spots \cite{258660}, \cite{sloppy:iptps03}, \cite{maymounkov03ratelesscodes}} &
1523  \parbox{110pt}{What will happen if some resource is extremely popular and only one node is hosting it ?} &  \parbox{110pt}{What will happen if some resource is extremely popular and only one node is hosting it ?} &
1524  \parbox{110pt}{Caching, multisource downloads, replication, load balancing, sloppy hashing} &  \parbox{110pt}{Caching, multi source downloads, replication, load balancing, sloppy hashing} &
1525  \parbox{110pt}{For query hotspots, caching and multisource downloads efficiently reduces hotspots, for routing hotspots, benefits are smaller}  \parbox{110pt}{For query hot spots, caching and multi source downloads efficiently reduces hot spots, for routing hot spots, benefits are smaller}
1526  \\ \hline  \\ \hline
1527    
1528    
1529  \parbox{90pt}{Load balancing \cite{rao03loadbalancing}, \cite{ledlie02selfp2p}, \cite{byers03dhtbalancing}} &  \parbox{90pt}{Load balancing \cite{rao03loadbalancing}, \cite{ledlie02selfp2p}, \cite{byers03dhtbalancing}} &
1530  \parbox{110pt}{Random (but uniformly distributed) identifier selection could cause systen imbalance among participants with different capabilities} &  \parbox{110pt}{Random (but uniformly distributed) identifier selection could cause system imbalance among participants with different capabilities} &
1531  \parbox{110pt}{Caching, virtual server transfers} &  \parbox{110pt}{Caching, virtual server transfers} &
1532  \parbox{110pt}{Effective, more research required in fully dynamic environment}  \parbox{110pt}{Effective, more research required in fully dynamic environment}
1533  \\ \hline  \\ \hline
# Line 1535  we list miscellaneous open problems. Line 1535  we list miscellaneous open problems.
1535  \parbox{90pt}{System in flux \cite{libennowell01observations}, \cite{571863}, \cite{ledlie02selfp2p}, \cite{albert-02-statistical}} &  \parbox{90pt}{System in flux \cite{libennowell01observations}, \cite{571863}, \cite{ledlie02selfp2p}, \cite{albert-02-statistical}} &
1536  \parbox{110pt}{Nodes join and leave system constantly. What about load balancing and performance ?} &  \parbox{110pt}{Nodes join and leave system constantly. What about load balancing and performance ?} &
1537  \parbox{110pt}{Half-life phenomenon (for analysis), simple overlay maintenance and construction algorithm} &  \parbox{110pt}{Half-life phenomenon (for analysis), simple overlay maintenance and construction algorithm} &
1538  \parbox{110pt}{Initial theoretical analysis have been created, but not comprehensive model for analysing different system states and its variations (e.g. complex usage patterns)}  \parbox{110pt}{Initial theoretical analysis have been created, but not comprehensive model for analyzing different system states and its variations (e.g. complex usage patterns)}
1539  \\ \hline  \\ \hline
1540    
1541  \parbox{90pt}{Sudden network partition \cite{harvey03skipnet1}, \cite{harvey03skipnet2}, \cite{rowston03controlloingreliability}} &  \parbox{90pt}{Sudden network partition \cite{harvey03skipnet1}, \cite{harvey03skipnet2}, \cite{rowston03controlloingreliability}} &
1542  \parbox{110pt}{Sub network is isolated from other network because of network disconnection} &  \parbox{110pt}{Sub network is isolated from other network because of network disconnection} &
1543  \parbox{110pt}{Self-tuning, environment observatorion, localized network connection for minimun latency (backup connections)} &  \parbox{110pt}{Self-tuning, environment observatorion, localized network connection for minimum latency (backup connections)} &
1544  \parbox{110pt}{Creates more overhead/space requirements per node}  \parbox{110pt}{Creates more overhead/space requirements per node}
1545  \\ \hline  \\ \hline
1546    
# Line 1554  we list miscellaneous open problems. Line 1554  we list miscellaneous open problems.
1554  \parbox{90pt}{Byzantine faults \cite{296824}} &  \parbox{90pt}{Byzantine faults \cite{296824}} &
1555  \parbox{110pt}{Faulty nodes may behave arbitrarily} &  \parbox{110pt}{Faulty nodes may behave arbitrarily} &
1556  \parbox{110pt}{Byzantine replication algorithms -> get information from multiple entities, trust majority's opinion} &  \parbox{110pt}{Byzantine replication algorithms -> get information from multiple entities, trust majority's opinion} &
1557  \parbox{110pt}{Much research has been done on this field, practical solutions, decreases the performance of system slighly}  \parbox{110pt}{Much research has been done on this field, practical solutions, decreases the performance of system slightly}
1558  \\ \hline  \\ \hline
1559    
1560  \caption{Performance and usability problems in Peer-to-Peer.}  \caption{Performance and usability problems in Peer-to-Peer.}
# Line 1619  we list miscellaneous open problems. Line 1619  we list miscellaneous open problems.
1619    
1620  \parbox{90pt}{Comprehensive simulations/analysis of Peer-to-Peer network} &  \parbox{90pt}{Comprehensive simulations/analysis of Peer-to-Peer network} &
1621  \parbox{110pt}{Ability to simulate whole Peer-to-Peer network's usage patterns, network traffics, flux state etc} &  \parbox{110pt}{Ability to simulate whole Peer-to-Peer network's usage patterns, network traffics, flux state etc} &
1622  \parbox{110pt}{Use same techniques as simulating/analysing the Internet} &  \parbox{110pt}{Use same techniques as simulating/analyzing the Internet} &
1623  \parbox{110pt}{Only small subset of Peer-to-Peer networks has been able to analyse, because of ad hoc properties of network, more poweful solutions needed}  \parbox{110pt}{Only small subset of Peer-to-Peer networks has been able to 22, because of ad hoc properties of network, more powerful solutions needed}
1624  \\ \hline  \\ \hline
1625    
1626    
1627  \parbox{90pt}{Overlay management and health monitoring \cite{zhang03somo}} &  \parbox{90pt}{Overlay management and health monitoring \cite{zhang03somo}} &
1628  \parbox{110pt}{System is self-capable to monitor it's status and health for better performance} &  \parbox{110pt}{System is self-capable to monitor it's status and health for better performance} &
1629  \parbox{110pt}{Build a metadata overlay atop of structured overlay (such as SOMO for structured overlays), make local decisions about overlay (unstrucured)} &  \parbox{110pt}{Build a meta data overlay atop of structured overlay (such as SOMO for structured overlays), make local decisions about overlay (unstructured)} &
1630  \parbox{110pt}{For structured overlays, efficient and simple to implement, fault-tolerance unknowns, for unstructured, not necessarily efficient because decisions are based on local knowledge}  \parbox{110pt}{For structured overlays, efficient and simple to implement, fault-tolerance unknowns, for unstructured, not necessarily efficient because decisions are based on local knowledge}
1631  \\ \hline  \\ \hline
1632    
# Line 1653  functionality. Line 1653  functionality.
1653  \section{Overview}  \section{Overview}
1654    
1655  Fenfire project \cite{fenfireurl} is an effort to build a distributed, hyperstructured user  Fenfire project \cite{fenfireurl} is an effort to build a distributed, hyperstructured user
1656  interface system. Fenfire is free software and it is licenced under GNU L-GPL. Fenfire's main goal  interface system. Fenfire is free software and it is licensed under GNU L-GPL. Fenfire's main goal
1657  is to implement xanalogical storage model \cite{ted-xu-model}. Fenfire was formely also a implementation  is to implement xanalogical storage model \cite{ted-xu-model}. Fenfire was formerly also a implementation
1658  of the ZigZag\texttrademark --structure, which was originally invented  of the ZigZag\texttrademark --structure, which was originally invented
1659  by Ted Nelson. Now, however, Fenfire uses Resource Description Framework (RDF) \cite{w3rdfurl}  by Ted Nelson. Now, however, Fenfire uses Resource Description Framework (RDF) \cite{w3rdfurl}
1660  for representing internal data structures and their relationships.  for representing internal data structures and their relationships.
# Line 1684  permanent, globally unique identifier. F Line 1684  permanent, globally unique identifier. F
1684  scenario: ''the character 'D' typed by Janne Kujala on 10/8/97 8:37:18''. In this  scenario: ''the character 'D' typed by Janne Kujala on 10/8/97 8:37:18''. In this
1685  example, when character 'D' is is first typed in, xanalogical storage model  example, when character 'D' is is first typed in, xanalogical storage model
1686  acquires a permanent identifier for that character and retains it when character  acquires a permanent identifier for that character and retains it when character
1687  is copied to different document. Thus, the identifier distinguishes chararacter from  is copied to different document. Thus, the identifier distinguishes character from
1688  all similar characters typed in independently\footnote{Xanalogical storage model  all similar characters typed in independently\footnote{Xanalogical storage model
1689  is not limited to text. It can support arbitrary data, e.g., pixels of picture or  is not limited to text. It can support arbitrary data, e.g., pixels of picture or
1690  frames of video.}. The connectivity in xanalogical storage model between data content  frames of video.}. The connectivity in xanalogical storage model between data content
# Line 1725  data storage operations. Storm stores al Line 1725  data storage operations. Storm stores al
1725  are immutable byte sequences. SHA-1\footnote{SHA-1 is considered a collision free  are immutable byte sequences. SHA-1\footnote{SHA-1 is considered a collision free
1726  hash function. Therefore, it is very unlikely that two different Storm data blocks  hash function. Therefore, it is very unlikely that two different Storm data blocks
1727  would have same identifier.} cryptographic content hash \cite{fips-sha-1} is used  would have same identifier.} cryptographic content hash \cite{fips-sha-1} is used
1728  for creating locatiotion-independent, globally unique identifiers for blocks. Additionally,  for creating location-independent, globally unique identifiers for blocks. Additionally,
1729  SHA-1 \cite{fips-sha-1} is used for verifying the integrity of Storm data blocks. Storm  SHA-1 \cite{fips-sha-1} is used for verifying the integrity of Storm data blocks. Storm
1730  blocks have much in common with regular files, except Storm blocks are \emph{immutable} as  blocks have much in common with regular files, except Storm blocks are \emph{immutable} as
1731  any change to the byte sequence would the change block's hash value, i.e., unique  any change to the byte sequence would the change block's hash value, i.e., unique
# Line 1755  pointer creation process. Pointer block Line 1755  pointer creation process. Pointer block
1755  pointer blocks, i.e., when a new version of scroll block is created, it supersedes  pointer blocks, i.e., when a new version of scroll block is created, it supersedes
1756  one older version which has been created in the past. The most current pointer  one older version which has been created in the past. The most current pointer
1757  block will 'obsolete' the pointer block targeting the superseded version. Next  block will 'obsolete' the pointer block targeting the superseded version. Next
1758  time, when the pointer is used for refering to a specific scroll block, only  time, when the pointer is used for referring to a specific scroll block, only
1759  the most recent pointer's block target is loaded.  the most recent pointer's block target is loaded.
1760    
1761  \begin{figure}  \begin{figure}
# Line 1796  file'', from Peer-to-Peer overlay as eff Line 1796  file'', from Peer-to-Peer overlay as eff
1796  \emph{direct} scroll block obtaining using globally unique identifier of Storm scroll block,  \emph{direct} scroll block obtaining using globally unique identifier of Storm scroll block,
1797  we also must support \emph{indirect} obtaining of Storm scroll block using pointer blocks.  we also must support \emph{indirect} obtaining of Storm scroll block using pointer blocks.
1798    
1799  Obviously, our objectives are yet simple but hard to fulfil. First, as a prerequisite  Obviously, our objectives are yet simple but hard to fulfill. First, as a prerequisite
1800  to implementing xanalogical storage model in Peer-to-Peer environment, system  to implementing xanalogical storage model in Peer-to-Peer environment, system
1801  supporting data lookups must be able to perform \emph{global} scale lookups. Thus,  supporting data lookups must be able to perform \emph{global} scale lookups. Thus,
1802  we must able to locate and fetch Storm scroll/pointer block, if it exists in the  we must able to locate and fetch Storm scroll/pointer block, if it exists in the
# Line 1804  Peer-to-Peer overlay. Second, data looku Line 1804  Peer-to-Peer overlay. Second, data looku
1804  one ''virtual file'' may need obtaining several data items, which are distributed  one ''virtual file'' may need obtaining several data items, which are distributed
1805  randomly throughout the overlay; if not efficient, construction of ''virtual file''  randomly throughout the overlay; if not efficient, construction of ''virtual file''
1806  may take reasonable amount time while rendering system very unusable. Third, Peer-to-Peer  may take reasonable amount time while rendering system very unusable. Third, Peer-to-Peer
1807  infrasctructure has to be scalable and robust againts hostile attacks.  infrastructure has to be scalable and robust against hostile attacks.
1808    
1809  Some research regarding to these problem has been made by Lukka et al.  Some research regarding to these problem has been made by Lukka et al.
1810  \cite{lukka02freenetguids}. Authors' work is mainly based on insight of implementing  \cite{lukka02freenetguids}. Authors' work is mainly based on insight of implementing
# Line 1824  publishing or not. Line 1824  publishing or not.
1824  \section{Evaluation of Peer-to-Peer approaches with regard to Fenfire}  \section{Evaluation of Peer-to-Peer approaches with regard to Fenfire}
1825    
1826  In chapter 2, we discussed main differences between loosely and tightly structured  In chapter 2, we discussed main differences between loosely and tightly structured
1827  approaches. As stated, the most significant difference is that tighly structured  approaches. As stated, the most significant difference is that tightly structured
1828  approach has logarithmical properties in all interal operations, while loosely  approach has logarithmical properties in all internal operations, while loosely
1829  structured approach doesn't have always even linear properties. Furthermore, the  structured approach doesn't have always even linear properties. Furthermore, the
1830  data lookup model of tightly structured overlay scales much better than loosely  data lookup model of tightly structured overlay scales much better than loosely
1831  structured overlays; tightly structured overlay supports global data lookups  structured overlays; tightly structured overlay supports global data lookups
# Line 1843  to provide general purpose \emph{interfa Line 1843  to provide general purpose \emph{interfa
1843  Domain Name System (DNS) \cite{rfc1101} is widely used RRS system in the Internet.}  Domain Name System (DNS) \cite{rfc1101} is widely used RRS system in the Internet.}
1844   \cite{balakrishnan03semanticfree}. Authors argue that next generation RRS must be   \cite{balakrishnan03semanticfree}. Authors argue that next generation RRS must be
1845  application-independent and references itself should be \emph{unstructured} and  application-independent and references itself should be \emph{unstructured} and
1846  \emph{semantic free}. Finally, as said, with tightly stuctured systems, it is feasible to  \emph{semantic free}. Finally, as said, with tightly structured systems, it is feasible to
1847  perform \emph{global} data lookups in the overlay. To summarize, these aspects may be the most important features  perform \emph{global} data lookups in the overlay. To summarize, these aspects may be the most important features
1848  of Peer-to-Peer infrastructure with regard to Fenfire as a \emph{distributed} hypermedia system.  of Peer-to-Peer infrastructure with regard to Fenfire as a \emph{distributed} hypermedia system.
1849  Thus, we see the tightly structured approach the best alternative to locate data in Peer-to-Peer  Thus, we see the tightly structured approach the best alternative to locate data in Peer-to-Peer
1850  environment.  environment.
1851    
1852  Once located, for \emph{fetching} Fenfire related data from the overlay, we can use reqular  Once located, for \emph{fetching} Fenfire related data from the overlay, we can use regular
1853  TCP/IP-protocols, such as Hypertext Transfer protocol (HTTP) \cite{rfc2068}. However, HTTP-protocol may  TCP/IP-protocols, such as Hypertext Transfer protocol (HTTP) \cite{rfc2068}. However, HTTP-protocol may
1854  not be optimal, when obtaining large amounts of data from the Peer-to-Peer overlay, for  not be optimal, when obtaining large amounts of data from the Peer-to-Peer overlay, for
1855  instance videos, images or music. In this case, multisource downloads can be very useful  instance videos, images or music. In this case, multi source downloads can be very useful
1856  for better efficiency \cite{maymounkov03ratelesscodes}, \cite{bittorrenturl}. Furthermore,  for better efficiency \cite{maymounkov03ratelesscodes}, \cite{bittorrenturl}. Furthermore,
1857  multisource downloads can be used for decreasing load of certain peer, thus avoiding query  multi source downloads can be used for decreasing load of certain peer, thus avoiding query
1858  hotspots in the system \cite{ratnasamy02routing}. Current mplementation of Fenfire uses  hot spots in the system \cite{ratnasamy02routing}. Current implementation of Fenfire uses
1859  standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content  standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content
1860  hash for verifying the integrity of data by recomputing the content hash  hash for verifying the integrity of data by recomputing the content hash
1861  for a scroll block. In face of multisource downloads, Fenfire must support  for a scroll block. In face of multi source downloads, Fenfire must support
1862  tree-based hash\footnote{With multisource downloads, tree based hash functions can be used  tree-based hash\footnote{With multi source downloads, tree based hash functions can be used
1863  to verify fixed length segments of data. If hash value of data segment is incorrect,  to verify fixed length segments of data. If hash value of data segment is incorrect,
1864  we need only to fetch \emph{segment} of data (instead of whole data, e.g., a file) from  we need only to fetch \emph{segment} of data (instead of whole data, e.g., a file) from
1865  other source.}, such as \cite{merkle87hashtree} and \cite{mohr02thex} for reliable and efficient  other source.}, such as \cite{merkle87hashtree} and \cite{mohr02thex} for reliable and efficient
# Line 1876  solved, since there is a strong and wide Line 1876  solved, since there is a strong and wide
1876  overlays \cite{projectirisurl}.  overlays \cite{projectirisurl}.
1877    
1878                
1879  \section{Fenfire system model in Peer-to-Peer enviroment}  \section{Fenfire system model in Peer-to-Peer environment}
1880    
1881  In this section present a proposal of Fenfire Peer-to-Peer system, which consists  In this section present a proposal of Fenfire Peer-to-Peer system, which consists
1882  of several techologies presented in this thesis.  Then, we introduce yet simple but  of several technologies presented in this thesis.  Then, we introduce yet simple but
1883  effective algorithms for obtaining Fenfire data from Peer-to-Peer environment.  effective algorithms for obtaining Fenfire data from Peer-to-Peer environment.
1884    
1885  \subsection{System proposal}  \subsection{System proposal}
1886    
1887  We see Kademlia \cite{maymounkov02kademlia} as the best algorithm for  We see Kademlia \cite{maymounkov02kademlia} as the best algorithm for
1888  locating data efficiently in the Peer-to-Peer overlay. There are two main  locating data efficiently in the Peer-to-Peer overlay. There are two main
1889  reasons for this. First, Kamdelia's XOR-based distance function is superior  reasons for this. First, Kademlia's XOR-based distance function is superior
1890  over the distance functions of other systems. Second, there are already some  over the distance functions of other systems. Second, there are already some
1891  real-life systems (e.g., \cite{overneturl}, \cite{edonkey2kurl}, \cite{kashmirurl},  real-life systems (e.g., \cite{overneturl}, \cite{edonkey2kurl}, \cite{kashmirurl},
1892  \cite{kato02gisp}), which means that Kademlia's algorithm is simple and easy to implement.  \cite{kato02gisp}), which means that Kademlia's algorithm is simple and easy to implement.
1893    
1894  On top of Kademlia, we propose the usage of Sloppy hashing \cite{sloppy:iptps03} which  On top of Kademlia, we propose the usage of Sloppy hashing \cite{sloppy:iptps03} which
1895  optimized for DOLR abstraction of tightly structured overlays. With Sloppy hashing,  optimized for DOLR abstraction of tightly structured overlays. With Sloppy hashing,
1896  we are able reduce of generation of query hotspots. Sloppy hashing enables to  we are able reduce of generation of query hot spots. Sloppy hashing enables to
1897  locate nearby data without looking up data from distant nodes. Moreover, authors'  locate nearby data without looking up data from distant nodes. Moreover, authors'
1898  proposal for self-organizing clusters using network diameters may be useful,  proposal for self-organizing clusters using network diameters may be useful,
1899  especially within small groups of working people. Thus, with Sloppy hashing  especially within small groups of working people. Thus, with Sloppy hashing
# Line 1906  as sudden network partition, or highly d Line 1906  as sudden network partition, or highly d
1906    
1907  Finally, for more efficient data transfer, we can use variable techniques for this purpose.  Finally, for more efficient data transfer, we can use variable techniques for this purpose.
1908  For small amounts of data, HTTP can be used \cite{rfc2068}. For big downloads, we can use  For small amounts of data, HTTP can be used \cite{rfc2068}. For big downloads, we can use
1909  multisource downloads for better efficiency and reliability. Specifically, techology based  multi source downloads for better efficiency and reliability. Specifically, technology based
1910  on rateless erasure codes \cite{maymounkov03ratelesscodes} seems very promising.  on rate less erasure codes \cite{maymounkov03ratelesscodes} seems very promising.
1911    
1912  \subsection{Algorithms}  \subsection{Algorithms}
1913    
1914  We use DOLR abstraction of tightly of structured approach, i.e., each participating peer hosts  We use DOLR abstraction of tightly of structured approach, i.e., each participating peer hosts
1915  the data and overlay maintains only the \emph{pointers} to the data. We descided to use DOLR in our  the data and overlay maintains only the \emph{pointers} to the data. We decided to use DOLR in our
1916  model, since DOLR systems locate data without specifiying a storage policy explicity \cite{rhea03benchmarks}.  model, since DOLR systems locate data without specifying a storage policy explicitly \cite{rhea03benchmarks}.
1917  DHT based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have  DHT based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have
1918  critical problems with load balancing in highly heterogeneous environment. This problem is caused by peers  critical problems with load balancing in highly heterogeneous environment. This problem is caused by peers
1919  which may not able to store relative great amount of data with key/value pair, assigned randomly by  which may not able to store relative great amount of data with key/value pair, assigned randomly by
# Line 1924  In the following subsections we assume t Line 1924  In the following subsections we assume t
1924  ''virtual file'' before hand, i.e., when assembling a ''virtual file'', we know all Storm  ''virtual file'' before hand, i.e., when assembling a ''virtual file'', we know all Storm
1925  scroll/pointer blocks, which are required when building the ''virtual file''. Also, we don't  scroll/pointer blocks, which are required when building the ''virtual file''. Also, we don't
1926  respond to security issues related to Peer-to-Peer systems, since there is no working solution  respond to security issues related to Peer-to-Peer systems, since there is no working solution
1927  available yet; we either assume that Fenfire has a reliable techique for identifying invidual entities, or  available yet; we either assume that Fenfire has a reliable technique for identifying individual entities, or
1928  there are no hostile entities among participating peers.  there are no hostile entities among participating peers.
1929    
1930  In our model, each peer maintains following data structures for local operations: data structure for listing all  In our model, each peer maintains following data structures for local operations: data structure for listing all
# Line 1975  block is known. Line 1975  block is known.
1975  Figure \ref{fig:storm_query_urn5} illustrates how Storm scroll block is located  Figure \ref{fig:storm_query_urn5} illustrates how Storm scroll block is located
1976  in a tightly structured overlay using DOLR method, where pointer random string is known.  in a tightly structured overlay using DOLR method, where pointer random string is known.
1977    
1978  Each of these algortihms can locate Fenfire related data in $O(\log{n})$ time:  Each of these algorithms can locate Fenfire related data in $O(\log{n})$ time:
1979  $O(\log{n})$ time for query routing to pointer peer and constant time for  $O(\log{n})$ time for query routing to pointer peer and constant time for
1980  locating hosting peer with a given reference link. Time required for transferring  locating hosting peer with a given reference link. Time required for transferring
1981  the data is not included.  the data is not included.
# Line 1999  the data is not included. Line 1999  the data is not included.
1999  \subsection{Problems}  \subsection{Problems}
2000    
2001  Perhaps the most biggest issue in Peer-to-Peer systems is non-maturity of  Perhaps the most biggest issue in Peer-to-Peer systems is non-maturity of
2002  security techologies. For instance, online entities cannot be identified  security technologies. For instance, online entities cannot be identified
2003  safely (e.g., the Sybil attack \cite{douceur02sybil}). For Fenfire, one  safely (e.g., the Sybil attack \cite{douceur02sybil}). For Fenfire, one
2004  security related problem occurs when user wants to perform global data lookup with a given  security related problem occurs when user wants to perform global data lookup with a given
2005  pointer random string; how user is able to verify the correctness  pointer random string; how user is able to verify the correctness
# Line 2009  mentioned problem; data lookup is perfor Line 2009  mentioned problem; data lookup is perfor
2009  from the system. How do we are able to know if this was a spam attack, or the  from the system. How do we are able to know if this was a spam attack, or the
2010  data really doesn't exist in the system ? Another problem related to Fenfire's  data really doesn't exist in the system ? Another problem related to Fenfire's
2011  security is that if a user downloads data from the network to local computer  security is that if a user downloads data from the network to local computer
2012  and after network disconnetcion, user wants to verify \emph{offline} the  and after network disconnection, user wants to verify \emph{off line} the
2013  authenticity of data. Obviously, optimal solution to all security issues would  authenticity of data. Obviously, optimal solution to all security issues would
2014  be that digital signatures are included to every message sent to the system.  be that digital signatures are included to every message sent to the system.
2015  However, these problems are not only limited to Fenfire, it concerns all  However, these problems are not only limited to Fenfire, it concerns all
# Line 2052  Our future work includes support for sea Line 2052  Our future work includes support for sea
2052  links in a Peer-to-Peer network. Specifically, we want to find transclusions  links in a Peer-to-Peer network. Specifically, we want to find transclusions
2053  or xanalogical links in a global scale. Preliminary analysis have showed  or xanalogical links in a global scale. Preliminary analysis have showed
2054  that these questions are rather different than locating scroll or pointer  that these questions are rather different than locating scroll or pointer
2055  blocks \emph{directly} from the network. Techiques used in distributed  blocks \emph{directly} from the network. Techniques used in distributed
2056  database systems may prove to be useful. Some fundamental results  database systems may prove to be useful. Some fundamental results
2057  regarding Peer-to-Peer and database systems has already been  regarding Peer-to-Peer and database systems has already been
2058  presented \cite{gribble01p2pdatabase}.  presented \cite{gribble01p2pdatabase}.
2059    
2060  As security techologies comes more mature, we wish to apply these  As security technologies comes more mature, we wish to apply these
2061  techologies with Fenfire, if applicable.  technologies with Fenfire, if applicable.
2062    
2063  In the following months, we will implement a Fenfire Peer-to-Peer  In the following months, we will implement a Fenfire Peer-to-Peer
2064  prototype.  prototype.

Legend:
Removed from v.1.128  
changed lines
  Added in v.1.129

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26