/[gzz]/gzz/Documentation/misc/hemppah-progradu/masterthesis.tex
ViewVC logotype

Diff of /gzz/Documentation/misc/hemppah-progradu/masterthesis.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.191 by hemppah, Tue Mar 25 16:22:34 2003 UTC revision 1.192 by hemppah, Wed Mar 26 09:16:53 2003 UTC
# Line 256  Adamic et al. \cite{adamic99small, adami Line 256  Adamic et al. \cite{adamic99small, adami
256  have studied different data lookup methods in power-law networks and have found that by  have studied different data lookup methods in power-law networks and have found that by
257  instructing the peers that forward data lookups to select high degree peers, the performance of data lookup  instructing the peers that forward data lookups to select high degree peers, the performance of data lookup
258  increases significantly. Figure \ref{fig:gnutella_powerlaw} presents an example topology of power-law network with three high  increases significantly. Figure \ref{fig:gnutella_powerlaw} presents an example topology of power-law network with three high
259  degree peers. Some of the most recent loosely structured Peer-to-Peer systems have adopted this method to improve the data lookup model of loosely structured  degree peers. Some of the most recent loosely structured Peer-to-Peer protocols have adopted this method to improve the data lookup model of loosely structured
260  systems \cite{gnutella2url, fasttrackurl}. Both protocols use high degree peers to optimize the data lookup model of the  systems \cite{gnutella2url, fasttrackurl}. Both protocols use high degree peers to optimize the data lookup model of the
261  system. Shareaza \cite{shareazaurl} uses the Gnutella2 protocol \cite{gnutella2url} in data lookups, Morpheus \cite{morpheusurl}  system. Shareaza \cite{shareazaurl} uses the Gnutella2-based flooding protocol \cite{gnutella2url} in data lookups, Morpheus \cite{morpheusurl}
262  and KaZaa \cite{kazaaurl} use the FastTrack protocol \cite{fasttrackurl}.  and KaZaa \cite{kazaaurl} use the FastTrack-based flooding protocol \cite{fasttrackurl}.
263  It is not clear whether the power-law method is scalable or not,  It is not clear whether the power-law method is scalable or not,
264  as the majority of the query requests are sent only to the high degree peers while making  as the majority of the query requests are sent only to the high degree peers while making
265  these peers to bear the load of the entire system.  these peers to bear the load of the entire system.
# Line 323  that \emph{peer identifiers} are assigne Line 323  that \emph{peer identifiers} are assigne
323  a large \emph{identifier space} by the overlay. Globally unique identifiers  a large \emph{identifier space} by the overlay. Globally unique identifiers
324  are also assigned to application-specific data items, \emph{keys},  are also assigned to application-specific data items, \emph{keys},
325  which are selected from the same identifier space. For instance, globally unique keys can be created  which are selected from the same identifier space. For instance, globally unique keys can be created
326  using a cryptographic content hash (e.g., \cite{fips-sha-1}) over the contents of a data item.  using a cryptographic content hash (e.g., SHA-1 \cite{fips-sha-1}) over the contents of a data item.
327  The form of identifier space differs between proposed systems. Geometrical circular form of identifier space (and variants)  The form of identifier space differs between proposed systems. Geometrical circular form of identifier space (and variants)
328  is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde},  is most widely used. For instance, Chord \cite{stoica01chord}, Koorde \cite{kaashoek03koorde},
329  Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry}  Pastry \cite{rowston01pastry}, SWAN \cite{bonsma02swan}, Tapestry \cite{zhao01tapestry}
# Line 340  Figure \ref{fig:structured_hashing} illu Line 340  Figure \ref{fig:structured_hashing} illu
340  process of data to key mapping in a tightly structured overlay.    process of data to key mapping in a tightly structured overlay.  
341  Also, each peer in the tightly structured overlay maintains a \emph{routing table}, which  Also, each peer in the tightly structured overlay maintains a \emph{routing table}, which
342  consists of identifiers and IP addresses of other peers in the overlay. Entries of the routing  consists of identifiers and IP addresses of other peers in the overlay. Entries of the routing
343  table represent peer's neighbors in the overlay network.  table represent peer's neighbors in the overlay network.
344    
345    Currently, all proposed tightly structured overlays provide at least
346    poly--logarithmical data lookup operations. However, there are some key
347    differences in the data structures representing the identifier space.
348    For example, Chord \cite{stoica01chord}, Skip graphs \cite{AspnesS2003} and SkipNet \cite{harvey03skipnet2} maintain a
349    distributed data structure which resembles Skip list \cite{78977}.
350    In figure \ref{fig:structured_query}, we present an overview of Chord's lookup process.
351    On the right side of Chord's lookup process, the same data lookup process
352    is shown as a binary-tree abstraction.  It can be noticed, that in each step, the distance
353    decreases with a logarithmic efficiency.
354    
355    \begin{figure}
356    \centering
357    \includegraphics[width=10cm, height=6cm]{structured_query.eps}
358    \caption{Chord's simplified data lookup process on top of tightly structured overlay.}
359    \label{fig:structured_query}
360    \end{figure}
361    
362    Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry
363    \cite{zhao01tapestry} uses balanced $k$-trees to implement the overlay. Figure
364    \ref{fig:kademlia_lookup} shows the process of Kademlia's
365    data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}),
366    which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup
367    efficiency. Koorde \cite{kaashoek03koorde}, a recent modification of Chord, uses de Bruijn graphs
368    \cite{debruijn46graph} to maintain local routing tables. Koorde \cite{kaashoek03koorde} requires
369    each peer to have only about two links to other peers to provide $O(\log{n})$ performance.
370    
371    
372    \begin{figure}
373    \centering
374    \includegraphics[width=10cm, height=8cm]{kademlia_lookup.eps}
375    \caption{Kademlia's simplified data lookup process on top of tightly structured overlay.}
376    \label{fig:kademlia_lookup}
377    \end{figure}
378    
379  \begin{figure}  \begin{figure}
380  \centering  \centering
# Line 349  table represent peer's neighbors in the Line 383  table represent peer's neighbors in the
383  \label{fig:structured_hashing}  \label{fig:structured_hashing}
384  \end{figure}  \end{figure}
385    
 Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} have listed four requirements  
 for tightly structured overlays\footnote{Authors use the term 'DHT' in their text, but in this context  
 it doesn't matter as they list \emph{general} properties of tightly structured overlays.} which have to be addressed in order  
 to perform efficient data lookups in tightly structured overlays.  
 First, mapping of keys to peers must be done in a load-balanced  
 way. Second, the overlay must be able to forward a data lookup for a  
 specific key to an appropriate peer. Third, overlay must  
 support efficient distance function. Finally,  routing tables for each peer  
 must be constructed and maintained adaptively.  
   
386  Currently, there are only three higher level abstractions which tightly structured overlays provide  Currently, there are only three higher level abstractions which tightly structured overlays provide
387  \cite{zhao03api}. Each of these abstractions represent a storage layer in the overlay, but  \cite{zhao03api}. Each of these abstractions represent a storage layer in the overlay, but
388  have semantical differences in the \emph{usage} of the overlay.  have semantical differences in the \emph{usage} of the overlay.
# Line 423  In the DOLR abstraction, a data item is Line 447  In the DOLR abstraction, a data item is
447  \label{fig:Strucutred_lookup_using_DOLR_model}  \label{fig:Strucutred_lookup_using_DOLR_model}
448  \end{figure}  \end{figure}
449    
 Currently, all proposed tightly structured overlays provide at least  
 poly--logarithmical data lookup operations. However, there are some key  
 differences in the data structures representing the identifier space.  
 For example, Chord \cite{stoica01chord}, Skip graphs \cite{AspnesS2003} and SkipNet \cite{harvey03skipnet2} maintain a  
 distributed data structure which resembles Skip lists \cite{78977}.  
 In figure \ref{fig:structured_query}, we present an overview of Chord's lookup process.  
 On the right side of Chord's lookup process, the same data lookup process  
 is shown as a binary-tree abstraction.  It can be noticed, that in each step, the distance  
 decreases with a logarithmic efficiency.  
   
 \begin{figure}  
 \centering  
 \includegraphics[width=10cm, height=6cm]{structured_query.eps}  
 \caption{Chord's simplified data lookup process on top of tightly structured overlay.}  
 \label{fig:structured_query}  
 \end{figure}  
   
 Kademlia \cite{maymounkov02kademlia}, Pastry \cite{rowston01pastry} and Tapestry  
 \cite{zhao01tapestry} uses balanced $k$-trees to implement the overlay. Figure  
 \ref{fig:kademlia_lookup} shows the process of Kademlia's  
 data lookup. Viceroy \cite{malkhi02viceroy} maintains a butterfly data structure (e.g., \cite{226658}),  
 which requires only a constant number of neighbor peers while providing $O(\log{n})$ data lookup  
 efficiency. Koorde \cite{kaashoek03koorde}, a recent modification of Chord, uses de Bruijn graphs  
 \cite{debruijn46graph} to maintain local routing tables. Koorde \cite{kaashoek03koorde} requires  
 each peer to have only about two links to other peers to provide $O(\log{n})$ performance.  
   
450    
451  \begin{figure}  In tightly structured system, messages are routed across the overlay towards peers, whose
 \centering  
 \includegraphics[width=10cm, height=8cm]{kademlia_lookup.eps}  
 \caption{Kademlia's simplified data lookup process on top of tightly structured overlay.}  
 \label{fig:kademlia_lookup}  
 \end{figure}  
   
   
 All messages are routed across the overlay towards peers, whose  
452  peer identifier is gradually ''closer'' to the key's identifier  peer identifier is gradually ''closer'' to the key's identifier
453  in the identifier space. The distance can be measured by numerical  in the identifier space. The distance can be measured by numerical
454  difference between identifiers (e.g., Chord \cite{stoica01chord}), number of  difference between identifiers (e.g., Chord \cite{stoica01chord}), number of
# Line 492  TCP/IP-protocols. PeerNet has the same p Line 482  TCP/IP-protocols. PeerNet has the same p
482  overlays, i.e., $O(\log{n})$ space required for maintaining information about other peers in  overlays, i.e., $O(\log{n})$ space required for maintaining information about other peers in
483  the system and $O(\log{n})$ data lookup efficiency.  the system and $O(\log{n})$ data lookup efficiency.
484    
485    Balakrishnan et al. \cite{balakrishanarticle03lookupp2p} have listed four requirements
486    for tightly structured overlays\footnote{Authors use the term 'DHT' in their text, but in this context
487    it doesn't matter as they list \emph{general} properties of tightly structured overlays.} which have to be addressed in order
488    to perform efficient data lookups in tightly structured overlays.
489    First, mapping of keys to peers must be done in a load-balanced
490    way. Second, the overlay must be able to forward a data lookup for a
491    specific key to an appropriate peer. Third, overlay must
492    support efficient distance function. Finally,  routing tables for each peer
493    must be constructed and maintained adaptively.
494    
495  Additionally, authors argue in \cite{balakrishnan03semanticfree} that tightly structured systems  Additionally, authors argue in \cite{balakrishnan03semanticfree} that tightly structured systems
496  are suitable for next generation Reference Resolutions Services (RRS)\footnote{  are suitable for next generation Reference Resolutions Services (RRS)\footnote{
497  Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system in the Internet.}. They present  Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system in the Internet.}. They present
# Line 522  peers \cite{osokine02distnetworks}, \cit Line 522  peers \cite{osokine02distnetworks}, \cit
522  whether all proposed algorithms can preserve logarithmic efficiency and scalability properties  whether all proposed algorithms can preserve logarithmic efficiency and scalability properties
523  in real-life applications or not; several tightly structured systems  in real-life applications or not; several tightly structured systems
524  assume that participating peers are homogeneous, and the rate of join or leave operation is low \cite{gurmeet03symphony,  assume that participating peers are homogeneous, and the rate of join or leave operation is low \cite{gurmeet03symphony,
525  libennowell01observations}.    libennowell01observations, rowston03controlloingreliability}.  
526    
527  To end user, the biggest difference between these systems is how data lookups are performed. Loosely  To end user, the biggest difference between these systems is how data lookups are performed. Loosely
528  structured systems provide more rich and user friendly way of searching data than tightly structured systems  structured systems provide more rich and user friendly way of searching data than tightly structured systems
529  as they have a support for keyword searches. Tightly structured  as they have a support for keyword searches \cite{yang02efficientsearch, lv02searchreplication}. Tightly structured
530  systems support only exact key lookups since each data item is identified by globally unique keys.  systems support only exact key lookups since each data item is identified by globally unique keys \cite{balakrishanarticle03lookupp2p,
531    harren02complex, ansaryefficientbroadcast03}.
532    
533  Table \ref{table_comparison_approach} lists the key differences between the loosely structured  Table \ref{table_comparison_approach} lists the key differences between the loosely structured
534  approach and the tightly structured approach.  approach and the tightly structured approach.
# Line 582  approach and the tightly structured appr Line 583  approach and the tightly structured appr
583  \parbox{100pt}{Yes}  \parbox{100pt}{Yes}
584  \\ \hline  \\ \hline
585    
 \parbox{90pt}{Construction and maintenance of the overlay} &  
 \parbox{100pt}{Uncontrolled and ad hoc}  &  
 \parbox{100pt}{Controlled and structured}  
 \\ \hline  
586                                        
587  \parbox{90pt}{Scalable query lookup model} &  \parbox{90pt}{Scalable data lookup model} &
588  \parbox{100pt}{No} &  \parbox{100pt}{No} &
589  \parbox{100pt}{Yes}  \parbox{100pt}{Yes}
590  \\ \hline  \\ \hline
591                                                    
592  \parbox{90pt}{Data item mapped into the overlay} &  \parbox{90pt}{Data items mapped into the overlay} &
593  \parbox{100pt}{No} &  \parbox{100pt}{No} &
594  \parbox{100pt}{Yes}  \parbox{100pt}{Yes}
595  \\ \hline  \\ \hline
# Line 829  Here, we describe the listed properties Line 826  Here, we describe the listed properties
826  In this chapter, we discuss open problems in Peer-to-Peer research.  In this chapter, we discuss open problems in Peer-to-Peer research.
827  Note that the open problems list considered here is not meant  Note that the open problems list considered here is not meant
828  to be an exhaustive survey of \emph{all} open problems in Peer-to-Peer domain;  to be an exhaustive survey of \emph{all} open problems in Peer-to-Peer domain;
829  we focus our attention to some security, scalability, usability and performance related  we focus our attention to some issues related security, scalability, usability and performance.
830  issues only.  
831    
832  \section{Overview}  \section{Overview}
833    
# Line 846  the data lookup process creates lot of e Line 843  the data lookup process creates lot of e
843    
844  In tightly structured system the main concern is to make overlay's data lookup process  In tightly structured system the main concern is to make overlay's data lookup process
845  more fault tolerant against hostile attacks. Other key problems in tightly structured  more fault tolerant against hostile attacks. Other key problems in tightly structured
846  systems are the lack of keyword searches, support for heterogeneous peers and load balancing  systems are the lack of keyword searches \cite{harren02complex, ansaryefficientbroadcast03}, support for heterogeneous peers
847  \cite{balakrishanarticle03lookupp2p}.  \cite{rowston03controlloingreliability} and load balancing \cite{balakrishanarticle03lookupp2p, byers03dhtbalancing}.
848    
849  \section{Security problems in Peer-to-Peer}  \section{Security problems in Peer-to-Peer}
850    
# Line 867  the Fail-stop attack, the Spam attack \c Line 864  the Fail-stop attack, the Spam attack \c
864  the Distributed Denial of Service attack.  the Distributed Denial of Service attack.
865    
866  In the Sybil attack model \cite{douceur02sybil}, a hostile entity presents multiple  In the Sybil attack model \cite{douceur02sybil}, a hostile entity presents multiple
867  entities, i.e., when a peer selects a subset of entities to perform a operation, a peer can select the same  entities, i.e., when a peer communicates with a subset of other participating entities to perform a operation, a peer communicates
868  hostile entity multiple times. Therefore, one hostile entity can control a large fraction of Peer-to-Peer system thereby  only with the same hostile entity. Therefore, one hostile entity can control a large fraction of Peer-to-Peer system while
869  repressing the redundancy of the system. Unfortunately, currently there are no realizable techniques for against the Sybil  repressing the redundancy of the system. Authors argue in  \cite{douceur02sybil} that without a centralized authority, Sybil attacks are always possible in a Peer-to-Peer
870  attack: without a centralized authority, Sybil attacks are always possible in a Peer-to-Peer  system except under extreme and unrealistic assumptions of resource parity and coordination among entities. According to \cite{douceur02sybil}, u
871  system except under extreme and unrealistic assumptions of resource parity and coordination among entities \cite{douceur02sybil}.  nrealistic assumptions include: all entities should be nearly homogeneous, all identities can be validated simultaneously by all
872  Castro et al. \cite{castro02securerouting} suggest the use of cryptographic content hashes in the creation process of peer identifier  entities across the system and when accepting identities that are not directly validated, the required number of certificates exceeds
873  against the Sybil attack. According to authors, in this technique the IP address of a peer can be verified by the other peer.  the number of systemwide failures. Castro et al. \cite{castro02securerouting} suggest the use of cryptographic content hashes in the
874    creation process of peer identifier against the Sybil attack. According to authors, in this technique the IP address of a peer can be verified by the other peer.
875  They call this method as a one form of \emph{self-certifying data}.  They call this method as a one form of \emph{self-certifying data}.
876    
877  In the Fail-stop attack model, cited in \cite{naor03simpledht}, a faulty peer is deleted from the Peer-to-Peer system. Thus,  In the Fail-stop attack model, cited in \cite{naor03simpledht}, a faulty peer is deleted from the Peer-to-Peer system. Thus,
878  a specific data item can be lost from the system temporaraly (or permanently). The reason for the faultiness of a peer can be a  a specific data item can be lost from the system temporaraly (or permanently). The reason for the faultiness of a peer can be a
879  software failure or a hostile attack. The Byzantine attack model \cite{357176} is closely related to Fail-stop model. The Byzantine model can  software failure or a hostile attack. The Byzantine attack model \cite{357176} is closely related to Fail-stop model. In the Byzantine attack model
880  be seen as more severe than Fail-stop model as there are no restrictions over the behavior of faulty peers; for instance,  $3f + 1$ is the minimum number of peers that allow system to provide the safety and liveness properties when up to $f$ peers are faulty \cite{357176}.  
881  the cooperation between multiple malicious faulty peers is possible  \cite{357176}. A practical solution for the Byzantine failures have been  The Byzantine model can be seen as more severe than Fail-stop model as there are no restrictions over the behavior of faulty peers, e.g., the cooperation
882  proposed by Castro et al. \cite{296824}.  between multiple \emph{malicious} faulty peers is possible \cite{357176}. A practical solution for the Byzantine failures have been
883    proposed by Castro et al. \cite{296824}. Authors use in their work replication algorithm to tolerate Byzantine faults and cryptographic
884    certificate techniques to prevent spoofing and replays and to detect corrupted messages.
885    
886  The Spam generating attack \cite{naor03simpledht} is an another known attack model against Peer-to-Peer system. In the Spam  The Spam generating attack \cite{naor03simpledht} is an another known attack model against Peer-to-Peer system. In the Spam
887  attack, a hostile or faulty peer may produce false information of the data, or refuses to (or is not able to) reply to requests.  attack, a hostile or faulty peer may produce false information of the data, or refuses to (or is not able to) reply to requests.
888  Possible solution against this attack is that peer should not trust a single entity. Instead, a peer should get  Naor et al. \cite{naor03simpledht} have proposed a partial solution against Spam attack in a \emph{faulty} peer environment (not hostile).
 information from multiple entities and trust on the majority's opinion. This method requires more messages to be  
 sent to the network while increasing the system load. However, if the Spam attack is combined with the Sybil attack, obviously  
 the previously mentioned solution doesn't work. Naor et al. \cite{naor03simpledht} have proposed a partial solution against Spam attack  
 in a \emph{faulty} peer environment (not hostile).  
889    
890  Overloading of targeted peers is a form of Distributed Denial of Service attack (DDoS) (see, e.g., \cite{372148}). For instance,  Overloading of targeted peers is a form of Distributed Denial of Service attack (DDoS) (see, e.g., \cite{372148}). For instance,
891  a hostile entity can attempt to burden targeted peers with garbage network packets. As a consequence, peers may act incorrectly or  a hostile entity can attempt to burden targeted peers with garbage network packets. As a consequence, peers may act incorrectly or
# Line 905  and replicas should be located physicall Line 901  and replicas should be located physicall
901  According to \cite{aberer01trust}, mutual trust ''...allows agents to cooperate in a game-theoretic situation that corresponds  According to \cite{aberer01trust}, mutual trust ''...allows agents to cooperate in a game-theoretic situation that corresponds
902  to the repeated prisoners dilemma and leads in the long term to an increased aggregated utility for the participating agents''.  to the repeated prisoners dilemma and leads in the long term to an increased aggregated utility for the participating agents''.
903  They define \emph{trust management} as a mechanism that allows to establish mutual trust. Furthermore, \emph{reputation} is a measure  They define \emph{trust management} as a mechanism that allows to establish mutual trust. Furthermore, \emph{reputation} is a measure
904  that is derived from knowledge on interactions in the past \cite{aberer01trust} In this subsection, we discuss mechanisms to maintain  that is derived from knowledge on interactions in the past \cite{aberer01trust}. In this subsection, we discuss mechanisms to maintain
905  trust in Peer-to-Peer systems.  trust in Peer-to-Peer systems.
906    
907  Trust in Peer-to-Peer systems is based on \emph{reputation}. Little research has been done on reputation models in Peer-to-Peer  Trust in Peer-to-Peer systems is based on \emph{reputation}. Little research has been done on reputation models in Peer-to-Peer
# Line 958  to split data into fragments \cite{Shami Line 954  to split data into fragments \cite{Shami
954  distributed systems which are able to provide some level of anonymity (e.g., \cite{mneturl}).  distributed systems which are able to provide some level of anonymity (e.g., \cite{mneturl}).
955    
956  Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no  Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no
957  such a system which is able to provide all types of anonymity. Specifically, the conflicts  such a system which is able to provide complete anonymity. Specifically, the conflicts
958  between anonymity and other properties of Peer-to-Peer system require more research work.  between anonymity and other properties of Peer-to-Peer system require more research work.
959    
960    
# Line 978  loosely structured systems. Line 974  loosely structured systems.
974    
975  \subsection{Hostile entities}  \subsection{Hostile entities}
976    
977  One serious problem in Peer-to-Peer systems is the inability to identify hostile entities.  One serious problem in Peer-to-Peer systems is the inability to distinguish hostile entities from regular entities
978    trustworthy.
979  One possible solution is to use a self-monitoring system, such as SOMO \cite{zhang03somo}, in which a self-monitoring overlay  One possible solution is to use a self-monitoring system, such as SOMO \cite{zhang03somo}, in which a self-monitoring overlay
980  constantly analyses the Peer-to-Peer overlay. Self-monitoring overlay is built on top of Peer-to-Peer overlay. Authors in  constantly analyses the Peer-to-Peer overlay. Self-monitoring overlay is built on top of Peer-to-Peer overlay. Authors in
981  \cite{sit02securitycons} suggest the use of system invariants. They emphasize that system invariants should be veriable, and if  \cite{sit02securitycons} suggest the use of system invariants. They emphasize that system invariants should be veriable, and if
# Line 995  identifier, such as crypto-based puzzles Line 992  identifier, such as crypto-based puzzles
992    
993  \subsection{Secure query routing}  \subsection{Secure query routing}
994    
995  By secure routing, we mean that a Peer-to-Peer system is able to deliver a network message  Secure query routing is essential to any Peer-to-Peer system. By secure routing in this context, we mean that a Peer-to-Peer system
996  thoughout the overlay to a correct destination.  is able to deliver a network message thoughout the overlay to a correct destination efficiently.
997    
998  Aspnes et al. in \cite{aspnes02faultrouting} and Kaashoek et al. in \cite{kaashoek03koorde} formally  Aspnes et al. in \cite{aspnes02faultrouting} and Kaashoek et al. in \cite{kaashoek03koorde} formally
999  prove the lower and upper bounds for the space requirements of locating a specific data item reliable in a  prove the lower and upper bounds for the space requirements of locating a specific data item reliable in a
# Line 1638  in the Fenfire system. Line 1635  in the Fenfire system.
1635  \section{Xanalogical storage model}  \section{Xanalogical storage model}
1636    
1637  Xanalogical storage model \cite{nelson99xanalogicalneeded} is a different kind of model for  Xanalogical storage model \cite{nelson99xanalogicalneeded} is a different kind of model for
1638  presenting data and relationships between data. \emph{Enfilade},  presenting data and relationships between data, e.g., while in the World Wide Web links are
1639  can be considered as a mutable ''virtual file'' (or part of one), which is a list  between \emph{documents}, in xanalogical storage model links are between individual
1640    \emph{characters}. \emph{Enfilade},can be considered as a mutable ''virtual file'' (or part of one), which is a list
1641  of fluid media content. Fluid media is the smallest units of data in xanalogical storage  of fluid media content. Fluid media is the smallest units of data in xanalogical storage
1642  model. \emph{Transclusion} is an inclusion in  model. \emph{Transclusion} is an inclusion in
1643  enfilade of contents already used in another enfilade. With the transclusion, a system  enfilade of contents already used in another enfilade. With the transclusion, a system
1644  implementing xanalogical storage model is able to show all data content that share the same  implementing xanalogical storage model is able to show \emph{all} data content that share the same
1645  fluid media with current data content (e.g., all documents in a system containing a specific  fluid media with current data content (e.g., all documents in a system containing document's text).
1646  document's text).  Figure \ref{fig:xanalogical_model}
   
 While in the World Wide Web links are  
 between \emph{documents}, in xanalogical storage model links are between individual  
 \emph{characters}. Figure \ref{fig:xanalogical_model}  
1647  illustrates xanalogical storage model with documents, text and characters.  illustrates xanalogical storage model with documents, text and characters.
1648  Links between data are external  
1649    In xanalogical storage model, links between data are external
1650  and bidirectional. A link is shown between any two data contents  and bidirectional. A link is shown between any two data contents
1651  containing a specific \emph{fluid media unit} (e.g., a character) that the link connects.  containing a specific \emph{fluid media unit} (e.g., a character) that the link connects.
1652  Each fluid media unit in xanalogical storage model has a  Each fluid media unit in xanalogical storage model has a
# Line 1691  for creating location-independent, globa Line 1686  for creating location-independent, globa
1686  content hash, all identifiers are directly the data verifiers as well. The uniquess of blocks creates  content hash, all identifiers are directly the data verifiers as well. The uniquess of blocks creates
1687  a basis for implementing xanalogical storage model in the Fenfire system. Storm blocks have much in common with regular files as they  a basis for implementing xanalogical storage model in the Fenfire system. Storm blocks have much in common with regular files as they
1688  both contain the data. The main difference is that Storm blocks are \emph{immutable} since any  both contain the data. The main difference is that Storm blocks are \emph{immutable} since any
1689  change to the byte sequence would change block's hash value (globally unique identifier).    change to the byte sequence would change block's hash value (i.e., globally unique identifier).  
1690    
1691  Support for immutable data is built on the immutable abstraction. Storm uses  Support for immutable data is built on the immutable abstraction. Storm uses
1692  \emph{pointers} and \emph{diffs} for dealing with this kind of data. Using diffs  \emph{pointers} and \emph{diffs} for dealing with this kind of data. Using diffs
# Line 1700  paper, however, we discuss only pointers Line 1695  paper, however, we discuss only pointers
1695  More information about diffs can be found from \cite{fallenstein03storm}.  More information about diffs can be found from \cite{fallenstein03storm}.
1696    
1697  \emph{Pointer} \cite{benja02urn5, fallenstein03storm} is a semantic-free updatable reference to  \emph{Pointer} \cite{benja02urn5, fallenstein03storm} is a semantic-free updatable reference to
1698  Storm data block, i.e., Storm scroll block. Pointer is unique reference to the data and it is usually  Storm data block. Pointer is a unique reference to the data and it is usually
1699  represented as a random string. Storm pointers are rather a \emph{concept} of data (e.g., ''The first page of the most recent  represented as a random string. Storm pointers are rather a \emph{concept} of data (e.g., ''The front page of the most recent
1700  version of New York Times newspaper'') whereas scroll blocks \emph{contain} the data  version of New York Times newspaper'') whereas scroll blocks \emph{contain} the data
1701  (''New York Times newspaper, 10.10.2002, version 1.0'').  (''New York Times newspaper, 10.10.2002, version 1.0'').
1702  Figure \ref{fig:storm_model} illustrates simplified Storm storage model with pointers.  Figure \ref{fig:storm_model} illustrates Storm storage model with pointers.
1703    
1704  Each pointer is associated with a collection of \emph{pointer blocks}.  Each pointer is \emph{linked} to a collection of \emph{pointer blocks}.
1705  Pointers can be created by a user, before the creation of scroll blocks. Pointer blocks  Pointers can be created by a user, before the creation of scroll blocks. Pointer blocks
1706  are created automatically by Storm when a scroll block is associated with a pointer  are created automatically by Storm when a scroll block is created and associated with a pointer
1707  (e.g., by a user when creating a concept of ''The first page of the most recent  (e.g., a user creates a scroll block associated with the concept ''The front page of the most recent
1708  version of New York Times newspaper''). Pointer block has always a single target (i.e., a scroll block)  version of New York Times newspaper''). Pointer block has always a single target (i.e., a scroll block)
1709  for the pointer, saying that pointer $P$ targets block $B$. In addition to this, pointer block  for the pointer, saying that pointer $P$ targets block $B$. In addition to this, pointer block
1710  may contain a list of zero or more obsoleted pointer blocks: when a new version of pointer  may contain a list of zero or more obsoleted pointer blocks: when a new version of pointer
# Line 1741  planned in future versions of Storm.} Line 1736  planned in future versions of Storm.}
1736  \chapter{Evaluation of Peer-to-Peer for Fenfire}  \chapter{Evaluation of Peer-to-Peer for Fenfire}
1737    
1738  In this chapter we evaluate Fenfire in Peer-to-Peer environment.  In this chapter we evaluate Fenfire in Peer-to-Peer environment.
1739  We start by giving a problem overview. Then, we define Fenfire's special needs and evaluate existing  We start by giving a problem overview. Then, we define special needs and evaluate existing
1740  Peer-to-Peer approaches in light of these requirements. After that, we propose a combination  Peer-to-Peer approaches in light of these requirements. After that, we propose a combination
1741  of Peer-to-Peer techniques reviewed in this thesis to be used with Fenfire and present simple methods to perform data  of Peer-to-Peer techniques reviewed in this thesis to be used with Fenfire and present simple methods to perform data
1742  lookups (data lookups are required by Alph module). In the end of this chapter, we discuss possible problems of using Fenfire  lookups. Data lookups are required by Alph module which implements the xanalogical storage model in Fenfire.
1743    In the end of this chapter, we discuss possible problems of using Fenfire
1744  in Peer-to-Peer environment.  in Peer-to-Peer environment.
1745    
1746    
# Line 1765  environment. At the Hypertext '02 panel, Line 1761  environment. At the Hypertext '02 panel,
1761  participants responded whether Peer-to-Peer systems are suitable for hypermedia  participants responded whether Peer-to-Peer systems are suitable for hypermedia
1762  publishing or not.  publishing or not.
1763    
   
1764  In Peer-to-Peer environment, our objectives are simple but yet hard to fulfill.  In Peer-to-Peer environment, our objectives are simple but yet hard to fulfill.
1765  First, as discussed in chapter 4, xanalogical document is a ''virtual  First, as discussed in chapter 4, xanalogical document is a ''virtual
1766  file'', in which parts of the document are fetched from a  file'', in which parts of the document are fetched from a
1767  \emph{global} data repository\footnote{Global repository is not a requirement. Locally constructed xanalogical  \emph{global} data repository\footnote{Global repository is not a requirement. Locally constructed xanalogical
1768  documents are feasible and they can be assembled without any global data.}. Thus, system implementing xanalogical storage model \emph{must}  documents are feasible and they can be assembled without global data repository.}. Thus, system implementing xanalogical storage model \emph{must}
1769  support global data lookups order to assemble the ''virtual file'' from fragments of data.  support global data lookups order to assemble the ''virtual file'' from fragments of data.
1770  Specifically, our task is to locate and fetch (i.e., obtain) \emph{all} Storm scroll blocks, associated to a specific ''virtual  Specifically, our task is to locate and fetch (i.e., obtain) \emph{all} Storm blocks\footnote{These blocks are called \emph{scroll} blocks.},
1771  file'' from the Peer-to-Peer once the construction of ''virtual'' file  associated to a specific ''virtual file'' from the Peer-to-Peer once the construction of ''virtual'' file
1772  is resolved (i.e., we know what scroll blocks are required to assemble the ''virtual file''). Also, in addition to the  is resolved (i.e., we know what blocks are required to assemble the ''virtual file''). Also, in addition to the
1773  \emph{direct} scroll block obtaining using globally unique identifier of Storm scroll block,  \emph{direct} block obtaining using globally unique identifier of Storm block,
1774  we also must support the \emph{indirect} obtaining of Storm scroll block using the pointers.  we also must support the \emph{indirect} obtaining of Storm block using the pointer mechanism.
1775  Second, we want that users' operations in Fenfire  Second, we want that users' operations in Fenfire
1776  are location transparent: data lookups have to be efficient, since constructing  are location transparent: data lookups have to be efficient, since constructing
1777  one ''virtual file'' may need obtaining several Storm blocks, which are distributed  one ''virtual file'' may need obtaining several Storm blocks, which are distributed
# Line 1858  Furthermore, multisource downloads can b Line 1853  Furthermore, multisource downloads can b
1853  hot spots in the system \cite{ratnasamy02routing}. Current client-server implementation of Fenfire uses  hot spots in the system \cite{ratnasamy02routing}. Current client-server implementation of Fenfire uses
1854  standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content  standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content
1855  hash for verifying the integrity of data by recomputing the content hash  hash for verifying the integrity of data by recomputing the content hash
1856  for block. In face of multisource downloads, Fenfire must support  for Storm block. In face of multisource downloads, Fenfire must support
1857  tree-based hashes\footnote{With multisource downloads, tree-based hash functions can be used  tree-based hashes\footnote{With multisource downloads, tree-based hash functions can be used
1858  to verify fixed length segments of data. If hash value of data segment is incorrect,  to verify fixed length segments of data. If hash value of data segment is incorrect,
1859  we need only to fetch \emph{segment} of data (instead of whole data) from  we need only to fetch \emph{segment} of data (instead of whole data) from
# Line 1874  DHT-based storage systems, such as CFS \ Line 1869  DHT-based storage systems, such as CFS \
1869  severe problems with load balancing in a highly heterogeneous environment \cite{rao03loadbalancing}. The problem is caused by peers  severe problems with load balancing in a highly heterogeneous environment \cite{rao03loadbalancing}. The problem is caused by peers
1870  which may not be able to store relatively large blocks, assigned randomly by the mapping function of the overlay.  which may not be able to store relatively large blocks, assigned randomly by the mapping function of the overlay.
1871    
1872  In our method, each peer maintains the following local data structures for local operations: a data structure for listing all  For simplicity, we assume that we have resolved the construction of the ''virtual file'' before locating any Storm blocks, i.e.,
 key-value pairs which are assigned by the overlay; a data structure for listing all key-value pairs which are assigned by the overlay  
 in the chronological order (the most recent block is topmost). We use Storm blocks' identifiers and pointers  
 as \emph{keys} of the overlay.  
   
 We assume that we have resolved the construction of the ''virtual file'' before locating any Storm blocks, i.e.,  
1873  when assembling the ''virtual file'' we know all the Storm blocks, which are required to complete the ''virtual file''.  when assembling the ''virtual file'' we know all the Storm blocks, which are required to complete the ''virtual file''.
1874  Also, we don't respond to the security issues related to Peer-to-Peer systems, since there is no working solution  Also, we don't respond to the security issues related to Peer-to-Peer systems, since there is no working solution
1875  available yet. Thus, we either assume that Fenfire has a reliable technique for identifying individual entities, or  available yet. Thus, we either assume that Fenfire has a reliable technique for identifying individual entities, or
# Line 1888  performing searches). In the next subsec Line 1878  performing searches). In the next subsec
1878    
1879    
1880  \begin{itemize}  \begin{itemize}
1881  \item Data lookup with a given identifier of Storm scroll block.  \item Data lookup with a given identifier of Storm block.
1882  \begin{enumerate}  \begin{enumerate}
1883  \item Submit the data lookup using scroll block's identifier.  \item Submit the data lookup using block's identifier.
1884  \item Each peer forwards the data lookup to a closer peer which hosts the given scroll block identifier instructed by the overlay.  \item Each peer forwards the data lookup to a closer peer which hosts the given block identifier instructed by the overlay.
1885  \item The pointer peer returns value (e.g., the IP address of provider peer) to the query originator throughout the overlay.  \item The pointer peer returns value (e.g., the IP address of provider peer) to the query originator throughout the overlay.
1886  \item The query originator requests the provider peer to return the scroll block.  \item The query originator requests the provider peer to return the block.
1887  \end{enumerate}  \end{enumerate}
1888  \end{itemize}  \end{itemize}
1889    
1890    
1891  \begin{itemize}  \begin{itemize}
1892  \item Data lookup with a given pointer returning most recent scroll block.  \item Data lookup with a given pointer returning most recent block.
1893  \begin{enumerate}  \begin{enumerate}
1894  \item The query originator locally computes a hash over given pointer.  \item The query originator locally computes a hash over given pointer.
1895  \item Each peer forwards the data lookup to a closer peer which hosts the given hash of pointer instructed by the overlay.  \item Each peer forwards the data lookup to a closer peer which hosts the given hash of pointer instructed by the overlay.
1896  \item The pointer peer returns most recent pointer block's key-value pair (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes throughout the overlay.  \item The pointer peer returns most recent pointer block's key-value pair (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes throughout the overlay.
1897  \item The query originator requests the provider peer to return the scroll block.  \item The query originator requests the provider peer to return the block.
1898  \end{enumerate}  \end{enumerate}
1899  \end{itemize}  \end{itemize}
1900    
1901  \begin{itemize}  \begin{itemize}
1902  \item Data lookup with a given pointer returning scroll block(s) for a given date or time range.  \item Data lookup with a given pointer returning block(s) for a given date or time range.
1903  \begin{enumerate}  \begin{enumerate}
1904  \item The query originator locally computes a hash over given pointer.  \item The query originator locally computes a hash over given pointer.
1905  \item Each peer forwards the data lookup to a closer peer which hosts the given hash of pointer instructed by the overlay.  \item Each peer forwards the data lookup to a closer peer which hosts the given hash of pointer instructed by the overlay.
1906  \item Pointer peer returns pointer block's key-value pair(s) (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes throughout the overlay.  \item Pointer peer returns pointer block's key-value pair(s) (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes throughout the overlay.
1907  \item The query originator requests the provider peer to return the scroll block.  \item The query originator requests the provider peer to return the block.
1908  \end{enumerate}  \end{enumerate}
1909  \end{itemize}  \end{itemize}
1910    
# Line 1937  Perhaps the most biggest issue in Peer-t Line 1927  Perhaps the most biggest issue in Peer-t
1927  security technologies. For the Fenfire system, one security related problem occurs when a user wants to  security technologies. For the Fenfire system, one security related problem occurs when a user wants to
1928  perform a global data lookup with a given pointer; how the user is able to verify  perform a global data lookup with a given pointer; how the user is able to verify
1929  the correctness of the search results, i.e.,  how she or he knows which one is the  the correctness of the search results, i.e.,  how she or he knows which one is the
1930  correct Storm scroll block ? Another problem related to the Fenfire's  correct Storm block ? Another problem related to the Fenfire's
1931  security is that if a user downloads data from the network to local computer  security is that if a user downloads data from the network to local computer
1932  and after a network disconnection, user wants to verify \emph{off line} the  and after a network disconnection, user wants to verify \emph{off line} the
1933  authenticity of data. Finally, if a data lookup is performed by a user, but there is no reply  authenticity of data. Finally, if a data lookup is performed by a user, but there is no reply
# Line 1978  wide and intensive co-operation among re Line 1968  wide and intensive co-operation among re
1968    
1969  Our future work includes a support for searching transclusions and xanalogical  Our future work includes a support for searching transclusions and xanalogical
1970  links in Peer-to-Peer environment. Preliminary analysis have shown  links in Peer-to-Peer environment. Preliminary analysis have shown
1971  that these questions are rather different than locating scroll or pointer  that these questions are rather different than locating Storm blocks
1972  blocks from Peer-to-Peer environment. Techniques used in distributed  from Peer-to-Peer environment. Techniques used in distributed
1973  database systems may prove to be useful. Some fundamental results  database systems may prove to be useful. Some fundamental results
1974  regarding Peer-to-Peer and database systems have already been  regarding Peer-to-Peer and database systems have already been
1975  presented in \cite{gribble01p2pdatabase}.    presented in \cite{gribble01p2pdatabase}.  

Legend:
Removed from v.1.191  
changed lines
  Added in v.1.192

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26