/[gzz]/gzz/Documentation/misc/hemppah-progradu/masterthesis.tex
ViewVC logotype

Diff of /gzz/Documentation/misc/hemppah-progradu/masterthesis.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.94 by hemppah, Thu Feb 27 14:09:19 2003 UTC revision 1.95 by hemppah, Fri Feb 28 09:09:15 2003 UTC
# Line 362  peer's content, specifically $ps$, $P$ = Line 362  peer's content, specifically $ps$, $P$ =
362  where $ps$ = $summaryindex(provider(s)) \bigwedge (p = provider(s))$\}  where $ps$ = $summaryindex(provider(s)) \bigwedge (p = provider(s))$\}
363    
364    
   
   
365  \section{Tightly structured}  \section{Tightly structured}
366    
367  In recents months, several tightly structured overlays has been proposed.  In recents months, several tightly structured overlays has been proposed.
# Line 386  value of $n$ varies among approaches. Ag Line 384  value of $n$ varies among approaches. Ag
384  to implement identifier space.  to implement identifier space.
385    
386  To store data into tightly structured overlay, each application-specific  To store data into tightly structured overlay, each application-specific
387  key is \emph{mapped} by the overlay to a existing peer in the overlay. Thus, tightly  unique key (e.g., SHA-1 \cite{fips-sha-1}) is \emph{mapped} by the overlay to a
388  structured overlay assigns a subset of all possible keys to every participating peer.  existing peer in the overlay. Thus, tightly structured overlay assigns a subset of all
389  Furtermore, each peer in the structured overlay maintains a \emph{routing table},  possible keys to every participating peer. Furtermore, each peer in the structured
390  which consists of identifiers and IP addresses of other peers in the overlay.  overlay maintains a \emph{routing table}, which consists of identifiers and IP addresses
391  These are peer's neighbors in the overlay network. Figure \ref{fig:structured_hashing}  of other peers in the overlay. These are peer's neighbors in the overlay network.
392  illustrates the process of data to key mapping in tightly strucuted overlays.  Figure \ref{fig:structured_hashing} illustrates the process of data to key mapping in tightly strucuted overlays.
393    
394  \begin{figure}  \begin{figure}
395  \centering  \centering
# Line 410  previously schemes, each hop in the over Line 408  previously schemes, each hop in the over
408  between current peer working with query and the key which was  between current peer working with query and the key which was
409  looked up.  looked up.
410    
411    Skip Graphs and Swan employ a key space very similar to a tightly structured
412    overlay, but in which queries are routed  to \emph{identifiers}. In these systems
413    peer occupies several positions in the identifier space, one for each
414    application-specific key. The indirection of placing close keys in the
415    custody of a storing peer\footnote{Storing peer is the peer in the overlay which stores the
416    assigned keys.} keys is removed at the cost of each peer maintaining one
417    ''resource node'' in the overlay network for each resource item pair it publishes.
418    
419  Stoica et al. \cite{balakrishanarticle03lookupp2p} have listed  Stoica et al. \cite{balakrishanarticle03lookupp2p} have listed
420  four requirements for tightly structured overlays, which have to be  four requirements for tightly structured overlays, which have to be
421  addressed in order to perform data lookups in tightly structured overlays.  addressed in order to perform data lookups in tightly structured overlays.
# Line 435  data lookup. Viceroy maintains a butterf Line 441  data lookup. Viceroy maintains a butterf
441  only constant number of neighbor peers while providing $O(\log{n})$ data lookup  only constant number of neighbor peers while providing $O(\log{n})$ data lookup
442  efficiency. Koorde, recent modification of Chord, uses de Bruijn graphs to maintain  efficiency. Koorde, recent modification of Chord, uses de Bruijn graphs to maintain
443  local routing tables. Koorde requires each peer to have only about two links to other  local routing tables. Koorde requires each peer to have only about two links to other
444  peers to to provide $O(\log{n})$ performance. Peernet  peers to to provide $O(\log{n})$ performance.
445    
446  \begin{figure}  \begin{figure}
447  \centering  \centering
# Line 453  peers to to provide $O(\log{n})$ perform Line 459  peers to to provide $O(\log{n})$ perform
459  \end{figure}  \end{figure}
460    
461    
462  Chord example ?  There are three higher level abstractions which tightly structured overlays provide
463    \cite{zhao03api}. Each of these abstractions fulfil a storage layer in an overlay, but
464  abstraction: DHT, DOLR, multicast/anycast  they have semantical differences in the \emph{usage} of overlay. First, Distributed Hash
465    Table (DHT) (see e.g., \cite{dabek01widearea}, \cite{rowstron01storage}),  
466    implements three operations: \texttt{lookup(key)}, \texttt{remove(key)} and
467  -service is data block, node/peer is a physical computer  \texttt{insert(key)}. As the name suggests, DHT implements the same functionality
468  -*servers* self-organize towards a lookup network  as a regular hashtable, by storing the mapping between a key and a value. DHT's
469  -DHTs can be thought as a 'structured overlay random graphs'  \emph{interface} is generic; values can be any size and type. Figure \ref{fig:Structured_lookup_using_DHT_model}
470  -There is a explicit metric space in every DHT. Term 'closeness' differs between existing DHTs: it can be XOR/numerical/eucklidean difference between identifiers  shows the DHT abstraction of tightly structured overlay. Second, Decentralized
471  -Resource can *be* a resource, or a *pointer* to a resource  Object Location (DOLR) (see e.g., \cite{kubiatowicz00oceanstore}, \cite{iyer02squirrel}) is distributed
472  -a service request is routed towards service based on node's local knowledge  directory service. DOLR stores \emph{pointers} to where data items are stored
473  -identifiers are expected to be distributed uniformly  throughout the overlay. DOLR's main operations are \texttt{publish(key)},
474  -DHTs require a knowledge of identifier space's size initially  \texttt{removePublished(key)} and \texttt{sendToObject(key)}. The key
475  -resembles a balanced tree structure  difference between DHT and DOLR abstraction is that DOLR routes overlay's messages
476  SWAN and Skip Graphs  to nearest available peer, hosting a specific data item. This form of locality
477  -in this scheme, node = service  is not supported by DHT. Finally, tightly structured overlay can be used for
478  -*key-value pairs* self-organise towards a lookup network  scalable group multicast/anycast operations (CAST) (see e.g., \cite{zhuang01bayeux}).
479  -There is a explicit metric space. Term 'closeness' differs between existing DHTs: it can be XOR/numerical/eucklidean difference between identifiers  The basic operation are \texttt{join(groupIdentifier)}, \texttt{leave(groupIdentifier)},
480  -Resource can *be* a resource, or a *pointer* to a resource  \texttt{multicast(message, groupIdentifier)},  \texttt{anycast(message, groupIdentifier)}.
481  -a service request is routed towards service based on node's local knowledge  Participating peers may join and leave a group and send multicast messages to
482  -services can be hosted locally (opposite to DHTs)  the group, or anycast message to a specific member of the group. DOLR's and CAST's
483  -identifiers are expected to be distributed uniformly  have much in common.For instance, they both use network proximity techniques
484  -system does find the service, if it exists  to optimize their operation in the overlay. Figure \ref{fig:Strucutred_lookup_using_DOLR_model}
485     presents basic operation of DOLR abstraction.
486    
487  +fast routing (aka searching)  \begin{figure}
488  %+scalable (10^9 users, 10^14 data items)  \centering
489  +robust  \includegraphics[width=10cm, height=8cm]{DHT_lookup.eps}
490  +little network traffic  \caption{DHT abstraction of tightly structured overley}
491  -own resources are mapped into the network (not necessary!!)  \label{fig:Structured_lookup_using_DHT_model}
492  -keyword/fuzzy search not possible yet  \end{figure}
 -routing/query hotspots  
 -ASSUME THAT ALL NODES HAVE IDENTICAL CABABILITIES! However, in real life, p2p enviroment is extremely heterogeneous!  
   
 -the basic idea behind many DHTs is the fact that they perform operations in a binary-like tree  
 -more space/node --> the search arity is higher --> k is higher in k-ary trees  
 -DHTs' performance efficiency is derived from these tree based operations (e.g. split to half the previous scope)  
   
 Skip graphs  
 +fast routing (aka searching)  
 +scalable  
 +little network traffic (however, more than DHTs)  
 +support *data* locality, opposite to DHTs where data is spread out evenly, destroying locality. In skip graphs, hashing is not requires, only *keys* matters (which we already have)  
 +support for partial repair mechanism/self-stabilization (DHTs lack of repair mechanism/self-stabilization)  
 +the most robust algorithm up to date, tolerance of adversial faults (DHTs support only of random faults)  
 +adaptive: doesn't need to know keyspace size before hand, instead of many DHTs, which require a priori knowledge about the size of the system or its keyspace  
 +support for range queries, e.g. natural for version control: 'find latest news from yesterday', 'find largest key < news:12/12/2002', 'find all related objects nearby'  
 (in DHTs this is a open problem)  
 +There are not hotspots in skip graphs (query/routing hotsports)  
 -not network/geographical locality  
 -for our purposes, Skip Graps could be used (sensible) only for block IDs, not for URNs: in Skip Graps, there have to be total order for data elements --> for URNs, there cannot be a total order  
 -for range queries, we would have to use local sensitive hashing, *if* we want fetch blocks like 'next block = i + 1'. SHA-1 is a (secure) random hash of content (it won't help for us in this case)  
493    
 Peernet  
 -Peernet is a p2p based *network layer* for large networks  
 -Peernet makes an explicit distinction between node identity and address  
 -requires no manual configuration (plug-and-play)  
 -can support wireless or wireline infrastructure  
 -log(n) routing table, log(n) search    
494    
495  \cite{aspnes02faultrouting}  \begin{figure}
496  \cite{ratnasamy02ght}  \centering
497  \cite{236713}  \includegraphics[width=10cm, height=8cm]{DOLR_lookup.eps}
498  \cite{258660}  \caption{DOLR abstraction of tightly structured overley}
499    \label{fig:Strucutred_lookup_using_DOLR_model}
500  \cite{fips-sha-1}  \end{figure}
501    
502    
503  \subsection{Formal definition}  \subsection{Formal definition}
# Line 537  which means that resources that a peer p Line 516  which means that resources that a peer p
516  Every $p$ has neighbor(s), named as $neighbor$, which are $P$ = \{$p \in P: \exists neighbor$,  Every $p$ has neighbor(s), named as $neighbor$, which are $P$ = \{$p \in P: \exists neighbor$,
517  where $difference(p,p_neighbor)= close$, and  $close$ is minimal difference $d$ in $(IS,d)$\}.  where $difference(p,p_neighbor)= close$, and  $close$ is minimal difference $d$ in $(IS,d)$\}.
518    
 \subsection{Protocols}  
   
   
   
   
   
   
   
   
 \cite{freedman02trie}  
   
 \cite{plaxton97accessingnearby}  
   
   
   
   
 \cite{78977}  
   
   
   
   
   
   
 \cite{garciamolina03sil}  
   
 \cite{rowston03controlloingreliability}  
   
   
   
 \cite{byers03dhtbalancing}  
   
 \cite{pias03lighthouse}  
   
   
 \cite{debruijn46graph}  
   
   
 \subsection{Distributed Hash Tables}  
   
 \subsection{Decentralized Object Location and Routing Networks}  
519    
 \subsection{Group multicast any anycast}  
520    
521    \section{Summary}
522    
523  \begin{figure}  Measures:
 \centering  
 \includegraphics[width=10cm, height=8cm]{DHT_lookup.eps}  
 \caption{Kademlia's lookup process}  
 \label{fig:Structured lookup using DHT model}  
 \end{figure}  
524    
525    degree:
526    
527  \begin{figure}  hop count:
 \centering  
 \includegraphics[width=10cm, height=8cm]{DOLR_lookup.eps}  
 \caption{Kademlia's lookup process}  
 \label{fig:Strucutred lookup using DOLR model}  
 \end{figure}  
528    
529  \cite{Gribble:2000:SDD}  fault-tolerance
530    
531  \cite{dabek01widearea}  maintenance overhead
532    
533  \cite{iyer02squirrel}  load balance
534    
 \cite{harrisoncircle}  
535    
536  \cite{rowstron01storage}  \subsection{Differences}
537    
 -CFS splits files into blocks (<50Kb), PAST distributed whole files  
538    
 \cite{kubiatowicz00oceanstore}  
539    
540    \scriptsize
541    \begin{longtable}{|l|l|l|}
542    
543    
544    \hline
545    \multicolumn{1}{|c|}{\textbf{Property}} &
546    \multicolumn{1}{c|}{\textbf{Unstructured}} &
547    \multicolumn{1}{c|}{\textbf{Structured}}  
548    
549    \\ \hline
550    \endfirsthead
551    
552    \multicolumn{3}{c}%
553    {{\tablename\ \thetable{} -- continued from previous page}} \\
554    \hline
555    \multicolumn{1}{|c|}{\textbf{Property}} &
556    \multicolumn{1}{c|}{\textbf{Loosely structured}} &
557    \multicolumn{1}{c|}{\textbf{Tightly structured}}
558    \\ \hline
559    \endhead
560    
561    \endfoot
562    
 \section{Summary}  
563    
 Measures:  
564    
565  degree:  \parbox{90pt}{Queries} &
566    \parbox{100pt}{Uncontrolled} &
567    \parbox{100pt}{Controlled}  
568    \\ \hline
569    
570  hop count:  \parbox{90pt}{A way for performing queries} &
571    \parbox{100pt}{Keywords} &
572    \parbox{100pt}{Exact keys}
573    \\ \hline
574              
575    \parbox{90pt}{Query traffic} &
576    \parbox{100pt}{$O(n)/O(n^{2})$}  &
577    \parbox{100pt}{$O(1)/O(\log{n})$}
578    \\ \hline
579    
580  fault-tolerance  \parbox{90pt}{Guaranteed data lookup} &
581    \parbox{100pt}{Not necessarily} &
582    \parbox{100pt}{Yes}
583    \\ \hline
584    
585  maintenance overhead  \parbox{90pt}{Overlay's structure} &
586    \parbox{100pt}{Uncontrolled and ad hoc}  &
587    \parbox{100pt}{Controlled and structured}
588    \\ \hline
589                      
590    \parbox{90pt}{Max. number of nodes} &
591    \parbox{100pt}{Millions} &
592    \parbox{100pt}{Billions}
593    \\ \hline
594                            
595    \parbox{90pt}{Data placement} &
596    \parbox{100pt}{Local} &
597    \parbox{100pt}{Not local}
598    \\ \hline
599                      
600    \parbox{90pt}{Support for heterogeneity} &
601    \parbox{100pt}{Yes} &
602    \parbox{100pt}{No}
603    \\ \hline
604            
605    \parbox{90pt}{Support for locality} &
606    \parbox{100pt}{Yes} &
607    \parbox{100pt}{Partial}
608    \\ \hline
609              
610    \parbox{90pt}{Possibility for routing hotspots} &
611    \parbox{100pt}{No} &
612    \parbox{100pt}{Yes}
613    \\ \hline
614            
615    \parbox{90pt}{Design/Implementation complexity} &
616    \parbox{100pt}{Low} &
617    \parbox{100pt}{High}
618    \\ \hline
619    
620  load balance  \parbox{90pt}{Fault-tolerant} &
621    \parbox{100pt}{High} &
622    \parbox{100pt}{High}
623    \\ \hline
624    
625    
626  \subsection{Differences}  \caption{Comparison of loosely structured and tighly structured approaches}
627    \label{table_comparison_approach}
628    
629    
630    \end{longtable}
631    \normalsize
632    
633    
634    
635  \subsection{Protocols}  \subsection{Algorithms}
636    
637  \scriptsize  \scriptsize
638  \begin{longtable}{|l|c|c|c|c|l|}  \begin{longtable}{|l|c|c|c|c|l|}
# Line 828  network connections required to maintain Line 824  network connections required to maintain
824  \normalsize  \normalsize
825    
826    
827    -service is data block, node/peer is a physical computer
828    -*servers* self-organize towards a lookup network
829    -DHTs can be thought as a 'structured overlay random graphs'
830    -There is a explicit metric space in every DHT. Term 'closeness' differs between existing DHTs: it can be XOR/numerical/eucklidean difference between identifiers
831    -Resource can *be* a resource, or a *pointer* to a resource
832    -a service request is routed towards service based on node's local knowledge
833    -identifiers are expected to be distributed uniformly
834    -DHTs require a knowledge of identifier space's size initially
835    -resembles a balanced tree structure
836    SWAN and Skip Graphs
837    -in this scheme, node = service
838    -*key-value pairs* self-organise towards a lookup network
839    -There is a explicit metric space. Term 'closeness' differs between existing DHTs: it can be XOR/numerical/eucklidean difference between identifiers
840    -Resource can *be* a resource, or a *pointer* to a resource
841    -a service request is routed towards service based on node's local knowledge
842    -services can be hosted locally (opposite to DHTs)
843    -identifiers are expected to be distributed uniformly
844    -system does find the service, if it exists
845    
846    
847    +fast routing (aka searching)
848    %+scalable (10^9 users, 10^14 data items)
849    +robust
850    +little network traffic
851    -own resources are mapped into the network (not necessary!!)
852    -keyword/fuzzy search not possible yet
853    -routing/query hotspots
854    -ASSUME THAT ALL NODES HAVE IDENTICAL CABABILITIES! However, in real life, p2p enviroment is extremely heterogeneous!
855    
856    -the basic idea behind many DHTs is the fact that they perform operations in a binary-like tree
857    -more space/node --> the search arity is higher --> k is higher in k-ary trees
858    -DHTs' performance efficiency is derived from these tree based operations (e.g. split to half the previous scope)
859    
860    Skip graphs
861    +fast routing (aka searching)
862    +scalable
863    +little network traffic (however, more than DHTs)
864    +support *data* locality, opposite to DHTs where data is spread out evenly, destroying locality. In skip graphs, hashing is not requires, only *keys* matters (which we already have)
865    +support for partial repair mechanism/self-stabilization (DHTs lack of repair mechanism/self-stabilization)
866    +the most robust algorithm up to date, tolerance of adversial faults (DHTs support only of random faults)
867    +adaptive: doesn't need to know keyspace size before hand, instead of many DHTs, which require a priori knowledge about the size of the system or its keyspace
868    +support for range queries, e.g. natural for version control: 'find latest news from yesterday', 'find largest key < news:12/12/2002', 'find all related objects nearby'
869    (in DHTs this is a open problem)
870    +There are not hotspots in skip graphs (query/routing hotsports)
871    -not network/geographical locality
872    -for our purposes, Skip Graps could be used (sensible) only for block IDs, not for URNs: in Skip Graps, there have to be total order for data elements --> for URNs, there cannot be a total order
873    -for range queries, we would have to use local sensitive hashing, *if* we want fetch blocks like 'next block = i + 1'. SHA-1 is a (secure) random hash of content (it won't help for us in this case)
874    
875    Peernet
876    -Peernet is a p2p based *network layer* for large networks
877    -Peernet makes an explicit distinction between node identity and address
878    -requires no manual configuration (plug-and-play)
879    -can support wireless or wireline infrastructure
880    -log(n) routing table, log(n) search  
881    
882    \cite{aspnes02faultrouting}
883    \cite{ratnasamy02ght}
884    \cite{236713}
885    \cite{258660}
886    
887    \cite{freedman02trie}
888    
889    \cite{plaxton97accessingnearby}
890    
891    
892    
893    
894    \cite{78977}
895    
896    
897    
898    
899    
900    
901    \cite{garciamolina03sil}
902    
903    \cite{rowston03controlloingreliability}
904    
905    
906    
907    \cite{byers03dhtbalancing}
908    
909    \cite{pias03lighthouse}
910    
911    
912    \cite{debruijn46graph}
913    
914    
915    
916    \cite{Gribble:2000:SDD}
917    
918    
919    
920    \cite{harrisoncircle}
921    
922    
923    
924    -CFS splits files into blocks (<50Kb), PAST distributed whole files
925    
926    
927    
928    
# Line 1753  which are essential to xanalogical model Line 1848  which are essential to xanalogical model
1848  Table \ref{table_comparison_approach} lists the key feature of both approaches.  Table \ref{table_comparison_approach} lists the key feature of both approaches.
1849    
1850    
 \scriptsize  
 \begin{longtable}{|l|l|l|}  
   
   
 \hline  
 \multicolumn{1}{|c|}{\textbf{Property}} &  
 \multicolumn{1}{c|}{\textbf{Unstructured}} &  
 \multicolumn{1}{c|}{\textbf{Structured}}    
   
 \\ \hline  
 \endfirsthead  
1851    
 \multicolumn{3}{c}%  
 {{\tablename\ \thetable{} -- continued from previous page}} \\  
 \hline  
 \multicolumn{1}{|c|}{\textbf{Property}} &  
 \multicolumn{1}{c|}{\textbf{Loosely structured}} &  
 \multicolumn{1}{c|}{\textbf{Tightly structured}}  
 \\ \hline  
 \endhead  
   
 \endfoot  
   
   
   
 \parbox{90pt}{Queries} &  
 \parbox{100pt}{Uncontrolled} &  
 \parbox{100pt}{Controlled}    
 \\ \hline  
   
 \parbox{90pt}{A way for performing queries} &  
 \parbox{100pt}{Keywords} &  
 \parbox{100pt}{Exact keys}  
 \\ \hline  
             
 \parbox{90pt}{Query traffic} &  
 \parbox{100pt}{$O(n)/O(n^{2})$}  &  
 \parbox{100pt}{$O(1)/O(\log{n})$}  
 \\ \hline  
   
 \parbox{90pt}{Guaranteed data lookup} &  
 \parbox{100pt}{Not necessarily} &  
 \parbox{100pt}{Yes}  
 \\ \hline  
   
 \parbox{90pt}{Overlay's structure} &  
 \parbox{100pt}{Uncontrolled and ad hoc}  &  
 \parbox{100pt}{Controlled and structured}  
 \\ \hline  
                     
 \parbox{90pt}{Max. number of nodes} &  
 \parbox{100pt}{Millions} &  
 \parbox{100pt}{Billions}  
 \\ \hline  
                           
 \parbox{90pt}{Data placement} &  
 \parbox{100pt}{Local} &  
 \parbox{100pt}{Not local}  
 \\ \hline  
                     
 \parbox{90pt}{Support for heterogeneity} &  
 \parbox{100pt}{Yes} &  
 \parbox{100pt}{No}  
 \\ \hline  
           
 \parbox{90pt}{Support for locality} &  
 \parbox{100pt}{Yes} &  
 \parbox{100pt}{Partial}  
 \\ \hline  
             
 \parbox{90pt}{Possibility for routing hotspots} &  
 \parbox{100pt}{No} &  
 \parbox{100pt}{Yes}  
 \\ \hline  
           
 \parbox{90pt}{Design/Implementation complexity} &  
 \parbox{100pt}{Low} &  
 \parbox{100pt}{High}  
 \\ \hline  
   
 \parbox{90pt}{Fault-tolerant} &  
 \parbox{100pt}{High} &  
 \parbox{100pt}{High}  
 \\ \hline  
   
   
 \caption{Comparison of loosely structured and tighly structured approaches}  
 \label{table_comparison_approach}  
   
   
 \end{longtable}  
 \normalsize  
1852    
1853    
1854    

Legend:
Removed from v.1.94  
changed lines
  Added in v.1.95

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26