/[gzz]/gzz/Documentation/misc/hemppah-progradu/masterthesis.tex
ViewVC logotype

Diff of /gzz/Documentation/misc/hemppah-progradu/masterthesis.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.166 by hemppah, Thu Mar 20 13:07:04 2003 UTC revision 1.167 by hemppah, Thu Mar 20 14:47:24 2003 UTC
# Line 860  Possible solution against this attack is Line 860  Possible solution against this attack is
860  information from multiple entities and trust on the majority's opinion. This method requires more messages to be  information from multiple entities and trust on the majority's opinion. This method requires more messages to be
861  sent to the network while increasing the system load. However, if the Spam attack is combined with the Sybil attack, obviously  sent to the network while increasing the system load. However, if the Spam attack is combined with the Sybil attack, obviously
862  the previously mentioned solution doesn't work. Naor et al. \cite{naor03simpledht} have proposed a partial solution against Spam attack  the previously mentioned solution doesn't work. Naor et al. \cite{naor03simpledht} have proposed a partial solution against Spam attack
863  in \emph{faulty} peer environment (not hostile).  in a \emph{faulty} peer environment (not hostile).
864    
865  Traditional overloading of targeted peers is the best known form of distributed Denial of Service attack (DDoS) (see, e.g., \cite{372148}).  Traditional overloading of targeted peers is the best known form of distributed Denial of Service attack (DDoS) (see, e.g., \cite{372148}).
866  For example, a hostile entity can attempt to burden targeted peers with garbage network packets. As an implication, peers may act  For example, a hostile entity can attempt to burden targeted peers with garbage network packets. As an implication, peers may act
# Line 889  Quite recently, widely used Public Key I Line 889  Quite recently, widely used Public Key I
889  systems \cite{rivest96sdsi}, \cite{spkiworkinggroup}. PKI is a reliable technology for securing  systems \cite{rivest96sdsi}, \cite{spkiworkinggroup}. PKI is a reliable technology for securing
890  data in rather \emph{static} computing systems, such as the Internet. However, in Peer-to-Peer  data in rather \emph{static} computing systems, such as the Internet. However, in Peer-to-Peer
891  networks, the problem of key-based security mechanism is the maintenance of the keys as participating  networks, the problem of key-based security mechanism is the maintenance of the keys as participating
892  peers constantly join and leave the system. These include revocation of keys and new key distribution in hostile  peers constantly join and leave the system. These include the revocation of keys and the distribution of
893  environment.  new keys in a hostile environment.
894    
895  ConChord \cite{ajmani02conchord} is the first Peer-to-Peer system which has a support for PKI based  ConChord \cite{ajmani02conchord} is the first Peer-to-Peer system which has a support for PKI based
896  security infrastructure. Still, however, ConChord \cite{ajmani02conchord} is in early phase of development and lacks  security infrastructure. Still, however, ConChord \cite{ajmani02conchord} is in early phase of development and lacks
# Line 899  important features of PKI to be fully us Line 899  important features of PKI to be fully us
899  Peer-to-Peer systems, in which hierarchy is intentionally missing.  Peer-to-Peer systems, in which hierarchy is intentionally missing.
900    
901  For data integrity, on the other hand, there are few working solutions. Cryptographic content hashes  For data integrity, on the other hand, there are few working solutions. Cryptographic content hashes
902  \cite{fips-sha-1}, their variations \cite{merkle87hashtree} and implementation techniques \cite{mohr02thex},  \cite{fips-sha-1}, their variations \cite{merkle87hashtree} and implementation techniques \cite{mohr02thex}
903  are efficient and reliable methods for identifying the integrity of data in Peer-to-Peer systems. One  are efficient and reliable methods for identifying the integrity of data in Peer-to-Peer systems. One
904  possible application of cryptographic content hashes may be in peer identifier creation process, in which  possible application of cryptographic content hashes may be in the creation process of peer identifier, in which
905  the IP address of a peer can be verified by the other peer. This is one form of \emph{self-certifying data}.  the IP address of a peer can be verified by the other peer. This is one form of \emph{self-certifying data}.
906    
907    
# Line 925  Anonymity outside of Peer-to-Peer contex Line 925  Anonymity outside of Peer-to-Peer contex
925  Obviously, existance of several types of anonymity often conflicts with other key properties of  Obviously, existance of several types of anonymity often conflicts with other key properties of
926  Peer-to-Peer systems. Let us consider anonymity and efficient data lookup. In efficient data lookup, we must know  Peer-to-Peer systems. Let us consider anonymity and efficient data lookup. In efficient data lookup, we must know
927  the peers responsible for given data. Of course, when we know the peers responsible  the peers responsible for given data. Of course, when we know the peers responsible
928  for the data, the anonymity of peer is lost. Fortunately, there are partial solutions to previously  for the data, the anonymity of peer is lost. Fortunately, there are partial solutions to these kinds of
929  mentioned situations, such as pseudonymity which is a partial form of anonymity. For instance, pseudonymity can be used for  situations, such as pseudonymity which is a partial form of anonymity. For instance, pseudonymity can be used for
930  addressing peer-anonymity by providing anonymous-like identifiers to peers (e.g., peer identifiers of a tightly  addressing peer-anonymity by providing anonymous-like identifiers to peers (e.g., peer identifiers of a tightly
931  structured system).  structured system).
932    
933  Anonymity is widely used in Peer-to-Peer system in which data publication and non-censorship are important properties  Anonymity is widely used in a Peer-to-Peer system in which data publication and non-censorship are important properties
934  of the system. These include  of the system. These include
935  Freenet \cite{clarke00freenet}, Publius \cite{pub00}, Free Haven \cite{dingledine00free}, Crowds \cite{reiter98crowds},  Freenet \cite{clarke00freenet}, Publius \cite{pub00}, Free Haven \cite{dingledine00free}, Crowds \cite{reiter98crowds},
936  Tangler \cite{502002} and upcoming Mnet \cite{mneturl}. Forwarding proxies are used in Freenet, Crowds and  Tangler \cite{502002} and upcoming Mnet \cite{mneturl}. Forwarding proxies are used in Freenet, Crowds and
# Line 940  sharing methods to split data into fragm Line 940  sharing methods to split data into fragm
940  of anonymity.  of anonymity.
941    
942  Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no  Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no
943  such system which is able to provide all kinds of anonymity as listed above. Specifically, the conflicts  such a system which is able to provide all kinds of anonymity as listed above. Specifically, the conflicts
944  between anonymity and other Peer-to-Peer system properties require more research work.  between anonymity and other Peer-to-Peer system properties require more research work.
945    
946    
# Line 970  of peer identifiers that cannot be contr Line 970  of peer identifiers that cannot be contr
970    
971  Naturally, centralized authorities could be used for the assignment of peer identifiers, but they may not be suitable  Naturally, centralized authorities could be used for the assignment of peer identifiers, but they may not be suitable
972  for ad hoc Peer-to-Peer environrment and have property of single point of failure. Moreover, distributed peer  for ad hoc Peer-to-Peer environrment and have property of single point of failure. Moreover, distributed peer
973  identification assignment can be problematic as long as Sybil attack \cite{douceur02sybil} remains unsolved.  identification assignment can be problematic as long as the Sybil attack \cite{douceur02sybil} remains unsolved.
974  However, there are some partial solutions for controlling the rate at which hostile entity is able to obtain peer  However, there are some partial solutions for controlling the \emph{rate} at which hostile entity is able to obtain peer
975  identifier, such as crypto-based puzzles \cite{juels99clientpuzzles}.  identifier, such as crypto-based puzzles \cite{juels99clientpuzzles}.
976    
 In the end, none of these problems solutions are able to identify hostile entities safely.  
   
   
977  \subsection{Secure query routing}  \subsection{Secure query routing}
978    
979  Much work has been done on secure routing, especially related to tightly structured systems. In  Much work has been done on secure routing, especially related to tightly structured systems. In
980  \cite{castro02securitystructured} and \cite{castro02securerouting}, authors suggest the usage  \cite{castro02securitystructured} and \cite{castro02securerouting}, authors suggest the use
981  of constrained routing tables and diverse routes, and the detection of faults during query routing.  of constrained routing tables and diverse routes, and the detection of faults during data lookup routing.
982  Additionally, authors present in \cite{castro02securerouting} an important aspect of the tightly structured approach with regard  Additionally, authors present in \cite{castro02securerouting} an important aspect of the tightly structured approach with regard
983  to fault-tolerant query routing: the probability of routing successfully between to arbitrary  to fault tolerant query routing: the probability of routing successfully between to arbitrary
984  correct peers, when a fraction $f$ of the other peers are faulty or hostile, is only $(1-f)^{h-1}$, where  correct peers, when a fraction $f$ of the other peers are faulty or hostile, is only $(1-f)^{h-1}$, where
985  $h$ is the number of hops in the overlay.  $h$ is the number of hops in the overlay.
986    
987  Sit and Morris \cite{sit02securitycons} discuss the possibility of allowing query originator  Sit and Morris \cite{sit02securitycons} discuss the possibility of allowing the query originator
988  to observe lookup progress and cross-check routing tables using random queries. However, their  to observe lookup progress and cross-check routing tables using random queries. However, their
989   approach is not very efficient, since proposals create a lot of additional network traffic when   approach is not very efficient, since this method creates lot of additional network traffic when
990  in function.  in function.
991    
992  Additionally, Lynch et al. \cite{lynch02atomicdataaccess} propose a solution to secure routing table  Additionally, Lynch et al. \cite{lynch02atomicdataaccess} propose a solution for secure routing table
993  maintenance, but their solution seems to have two major problems \cite{castro02securitystructured}. First,  maintenance, but their solution seems to have two major problems \cite{castro02securitystructured}. First,
994  the solution is very expensive even without faulty or hostile entities. Second, each group of replicas  the solution is very expensive even without faulty or hostile entities. Second, each group of replicas
995  in their solution must have less than 1/3 of its peers faulty. Thus, this feature results in a low  in their solution must have less than 1/3 of its peers faulty. Thus, this feature results in a low
996  probability of successful routing.  probability of successful routing.
997    
998  Aspnes et al. in \cite{aspnes02faultrouting} and Kaashoek et al. in \cite{kaashoek03koorde} formally  Aspnes et al. in \cite{aspnes02faultrouting} and Kaashoek et al. in \cite{kaashoek03koorde} formally
999  prove the lower and upper bounds for space requirements of locating a specific data item in a  prove the lower and upper bounds for the space requirements of locating a specific data item in a
1000  Peer-to-Peer system. They show that to provide high degree of fault tolerance and efficiency in the system, each  Peer-to-Peer system. They show that to provide high degree of fault tolerance and efficiency in the system, each
1001  participating peer must maintain average of $O(\log{n})$ neighbors.  participating peer must maintain average of $O(\log{n})$ neighbors.
1002    
# Line 1011  must maintain information of $O(\log^3{n Line 1008  must maintain information of $O(\log^3{n
1008    
1009  Finally, Ratnasamy and Gavoille \cite{ratnasamy02routing, gavoille01routing} list several open problems  Finally, Ratnasamy and Gavoille \cite{ratnasamy02routing, gavoille01routing} list several open problems
1010  regarding routing in distributed networks. Obviously, more research is required in order to provide secure  regarding routing in distributed networks. Obviously, more research is required in order to provide secure
1011  data lookup routing possible in Peer-to-Peer networks.  data lookup routing in Peer-to-Peer networks.
1012    
1013    
1014  \subsection{Other security threats}  \subsection{Other security threats}
1015    
1016  Ross Lee Graham lists several external threats against Peer-to-Peer networks \cite{grahamp2psecurity}. Most important,  Ross Lee Graham lists several external threats against Peer-to-Peer networks \cite{grahamp2psecurity}. Most important,
1017  the list includes viruses and trojans. Currently, there are not even partial solutions  the list includes viruses and trojans. Currently, there are not even partial solutions
1018  to the problems mentioned above. General robustness properties of a Peer-to-Peer system is able to  to the problems mentioned above.  The reason for this is that there are no experience about these kinds of
1019  deal with software failures and hostile attacks, but fault tolerance against external threats is unknown.  attacks. Possible solution would be a distributed anti-virus software, but much more intensive research is required until
 The reason for this is that there are no experience on these kinds of attacks. Possible solution  
 would be distributed anti-virus software, but much more intensive research is required until  
1020  this kind of solution would be applicable.  this kind of solution would be applicable.
1021    
1022    
# Line 1032  In this section, we discuss performance Line 1027  In this section, we discuss performance
1027  \subsection{Efficient data lookup}  \subsection{Efficient data lookup}
1028    
1029  The most intensive research in Peer-to-Peer domain has been focused on efficient data lookup methods,  The most intensive research in Peer-to-Peer domain has been focused on efficient data lookup methods,
1030  especially with the loosely structured approach. In addition to the ''super-peer'' method presented in chapter  especially with the loosely structured approach. In iterative deepening
 2, there has been other improvements also. In iterative deepening  
1031  \cite{yang02improvingsearch}, multiple BFS searches are initiated  \cite{yang02improvingsearch}, multiple BFS searches are initiated
1032  with successively larger TTL depth limits, until either the query is satisfied,  with successively larger TTL depth limits, until either the query is satisfied,
1033  or the maximum depth $D$ has been reached. To perform a data lookup, query  or the maximum depth $D$ has been reached. To perform a data lookup, the query
1034  originator starts the data lookup with a small TTL value. If the search is not successful,  originator starts the data lookup with a small TTL value. If the search is not successful,
1035  the query originator increases the TTL value and performs another data lookup. This  the query originator increases the TTL value and performs another data lookup. This
1036  process is repeated until the desired data is found or the maximum depth $D$  process is repeated until the desired data is found or the maximum depth $D$
1037  has been reached. Expanding ring, proposed by Shenker et al. in \cite{lv02searchreplication},  has been reached. Expanding ring, proposed by Shenker et al. in \cite{lv02searchreplication},
1038  is similar to iterative deepening technique. With these techniques, search  is similar to the iterative deepening technique. With these techniques, searches
1039  may not be fast when desired data item requires many consecutive flooding rounds.  may not be fast when desired data item requires several consecutive flooding rounds.
1040    
1041  Directed BFS \cite{yang02improvingsearch} optimizes the original  Directed BFS \cite{yang02improvingsearch} optimizes the original
1042  BFS in a way that a peer selects neighbors with many quality results are reached in the past,  BFS in a way that a peer selects the neighbors which have provided many quality results in the past,
1043  thereby maintaining the quality of costs and decreasing the amount  thereby maintaining the quality of costs and decreasing the amount
1044  of messages sent to network. Alpine \cite{alpineurl} and NeuroGrid \cite{joseph02neurogrid}  of messages sent to network. Alpine \cite{alpineurl} and NeuroGrid \cite{joseph02neurogrid}
1045  are Peer-to-Peer systems which use somewhat similar method when performing data lookups.  are Peer-to-Peer systems which use somewhat similar method when performing data lookups.
# Line 1053  are Peer-to-Peer systems which use somew Line 1047  are Peer-to-Peer systems which use somew
1047  Local indices \cite{yang02improvingsearch} is a variation of active caching.  Local indices \cite{yang02improvingsearch} is a variation of active caching.
1048  In this scheme, each peer maintains an index over the data of all peers within  In this scheme, each peer maintains an index over the data of all peers within
1049  $h$ hops of itself, where $h$ is a system-wide variable, called radius of the  $h$ hops of itself, where $h$ is a system-wide variable, called radius of the
1050  index\footnote{In normal BFS case, the value of $h$ is 0, as peer only has index  index\footnote{In the normal BFS case, the value of $h$ is 0, as a peer only has index
1051  over its local content.}. Mutual index caching architecture, as proposed in  over its local content.}. Mutual index caching architecture, as proposed in
1052  \cite{osokine02distnetworks}, is one variation of local indices technique.  \cite{osokine02distnetworks}, is a variation of local indices technique.
1053    
1054  In random walk approach \cite{lv02searchreplication}, a peer forwards query to a  In the random walk approach \cite{lv02searchreplication}, a peer forwards query to a
1055  randomly selected neighbor. The basic random walk approach  randomly selected neighbor. The basic random walk approach
1056  has a poor response time but it doesn't generate as much network traffic as  has a poor response time but it doesn't generate as much network traffic as
1057  the original BFS. As suggested in \cite{lv02searchreplication}, the  the original BFS. As suggested in \cite{lv02searchreplication}, the
1058  random walk approach can be made more effective by introducing  random walk approach can be made more effective by introducing
1059  multiple ''walkers''. Freenet \cite{clarke00freenet} uses  multiple ''walkers''. Freenet \cite{clarke00freenet} uses
1060  random walk searches in query lookups. Indeed, Freenet's query resembles  random walk searches in data lookups. Freenet's data lookup model resembles
1061  Depth-First-Search (DFS) and peers' routing tables are dynamically built  Depth-First-Search (DFS) and peers' routing tables are dynamically built
1062  using caching. This is an outcome of Freenet's main design principles, anonymity.  using caching. This is an outcome of Freenet's main design principles, anonymity.
1063  Another property of the Freenet's data lookup model is that  Another property of the Freenet's data lookup model is that
1064  it adapts well with varying usage patterns. Improvements to Freenet's data lookup using  it adapts well with varying usage patterns. Improvements to Freenet's data lookup using
1065  the ''small-world phenomenon'' has been proposed by Zhang et al. \cite{zhang02using}.  the ''small-world phenomenon'' have been proposed by Zhang et al. \cite{zhang02using}.
1066    
1067  Since tightly structured systems have an efficient data lookup at the application level overlay,  Since tightly structured systems have an efficient data lookup at the application level overlay,
1068  current research efforts are focused on a proximity based data lookup. In the proximity based data lookup,  current research efforts are focused on the proximity-based data lookup. In the proximity-based data lookup,
1069  peers try to choose entries of routing-tables referring to other peers that are \emph{nearby} in the  peers try to choose entries of routing-tables referring to other peers that are \emph{nearby} in the
1070  underlying network. In this way, tightly structured systems are able to decrease actual  underlying network. In this way, tightly structured systems are able to decrease actual
1071  lookup \emph{latency}. CAN \cite{ratnasamy01can}, Kademlia \cite{maymounkov02kademlia},  lookup \emph{latency}. CAN \cite{ratnasamy01can}, Kademlia \cite{maymounkov02kademlia},
1072  Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry} have advanced heuristics for  Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry} have advanced heuristics for
1073  the proximity based routing. Additionally, most recent version of Chord uses proximity based  the proximity-based routing. Additionally, most recent version of Chord uses proximity-based
1074  routing inspired by Karger and Ruhl \cite{karger02findingnearest}. SkipNet \cite{harvey03skipnet1}  routing, inspired by Karger and Ruhl \cite{karger02findingnearest}. SkipNet \cite{harvey03skipnet1}
1075  uses a combination of proximity and application level overlay routing when performing data  uses a combination of proximity and application level overlay routing when performing data
1076  lookups. Authors call this feature as a \emph{constrained load balancing}.  lookups. Authors call this feature as a \emph{constrained load balancing}.
1077    
1078  Additional research related to proximity based routing include \cite{karger02findingnearest, hildrum02distributedobject,  Additional research related to proximity-based routing include \cite{karger02findingnearest, hildrum02distributedobject,
1079  brinkmann02compactplacement, rhea02probabilistic, castro02networkproximity, ng02predicting, pias03lighthouse}.  brinkmann02compactplacement, rhea02probabilistic, castro02networkproximity, ng02predicting, pias03lighthouse}.
1080    
1081  \subsection{Fast and usable search}  \subsection{Fast and usable search}
# Line 1091  and easy search methods. For instance, I Line 1085  and easy search methods. For instance, I
1085  is the ability to perform keyword searches (e.g., Google \cite{googleurl}). Currently, only loosely  is the ability to perform keyword searches (e.g., Google \cite{googleurl}). Currently, only loosely
1086  structured systems are able to carry out this requirement. Unfortunately, as discussed in this text,  structured systems are able to carry out this requirement. Unfortunately, as discussed in this text,
1087  the data lookup model of the loosely structured approach doesn't scale. Thus, research efforts have  the data lookup model of the loosely structured approach doesn't scale. Thus, research efforts have
1088  been focused on tightly structured systems. The main problem with tightly structured systems is the  been focused towards tightly structured systems. The main problem with tightly structured systems is the
1089  fact that tightly structured algorithms perform data lookups based on a globally unique identifier (key).  fact that tightly structured algorithms perform data lookups based on a globally unique identifier (key).
1090    
1091  Recent study has been focused on the feasibility of Peer-to-Peer Web-like indexing and searching  Recent study has been focused on the feasibility of Peer-to-Peer Web-like indexing and searching
1092  on top of tightly structured overlays \cite{li03feasibility} . Authors argue, that it is possible to implement  on top of tightly structured overlays \cite{li03feasibility} . Authors argue, that it is possible to implement
1093  Peer-to-Peer Web-like search with certain compromises. First, Peer-to-Peer search engine may need to  Peer-to-Peer Web-like search with certain compromises. First, Peer-to-Peer search engine may need to
1094  decrease the result quality in order to make searching more efficient. Second, Peer-to-Peer systems must  decrease the result quality in order to make searching more efficient. Second, Peer-to-Peer systems must
1095  observe the properties of underlying network for better performance.  consult the properties of underlying network for better performance.
1096    
1097  Some studies have been concentrated on SQL-like queries \cite{harren02complex}  Some studies have been concentrated on SQL-like queries \cite{harren02complex}
1098  in tightly structured overlays. Other approaches include adaption of the data lookup model of the loosely  in tightly structured overlays. Other approaches include adaption of the data lookup model of the loosely
# Line 1118  trees \cite{Bhattacharjee03resultcache} Line 1112  trees \cite{Bhattacharjee03resultcache}
1112  in \cite{li03feasibility} use Gap compression \cite{wittengigabytes}, Adaptive Set Intersection \cite{338634}    in \cite{li03feasibility} use Gap compression \cite{wittengigabytes}, Adaptive Set Intersection \cite{338634}  
1113  and clustering with their search optimizations.  and clustering with their search optimizations.
1114    
   
1115  While it is expected that web-like searches can be layered on a top of tightly structured overlay, much  While it is expected that web-like searches can be layered on a top of tightly structured overlay, much
1116  more research is required to make indexing and searching more efficient.  more research is required to make indexing and searching more efficient.
1117    
# Line 1137  All presented algorithms of the tightly Line 1130  All presented algorithms of the tightly
1130  environments. Furthermore, proposed tightly structured overlays are configured statically to achieve  environments. Furthermore, proposed tightly structured overlays are configured statically to achieve
1131  the desired reliability even in a uncommon and adverse environment \cite{rowston03controlloingreliability}.  the desired reliability even in a uncommon and adverse environment \cite{rowston03controlloingreliability}.
1132  The most important factor for future research is to get real-life experiences from tightly structured  The most important factor for future research is to get real-life experiences from tightly structured
1133  systems, when there are frequent joins and leaves in the system. Some research has been done already in this area.  systems, when there are frequent joins and leaves in the system.
1134    
1135  A concept of ''half-life'' was introduced by Liben-Nowell \cite{libennowell01observations} since Peer-to-Peer  The concept of ''half-life'' was introduced by Liben-Nowell \cite{libennowell01observations} since Peer-to-Peer
1136  system is \emph{never} in the ''ideal'' state as it is continiously evolving system. Half-life is defined  system is \emph{never} in the ''ideal'' state as Peer-to-Peer system is continiously evolving system. Half-life is defined
1137  as follows: let there be $N$ live peers at time $t$. The doubling from time $t$ is the time that pass before  as follows: let there be $N$ live peers at time $t$. The doubling from time $t$ is the time that pass before
1138  $N$ new additional peers arrive into the system. The halving time from time $t$ is the time  $N$ new additional peers arrive into the system. The halving time from time $t$ is the time
1139  required for half of the living peers at time $t$ to leave the system. The half-life from  required for half of the living peers at time $t$ to leave the system. The half-life from
# Line 1166  computing power or network bandwidth, al Line 1159  computing power or network bandwidth, al
1159  a serious problem of tightly structured overlays in face of performance and load balancing. Measurement study  a serious problem of tightly structured overlays in face of performance and load balancing. Measurement study
1160  by Saroiu et al. show that there is a extreme heterogeneity among participating peers in already deployed Peer-to-Peer  by Saroiu et al. show that there is a extreme heterogeneity among participating peers in already deployed Peer-to-Peer
1161  systems \cite{saroiu02measurementstudyp2p}. Symphony \cite{gurmeet03symphony} seems to be the first tightly structured overlay system  systems \cite{saroiu02measurementstudyp2p}. Symphony \cite{gurmeet03symphony} seems to be the first tightly structured overlay system
1162  which supports heterogeneity. Zhao et al. have proposed a secondary layer on top of structured overlay  which supports heterogeneity. Zhao et al. have proposed a secondary layer on top of a structured overlay
1163  to support heterogeneity better \cite{zhao02brocade}.  to support heterogeneity better \cite{zhao02brocade}.
1164    
1165  Research has been done on self-organization. Ledlie et al. propose techniques for forming and maintaining  Research has been done on self-organization. Ledlie et al. propose techniques for forming and maintaining
1166  groups in highly dynamic environment \cite{ledlie02selfp2p}. Unfortunately their work relies on the idea that  groups in a highly dynamic environment \cite{ledlie02selfp2p}. Unfortunately their work relies on the idea that
1167  participating peers would create multiple hierarchical groups. It is not clear whether this approach  participating peers would create multiple hierarchical groups. It is not clear whether this approach
1168  is fault tolerant and suitable for a Peer-to-Peer environment. More promising work has been done by Rowston et al.  is fault tolerant and suitable for Peer-to-Peer environment. More promising work has been done by Rowston et al.
1169  in \cite{rowston03controlloingreliability}. Authors propose techniques for self-tuning, dealing with  in \cite{rowston03controlloingreliability}. Authors propose techniques for self-tuning, dealing with
1170  uncommon conditions (e.g., network partition and high failure rates). Moreover, authors argue that  uncommon conditions (e.g., network partition and high failure rates). Moreover, authors argue that
1171  with these techniques, the concerns over tightly structured overlay maintenance costs are no more  with these techniques, the concerns over tightly structured overlay maintenance costs are no more
1172  an open issue.  an open issue.
1173    
1174  Finally, little research has been done regarding self-monitoring and data availability. Zhang et al.  Finally, little research has been done regarding self-monitoring and data availability. Zhang et al.
1175  describe an arbitrary data structure on top of a tightly structured overlay \cite{zhang03somo}. They  describe an arbitrary data structure on top of a tightly structured overlay \cite{zhang03somo}. Authors
1176  call their proposal as a \emph{data overlay}, since it supports several fundamental data structures.  call their technique as a \emph{data overlay}, since it supports several fundamental data structures.
1177  Authors use this data overlay to build Self-Organized Meta data Overlay (SOMO), which can be used  Authors have used this data overlay when building a Self-Organized Meta data Overlay (SOMO), which can be used
1178  for monitoring the health of a tightly structured overlay. The fault tolerance of SOMO itself is currently  for monitoring the health of a tightly structured overlay. The fault tolerance of SOMO itself is currently
1179  unknown.  unknown.
1180    
# Line 1195  In this section we discuss miscellaneous Line 1188  In this section we discuss miscellaneous
1188  All existing Peer-to-Peer systems have rather different interfaces even though they have common properties and  All existing Peer-to-Peer systems have rather different interfaces even though they have common properties and
1189  components. More important, all existing Peer-to-Peer systems are incompatible with each other. One  components. More important, all existing Peer-to-Peer systems are incompatible with each other. One
1190  of the most important area of future research is to create common programming abstractions, i.e.,  of the most important area of future research is to create common programming abstractions, i.e.,
1191  interfaces, design patters and frameworks. Also, equal benchmarks are needed for comparing  interfaces, design patters and frameworks. Also, benchmarks are needed for comparing
1192  different algorithms. Recently, there have been few proposals towards common programming  different algorithms equally. Recently, there have been few proposals towards common programming
1193  guidelines. This list includes \cite{zhao03api, frise02p2pframework, babaoglu02anthill}.  guidelines. This list includes \cite{zhao03api, frise02p2pframework, babaoglu02anthill}.
1194  Early experiments with Peer-to-Peer benchmarking include \cite{ratnasamy02routing, rhea03benchmarks}.  Early experiments with Peer-to-Peer benchmarking include \cite{ratnasamy02routing, rhea03benchmarks}.
1195    
1196  \subsection{Social behavior}  \subsection{Social behavior}
1197    
1198  Frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate. Another belief  Frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate. Another belief
1199  is that all peers would behave equally, i.e., all peers both consume and contribute resources.  is that all peers would behave equally, i.e., all peers both consume and contribute services.
1200  However, these assumptions are not true as several studies show \cite{saroiu02measurementstudyp2p,  However, these assumptions are not true as several studies show \cite{saroiu02measurementstudyp2p,
1201  oram01harnessingpower, hearn02mojonation}. Peers rather consume than contribute and peers are  oram01harnessingpower, hearn02mojonation}. Peers rather consume than contribute and peers are
1202  unwilling to cooperate.  unwilling to cooperate.
# Line 1224  Peer-to-Peer system, which uses empirica Line 1217  Peer-to-Peer system, which uses empirica
1217    
1218  Very little research has been done on simulating a Peer-to-Peer system. Presumably, this  Very little research has been done on simulating a Peer-to-Peer system. Presumably, this
1219  is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very  is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very
1220  difficult. Floyd et al. has been studying the simulation of the Internet in \cite{504642}. Authors  difficult. Floyd et al. have been studying the simulation of the Internet in \cite{504642}. Authors
1221  state that simulating the Internet is very challenging task, because of its heterogeneity  state that simulating the Internet is very challenging task, because of its heterogeneity
1222  and rapid change. Obviously, these factors exist also in Peer-to-Peer systems even with higher  and rapid change. Obviously, these factors exist also in Peer-to-Peer systems even with higher
1223  rates.  rates.
1224    
1225  As long as comprehensive simulations of a Peer-to-Peer systems are lacking, we cannot make any detailed  As long as comprehensive simulations of a Peer-to-Peer systems are lacking, we cannot make any detailed
1226  analysis on general properties of Peer-to-Peer system such as usage patterns. However, we can assume  analysis on general properties of a Peer-to-Peer system, such as usage patterns. However, we can assume
1227  that, e.g., query keywords follow the Zipf-like distribution \cite{breslau98implications} both in the  that, e.g., query keywords follow the Zipf-like distribution \cite{breslau98implications} both in the
1228  Internet and in Peer-to-Peer systems.  Internet and in Peer-to-Peer systems.
1229    
# Line 1559  ganesan02yappers}} & Line 1552  ganesan02yappers}} &
1552  \parbox{90pt}{Locating Peer-to-Peer network} &  \parbox{90pt}{Locating Peer-to-Peer network} &
1553  \parbox{110pt}{How old peers or new peers are able to locate Peer-to-Peer network, if it exists} &  \parbox{110pt}{How old peers or new peers are able to locate Peer-to-Peer network, if it exists} &
1554  \parbox{110pt}{Servers maintaining online peers (e.g. gnutellahosts.com), peer's history information} &  \parbox{110pt}{Servers maintaining online peers (e.g. gnutellahosts.com), peer's history information} &
1555  \parbox{110pt}{Depends on implementation and purpose of the system, for desktop based system there are working solutions, for mobile ad hoc networks more research is needed (Mobile ad hoc  \parbox{110pt}{Depends on implementation and purpose of the system, for a desktop system there are working solutions, for mobile ad hoc networks more research is needed (Mobile ad hoc
1556  networks (MANETs) can be only connected through radio resource interface, i.e., peers which are in same geographical area)}  networks (MANETs) can be only connected through radio resource interface, i.e., peers which are in same geographical area)}
1557  \\ \hline  \\ \hline
1558    
# Line 1582  functionality. Line 1575  functionality.
1575  The Fenfire project \cite{fenfireurl} is an effort to build a location transparent, hyperstructured desktop  The Fenfire project \cite{fenfireurl} is an effort to build a location transparent, hyperstructured desktop
1576  environment. Fenfire uses xanalogical storage model \cite{ted-xu-model} as a basis for hyperstructured  environment. Fenfire uses xanalogical storage model \cite{ted-xu-model} as a basis for hyperstructured
1577  media. Fenfire deploys innovative user interfaces for displaying data to the end users. All data in the Fenfire  media. Fenfire deploys innovative user interfaces for displaying data to the end users. All data in the Fenfire
1578  is stored in a unified format, blocks. This should allow making references between data easier and more  is stored in a unified format, i.e., blocks. This features should allow making references between \emph{any}
1579  seamlessly interoperating than in other systems. For location transparency in a distributed system, Fenfire  data easier and more seamlessly interoperating than in other systems. For location transparency in a distributed system, Fenfire
1580  uses Peer-to-Peer network for locating and fetching blocks.  uses Peer-to-Peer network for locating and fetching blocks.
1581    
1582  Fenfire is a free software and it is licensed under GNU LGPL.  Fenfire was formerly also a partial implementation  Fenfire is a free software and it is licensed under GNU LGPL.  Fenfire was formerly also a partial implementation
1583  of the ZigZag\texttrademark --structure, which was originally invented  of the ZigZag\texttrademark -- structure, which has been originally invented
1584  by Ted Nelson. Now, however, Fenfire uses Resource Description Framework (RDF) \cite{w3rdfurl}  by Ted Nelson. Now, however, Fenfire uses Resource Description Framework (RDF) \cite{w3rdfurl}
1585  for representing internal data structures and their relationships.  for representing internal data structures and their relationships.
1586    
# Line 1595  Fenfire is a high modular software syste Line 1588  Fenfire is a high modular software syste
1588    
1589  \begin{itemize}  \begin{itemize}
1590  \item \textbf{Storm}: a distributed storage module for storing arbitrary data items  \item \textbf{Storm}: a distributed storage module for storing arbitrary data items
1591  \item \textbf{Navidoc}: an UML based tool for generating software documentation    \item \textbf{Navidoc}: an UML-based tool for generating software documentation  
1592  \item \textbf{Alph}: a xanalogical hypertext built upon Storm storage model  \item \textbf{Alph}: a xanalogical hypertext built upon Storm storage model
1593  \item \textbf{GLMosaicText}: a flexible OpenGL interface for font manipulation  \item \textbf{GLMosaicText}: a flexible OpenGL interface for font manipulation
1594  \item \textbf{CallGL}: a wrapping library used for OpenGL calls  \item \textbf{CallGL}: a wrapping library used for OpenGL calls
1595  \item \textbf{LibVob}: a graphic library used for creating navigation interfaces in complex data views  \item \textbf{LibVob}: a graphic library used for creating navigation interfaces in complex data views
1596  \end{itemize}  \end{itemize}
1597    
1598  In this thesis, we focus on Storm and Alph modules, since they are the foundation of Fenfire's  
1599  Peer-to-Peer functionality. If not otherwise mentioned, we use term 'Storm' when referring to both  For location transparency in the Fenfire system, Storm software module
 Storm and Alph software modules. For location transparency in the Fenfire system, Storm software module  
1600  must have a support for Peer-to-Peer functionality as it provides low-level data storage operations  must have a support for Peer-to-Peer functionality as it provides low-level data storage operations
1601  in the Fenfire system.  in the Fenfire system. Therefore, we focus on Storm and Alph modules, since they are the foundation of Fenfire's
1602    Peer-to-Peer functionality. If not otherwise mentioned, we use term 'Storm' when referring to both
1603    Storm and Alph software modules.
1604    
1605    
1606  \section{Xanalogical storage model}  \section{Xanalogical storage model}
# Line 1618  between \emph{documents}, in xanalogical Line 1612  between \emph{documents}, in xanalogical
1612  permanent, globally unique identifier. For instance, let's consider the following  permanent, globally unique identifier. For instance, let's consider the following
1613  scenario: ''the character 'D' typed by Janne Kujala on 10/8/97 8:37:18''. In this  scenario: ''the character 'D' typed by Janne Kujala on 10/8/97 8:37:18''. In this
1614  example, when character 'D' is first typed in, xanalogical storage model  example, when character 'D' is first typed in, xanalogical storage model
1615  acquires a permanent identifier for that character and retains it when character  acquires a permanent identifier for that character and retains it when the character
1616  is copied to different document. Thus, the identifier distinguishes the character from  is copied to different document. Thus, the identifier distinguishes the character from
1617  all similar characters typed in independently\footnote{Xanalogical storage model  all similar characters typed in independently\footnote{Xanalogical storage model
1618  is not limited to text. It can support arbitrary data, e.g., pixels of picture or  is not limited to text. It can support arbitrary data, e.g., pixels of picture or
1619  frames of video.}. The connectivity in the xanalogical storage model between data content  frames of video.}. The connectivity in xanalogical storage model between data content
1620  is more substantial than in other models; a link is shown between any two data contents  is more substantial than in other models; a link is shown between any two data contents
1621  containing a specific \emph{fluid media unit} (e.g., a character) that the link connects.  containing a specific \emph{fluid media unit} (e.g., a character) that the link connects.
1622  In practice, however, xanalogical storage model uses \emph{spans}, ranges of consecutive  In practice, however, xanalogical storage model uses \emph{spans}, ranges of consecutive
1623  fluid media units to perform storage operations. This is done for better performance as  fluid media units to perform storage operations. This is done for better performance as
1624  doing expensive operations for every fluid media unit is not efficient. As a implication,  doing expensive operations for every fluid media unit is not efficient. Xanalogical
1625  the xanalogical storage model stores fluid media units to append-only \emph{scrolls}.  storage model stores fluid media units to append-only \emph{scrolls}.
1626    
1627  \emph{Enfilade} can be considered as a ''virtual file'' (or part of one), which is a list  An \emph{enfilade} can be considered as a ''virtual file'' (or part of one), which is a list
1628  of fluid media content. In the xanalogical storage model, links between content are external  of fluid media content. In xanalogical storage model, links between content are external
1629  and bidirectional. Xanalogical link is an \emph{association} of two enfilades, such as an  and bidirectional. Xanalogical link is an \emph{association} of two enfilades, such as an
1630  annotation to a specific part of a another document. \emph{Transclusion} is an inclusion in  annotation to a specific part of a another document. \emph{Transclusion} is an inclusion in
1631  enfilade of contents already used in another enfilade, i.e., current fluid media is copied into  enfilade of contents already used in another enfilade, i.e., current fluid media is copied into
1632  different data contents. By using this mechanism, a system implementing the xanalogical storage model  different data content. By using this mechanism, a system implementing xanalogical storage model
1633  is able to show all data content that share the same fluid media with current data content  is able to show all data content that share the same fluid media with current data content
1634  (e.g., all documents containing current document's text). Figure \ref{fig:xanalogical_model}  (e.g., all documents containing current document's text). Figure \ref{fig:xanalogical_model}
1635  illustrates xanalogical storage model with documents, text and characters.  illustrates xanalogical storage model with documents, text and characters.
# Line 1664  for creating location-independent, globa Line 1658  for creating location-independent, globa
1658  SHA-1 \cite{fips-sha-1} is used for verifying the integrity of Storm data blocks. Storm  SHA-1 \cite{fips-sha-1} is used for verifying the integrity of Storm data blocks. Storm
1659  blocks have much in common with regular files, except that Storm blocks are \emph{immutable} as  blocks have much in common with regular files, except that Storm blocks are \emph{immutable} as
1660  any change to the byte sequence would change block's hash value (globally unique  any change to the byte sequence would change block's hash value (globally unique
1661  identifier). This mechanism creates a basis for implementing xanalogical storage model in  identifier). This mechanism creates a basis for implementing xanalogical storage model
1662  in Fenfire system. Figure \ref{fig:storm_model} illustrates simplified Storm storage model.  in the Fenfire system. Figure \ref{fig:storm_model} illustrates simplified Storm storage model.
1663    
1664  \begin{figure}  \begin{figure}
1665  \centering  \centering
# Line 1686  In practice, pointer is a random string, Line 1680  In practice, pointer is a random string,
1680  (URN) \cite{rfc2396}. Pointer itself doesn't contain any data, it is rather a \emph{concept} of  (URN) \cite{rfc2396}. Pointer itself doesn't contain any data, it is rather a \emph{concept} of
1681  data. Pointers are created automatically by Storm and each pointer is  data. Pointers are created automatically by Storm and each pointer is
1682  associated with a collection of \emph{pointer blocks}. Pointer block has a single  associated with a collection of \emph{pointer blocks}. Pointer block has a single
1683  target for the pointer. In figure \ref{fig:storm_model}, we present the overall  target for the pointer. In figure \ref{fig:storm_model}, we show the overall
1684  pointer creation process. Pointer block may contain zero or more obsoleted  pointer creation process. Pointer block may contain zero or more obsoleted
1685  pointer blocks, i.e., when a new version of scroll block is created, it supersedes  pointer blocks, i.e., when a new version of scroll block is created, it supersedes
1686  one older version which has been created in the past. The most current pointer  one older version which has been created in the past. The most current pointer
# Line 1704  the most recent pointer's block target i Line 1698  the most recent pointer's block target i
1698    
1699  \chapter{Evaluation of Peer-to-Peer for Fenfire}  \chapter{Evaluation of Peer-to-Peer for Fenfire}
1700    
1701  In this chapter we evaluate Fenfire in a Peer-to-Peer environment.  In this chapter we evaluate Fenfire in Peer-to-Peer environment.
1702  We start by giving a problem overview when considering Fenfire in a Peer-to-Peer  We start by giving a problem overview when considering Fenfire in Peer-to-Peer
1703  environment. We define Fenfire's special needs and evaluate existing  environment. We define Fenfire's special needs and evaluate existing
1704  Peer-to-Peer approaches in light of these requirements. After that, we propose a system  Peer-to-Peer approaches in light of these requirements. After that, we propose a system
1705  model for Fenfire and present simple methods to perform data  model for Fenfire and present simple methods to perform data
1706  lookups in a Peer-to-Peer environment. In the end of this chapter, we discuss possible problems of using Fenfire  lookups in Peer-to-Peer environment. In the end of this chapter, we discuss possible problems of using Fenfire
1707  in a Peer-to-Peer environment.  in Peer-to-Peer environment.
1708    
1709    
1710  \section{Problem overview}  \section{Problem overview}
1711    
1712  As already mentioned in chapter 4, a xanalogical document is a ''virtual  As already mentioned in chapter 4, xanalogical document is a ''virtual
1713  file'', in which parts of the document are fetched from a  file'', in which parts of the document are fetched from a
1714  \emph{global} data repository. Thus, system implementing the xanalogical storage model \emph{must}  \emph{global} data repository. Thus, system implementing xanalogical storage model \emph{must}
1715  support global data lookups efficiently in order to assemble the ''virtual file''  support global data lookups efficiently in order to assemble the ''virtual file''
1716  from fragments of data.  from fragments of data.
1717    
1718  In the xanalogical storage model, each fragment of data is identified by a globally  In xanalogical storage model, each fragment of data is identified by a globally
1719  unique identifier. In the Fenfire system, data fragments are scroll blocks generated by Storm storage module.  unique identifier. In the Fenfire system, data fragments are scroll blocks generated by Storm storage module.
1720  As we discussed already in chapter 4, Fenfire's Storm design  As we discussed already in chapter 4, Fenfire's Storm design
1721  uses SHA-1 \cite{fips-sha-1} hash over the contents of a scroll block for creating globally unique  uses SHA-1 \cite{fips-sha-1} hash over the contents of a scroll block for creating globally unique
1722  identifiers for each scroll block.  In our scenario, fragments of data is distributed  identifiers for each scroll block.  In our scenario, fragments of data is distributed
1723  throughout the Peer-to-Peer overlay. We want that user operations in Fenfire are location transparent.  throughout the Peer-to-Peer overlay network. We want that user operations in Fenfire are location transparent.
1724  Therefore, our task is to locate and fetch (i.e. obtain) \emph{all} Storm scroll blocks, associated to a specific ''virtual  Therefore, our task is to locate and fetch (i.e. obtain) \emph{all} Storm scroll blocks, associated to a specific ''virtual
1725  file'' from the Peer-to-Peer overlay as efficiently as possible. In addition to the  file'' from the Peer-to-Peer overlay as efficiently as possible. In addition to the
1726  \emph{direct} scroll block obtaining using globally unique identifier of Storm scroll block,  \emph{direct} scroll block obtaining using globally unique identifier of Storm scroll block,
1727  we also must support the \emph{indirect} obtaining of Storm scroll block using pointer blocks.  we also must support the \emph{indirect} obtaining of Storm scroll block using the pointer blocks.
1728    
1729  Obviously, our objectives are simple but yet hard to fulfill. First, as a prerequisite  Our objectives are simple but yet hard to fulfill. First, as a prerequisite
1730  to implementing xanalogical storage model in a Peer-to-Peer environment, a system  to implementing xanalogical storage model in Peer-to-Peer environment, a system
1731  supporting data lookups must be able to perform \emph{global} scale lookups. Thus,  supporting data lookups must be able to perform \emph{global} scale lookups. Thus,
1732  we must be able to locate and fetch Storm block, if it exists in the  we must be able to obtain the Storm block, if it exists in the
1733  Peer-to-Peer overlay. Second, data lookups have to be efficient, since constructing  Peer-to-Peer overlay. Second, data lookups have to be efficient, since constructing
1734  one ''virtual file'' may need obtaining several Storm blocks, which are distributed  one ''virtual file'' may need obtaining several Storm blocks, which are distributed
1735  randomly throughout the overlay; if not efficient, construction of the ''virtual file''  randomly throughout the overlay; if not efficient, construction of the ''virtual file''
1736  may take reasonable amount of time while rendering system very unusable. Third, Peer-to-Peer  may take reasonable amount of time while rendering system very unusable. Third, Peer-to-Peer
1737  infrastructure has to be scalable and robust against hostile attacks.  infrastructure has to be scalable and fault tolerant against hostile attacks.
1738    
1739  Some research regarding to these problem has been made by Lukka et al.  Some research regarding to these problems have been made by Lukka et al.
1740  \cite{lukka02freenetguids}. Authors' work is mainly based on the insight of implementing the  \cite{lukka02freenetguids}. Authors' work is mainly based on the insight of implementing  
1741  xanalogical storage model in a Peer-to-Peer environment with globally unique identifiers. Lukka et al.  xanalogical storage model in Peer-to-Peer environment with globally unique identifiers. Lukka et al.
1742  use Freenet \cite{clarke00freenet} as an example Peer-to-Peer system supporting  use Freenet \cite{clarke00freenet} as an example Peer-to-Peer system supporting
1743  globally unique identifiers. The work presented in this thesis extends their work by  globally unique identifiers. The work presented in this thesis extends their work by
1744  evaluating different Peer-to-Peer systems more extensively to Fenfire's needs.  evaluating different Peer-to-Peer systems more extensively to Fenfire's needs.
1745    
1746  Additionally, related to non-xanalogical hypermedia systems, Bouving  Additionally, related to non-xanalogical hypermedia systems, Bouving
1747  \cite{bouvin02openhypermedia} has done initial work regarding  ways in which  \cite{bouvin02openhypermedia} has done initial work regarding ways in which
1748  Peer-to-Peer can be used in non-xanalogical hypermedia systems. Thompson and de Roure  Peer-to-Peer can be used in non-xanalogical hypermedia systems. Thompson and de Roure
1749  \cite{thompson01hypermedia} have studied locating documents and links in Peer-to-Peer  \cite{thompson01hypermedia} have studied locating documents and links in Peer-to-Peer
1750  environment. At the Hypertext '02 panel, moderated by Wiil \cite{wiil02p2phypertext},  environment. At the Hypertext '02 panel, moderated by Wiil \cite{wiil02p2phypertext},
# Line 1759  publishing or not. Line 1753  publishing or not.
1753    
1754  \section{Evaluation of Peer-to-Peer approaches with regard to Fenfire}  \section{Evaluation of Peer-to-Peer approaches with regard to Fenfire}
1755    
1756  In chapter 2, we discussed main differences between loosely and tightly structured  In chapter 2, we discussed main differences between the loosely and the tightly structured
1757  approaches. As stated, the most significant difference is that the tightly structured  approach. As stated, the most significant difference is that the tightly structured
1758  approach has logarithmical properties in all internal operations, while the loosely  approach has logarithmical properties in all internal operations, while the loosely
1759  structured approach doesn't always have even linear properties. Furthermore, the  structured approach doesn't always have even linear properties. Furthermore, the
1760  data lookup model of tightly structured overlay scales much better than loosely  data lookup model of the tightly structured overlay scales much better than in loosely
1761  structured overlays; tightly structured overlay supports global data lookups  structured overlays; the tightly structured overlay supports global data lookups
1762  in the overlay, whereas the data lookup model of the loosely structured approach  in the overlay, whereas the data lookup model of the loosely structured approach
1763  is limited to certain area of the overlay\footnote{The area depends on where the query  is limited to a certain area of the overlay\footnote{The area depends on where the query
1764  originator is located in the overlay.}.  originator is located in the overlay.}.
1765    
1766  For Fenfire's special needs for \emph{locating} data, an important advantage of the  For Fenfire's needs for \emph{locating} data, an important advantage of the
1767  tightly structured approach over the loosely structured approach is that tightly  tightly structured approach over the loosely structured approach is that tightly
1768  structured systems use location-independent, globally unique identifiers for  structured systems use location-independent, globally unique identifiers for
1769  identifying data in the system. Indeed, this  identifying data in the system. Indeed, this
1770  feature is almost analogical to Fenfire's (and xanalogical storage model's) way of  feature is similar to Fenfire's (and xanalogical storage model's) way of
1771  handling data. Another key feature of tightly structured overlays is that they are able  handling data. Another key feature of tightly structured overlays is that they are able
1772  to provide general purpose \emph{interface} for Reference Resolution Services (RRS)\footnote{  to provide general purpose \emph{interface} for Reference Resolution Services (RRS)\footnote{
1773  Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system in the Internet.}  Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system in the Internet.}
1774   \cite{balakrishnan03semanticfree}. Authors argue that next generation RRS must be   \cite{balakrishnan03semanticfree}. Authors argue that next generation RRS must be
1775  application-independent and references itself should be \emph{unstructured} and  application-independent and references itself should be \emph{unstructured} and
1776  \emph{semantically free}. Finally, as said, with tightly structured systems, it is feasible to  \emph{semantically free}. Finally, as said, with tightly structured systems it is feasible to
1777  perform \emph{global} data lookups in the overlay. To summarize, these aspects may be the most important features  perform \emph{global} data lookups in the overlay. To summarize, these aspects may be the most important features
1778  of Peer-to-Peer infrastructure with regard to Fenfire as a distributed, location transparent hypermedia system.  of Peer-to-Peer infrastructure with regard to Fenfire as a distributed, location transparent hypermedia system.
1779  Thus, we see the tightly structured approach as the best alternative to locate data in a Peer-to-Peer  Thus, we see the tightly structured approach as the best alternative to locate data in Peer-to-Peer
1780  environment.  environment.
1781    
1782  Once located, we can use regular TCP/IP-protocols, such as Hypertext Transfer protocol (HTTP)  Once located, we can use regular TCP/IP-protocols, such as Hypertext Transfer protocol (HTTP)
1783  \cite{rfc2068} for \emph{fetching} Storm blocks from the overlay. However, HTTP-protocol may  \cite{rfc2068} for \emph{fetching} Storm blocks from the overlay. However, HTTP-protocol may
1784  not be optimal when obtaining large amounts of data from a Peer-to-Peer network (e.g.,  not be a optimal solution when obtaining large amounts of data from a Peer-to-Peer network (e.g.,
1785  videos, images or music). In this case, multisource downloads can be very useful  videos, images or music). In this case, multisource downloads can be very useful
1786  for better efficiency \cite{maymounkov03ratelesscodes, bittorrenturl}. Furthermore,  for better efficiency \cite{maymounkov03ratelesscodes, bittorrenturl}. Furthermore,
1787  multisource downloads can be used for decreasing load of a certain peer, thus avoiding query  multisource downloads can be used for decreasing load of a certain peer, thus avoiding query
# Line 1795  hot spots in the system \cite{ratnasamy0 Line 1789  hot spots in the system \cite{ratnasamy0
1789  standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content  standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content
1790  hash for verifying the integrity of data by recomputing the content hash  hash for verifying the integrity of data by recomputing the content hash
1791  for a scroll block. In face of multisource downloads, Fenfire must support  for a scroll block. In face of multisource downloads, Fenfire must support
1792  tree-based hashes\footnote{With multisource downloads, tree based hash functions can be used  tree-based hashes\footnote{With multisource downloads, tree-based hash functions can be used
1793  to verify fixed length segments of data. If hash value of data segment is incorrect,  to verify fixed length segments of data. If hash value of data segment is incorrect,
1794  we need only to fetch \emph{segment} of data (instead of whole data) from  we need only to fetch \emph{segment} of data (instead of whole data) from
1795  an other source.}, such as \cite{merkle87hashtree, mohr02thex} for reliable and efficient  an other source.}, such as \cite{merkle87hashtree, mohr02thex} for reliable and efficient
# Line 1808  proximity routing, hostile entities and Line 1802  proximity routing, hostile entities and
1802  Additionally, there is only little real world experiments yet with tightly structured systems  Additionally, there is only little real world experiments yet with tightly structured systems
1803  (e.g., \cite{overneturl, edonkey2kurl}). Therefore, we can't say for sure, how well these  (e.g., \cite{overneturl, edonkey2kurl}). Therefore, we can't say for sure, how well these
1804  systems would perform in real Peer-to-Peer environment. However, we believe that these issues are  systems would perform in real Peer-to-Peer environment. However, we believe that these issues are
1805  solved, since there is a strong and wide research community towards to tightly structured  solved, since there is a strong and wide research community towards to the tightly structured
1806  overlays \cite{projectirisurl}.  overlays \cite{projectirisurl}.
1807    
1808                
1809  \section{Fenfire system model in Peer-to-Peer environment}  \section{Fenfire system model in Peer-to-Peer environment}
1810    
1811  In this section we give a proposal for Fenfire Peer-to-Peer system, which consists  In this section we give a proposal for Fenfire Peer-to-Peer system, which consists
1812  of several technologies reviewed in this thesis.  Then, we introduce simple but  of several technologies reviewed in this thesis.  Then, we introduce methods for
1813  yet effective methods for obtaining Fenfire data from a Peer-to-Peer environment.  obtaining Fenfire data from a Peer-to-Peer network.
1814    
1815  \subsection{System proposal}  \subsection{System proposal}
1816    
# Line 1833  we are able to reduce the generation of Line 1827  we are able to reduce the generation of
1827  locate nearby data without looking up data from distant peers. Moreover, authors'  locate nearby data without looking up data from distant peers. Moreover, authors'
1828  proposal for self-organizing clusters using network diameters may be useful,  proposal for self-organizing clusters using network diameters may be useful,
1829  especially within small groups of working people. Thus, with Sloppy hashing  especially within small groups of working people. Thus, with Sloppy hashing
1830  we can provide locality properties for Fenfire.  we can provide locality properties for the Fenfire system.
1831    
1832  For better fault tolerance and self-monitoring for Fenfire, we propose techniques  For better fault tolerance and self-monitoring for Fenfire, we propose techniques
1833  presented by Rowston et al. \cite{rowston03controlloingreliability}.  With these  presented by Rowston et al. \cite{rowston03controlloingreliability}.  With these
1834  techniques, we can ensure the performance of Fenfire in a highly adverse conditions, such  techniques, we can ensure the performance of the Fenfire system in a highly adverse conditions, such
1835  as sudden network partition, or highly dynamic and heterogeneous environment.  as sudden network partition, or highly dynamic and heterogeneous environment.
1836    
1837  Finally, for more efficient data transfer, we can use variable techniques for this purpose.  Finally, for more efficient data transfer, we can use variable techniques for this purpose.
1838  For small amounts of data, HTTP can be used \cite{rfc2068}. For big amounts of data, we can use  For small amounts of data, HTTP can be used \cite{rfc2068}. For big amounts of data, we can use
1839  multisource downloads for better efficiency and reliability. Specifically, technology based  multisource downloads for better efficiency and reliability. Specifically, the technology based
1840  on rateless erasure codes \cite{maymounkov03ratelesscodes} seems very promising.  on rateless erasure codes \cite{maymounkov03ratelesscodes} seems very promising.
1841    
1842  \subsection{Methods}  \subsection{Methods}
# Line 1850  on rateless erasure codes \cite{maymounk Line 1844  on rateless erasure codes \cite{maymounk
1844  We use the DOLR abstraction of the tightly structured approach, i.e., each participating peer hosts  We use the DOLR abstraction of the tightly structured approach, i.e., each participating peer hosts
1845  the data and the overlay maintains only the \emph{pointers} to the data. We decided to use the DOLR  the data and the overlay maintains only the \emph{pointers} to the data. We decided to use the DOLR
1846  abstraction in our model, since DOLR systems locate data without specifying a storage policy explicitly \cite{rhea03benchmarks}.  abstraction in our model, since DOLR systems locate data without specifying a storage policy explicitly \cite{rhea03benchmarks}.
1847  DHT based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have  DHT-based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have
1848  critical problems with load balancing in a highly heterogeneous environment. This problem is caused by peers  severe problems with load balancing in a highly heterogeneous environment. The problem is caused by peers
1849  which may not be able to store relatively large amount of data with key-value pair, assigned randomly by  which may not be able to store relatively large amount of data with a key-value pair, assigned randomly by
1850  the mapping function of the overlay. These systems waste both storage and bandwidth, and  the mapping function of the overlay. These systems waste both storage and bandwidth, and
1851  are sensitive to certain attacks (e.g., DDoS attack). Additionally, we emphasize that we prefer \emph{abstraction}  are sensitive to certain attacks (e.g., the DDoS attack). Additionally, we emphasize that we prefer \emph{abstraction}
1852  level analysis as very recently better and better tightly structured algorithms have been proposed.  level analysis as very recently better and better tightly structured algorithms have been proposed.
1853  Thus, we don't want to bind our system proposal to a specific algorithm definitively as we expect  Thus, we don't want to bind our system proposal to a specific algorithm definitively as we expect
1854  that this development continues.  that this development continues. In this model, we use Kademlia's \cite{maymounkov02kademlia} algorithm for
1855    locating data in the overlay.
1856    
1857  In the following subsections we assume that we know the structure of  In the following subsections we assume that we know the structure of
1858  the ''virtual file'' before hand, i.e., when assembling the ''virtual file'', we know all Storm  the enfilade before hand, i.e., when assembling the ''virtual file'' we know all the Storm
1859  blocks, which are required when building the ''virtual file''. Also, we don't  blocks, which are required to complete the enfilade. Also, we don't
1860  respond to security issues related to Peer-to-Peer systems, since there is no working solution  respond to the security issues related to Peer-to-Peer systems, since there is no working solution
1861  available yet; we either assume that Fenfire has a reliable technique for identifying individual entities, or  available yet; we either assume that Fenfire has a reliable technique for identifying individual entities, or
1862  there are no hostile entities among participating peers.  there are no hostile entities among participating peers.
1863    
1864  In our model, each peer maintains following data structures for local operations: a data structure for listing all  In our method, each peer maintains the following data structures for local operations: a data structure for listing all
1865  key-value pairs which peer maintains; a data structure for listing all key-value pairs in the  key-value pairs which peer maintains; a data structure for listing all key-value pairs in the
1866  chronological order (the most recent block is topmost) which peer maintains. We use Storm blocks' identifiers  chronological order (the most recent block is topmost) which peer maintains. We use Storm blocks' identifiers
1867  as \emph{keys} of the overlay. Every key-value pair consists of either a hash of pointer random string  as \emph{keys} of the overlay. Every key-value pair consists of either a hash of pointer random string
1868  (pointer blocks), or a hash of block's content (scroll blocks) as a key. Value is always a reference to a hosting  (pointer blocks), or a hash of block's content (scroll blocks) as a key. The value is always a reference to a hosting
1869  peer (e.g., IP address). We use Kademlia's \cite{maymounkov02kademlia} algorithm for locating data in the overlay.  peer (e.g., IP address). Finally, we assume that all local operations can be done in a constant time.
 Finally, we assume that all local operations can be done in a constant time.  
1870    
1871    
1872  \begin{itemize}  \begin{itemize}
1873  \item Data lookup with a given identifier of Storm scroll block.  \item Data lookup with a given identifier of Storm scroll block.
1874  \begin{enumerate}  \begin{enumerate}
1875  \item Submit data lookup using scroll block's identifier.  \item Submit the data lookup using scroll block's identifier.
1876  \item Repeat until the pointer peer is found: each peer forwards the data lookup to a closer peer which hosts the given scroll block identifier.  \item Repeat until the pointer peer is found: each peer forwards the data lookup to a closer peer which hosts the given scroll block identifier.
1877  \item The pointer peer returns value (e.g., the IP address of provider peer) to the query originator.  \item The pointer peer returns value (e.g., the IP address of provider peer) to the query originator.
1878  \item The query originator requests the provider peer to return the scroll block.  \item The query originator requests the provider peer to return the scroll block.
1879  \end{enumerate}  \end{enumerate}
1880  \end{itemize}  \end{itemize}
1881    
1882  Figure \ref{fig:storm_query_blockid} illustrates how a Storm scroll block is located  Figure \ref{fig:storm_query_blockid} illustrates how Storm scroll block is located
1883  in tightly structured overlay using the DOLR abstraction, where the identifier of Storm scroll  in a tightly structured overlay using the DOLR abstraction, where the identifier of Storm scroll
1884  block is known.  block is given.
1885    
1886    
1887  \begin{itemize}  \begin{itemize}
1888  \item Data lookup with a given pointer random string returning most recent scroll block.  \item Data lookup with a given pointer random string returning most recent scroll block.
1889  \begin{enumerate}  \begin{enumerate}
1890  \item The query originator locally computes a hash for given pointer random string.  \item The query originator locally computes a hash over given pointer random string.
1891  \item Repeat until the pointer peer is found: each peer forwards the data lookup to a closer peer which hosts the given hash of pointer random string.  \item Repeat until the pointer peer is found: each peer forwards the data lookup to a closer peer which hosts the given hash of pointer random string.
1892  \item The pointer peer returns most recent pointer block's key-value pair (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes.  \item The pointer peer returns most recent pointer block's key-value pair (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes.
1893  \item The query originator requests the provider peer to return the scroll block.  \item The query originator requests the provider peer to return the scroll block.
# Line 1903  block is known. Line 1897  block is known.
1897  \begin{itemize}  \begin{itemize}
1898  \item Data lookup with a given pointer random string returning scroll block(s) for a given date or time range.  \item Data lookup with a given pointer random string returning scroll block(s) for a given date or time range.
1899  \begin{enumerate}  \begin{enumerate}
1900  \item The query originator locally computes a hash for given pointer random string.  \item The query originator locally computes a hash over given pointer random string.
1901  \item Repeat until the pointer peer is found: each peer forwards the data lookup to a closer peer which hosts the given hash of pointer random string.  \item Repeat until the pointer peer is found: each peer forwards the data lookup to a closer peer which hosts the given hash of pointer random string.
1902  \item Pointer peer returns pointer block's key-value pair(s) (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes.  \item Pointer peer returns pointer block's key-value pair(s) (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes.
1903  \item The query originator requests the provider peer to return the scroll block.  \item The query originator requests the provider peer to return the scroll block.
# Line 1911  block is known. Line 1905  block is known.
1905  \end{itemize}  \end{itemize}
1906    
1907  Figure \ref{fig:storm_query_urn5} illustrates how Storm scroll block is located  Figure \ref{fig:storm_query_urn5} illustrates how Storm scroll block is located
1908  in tightly structured overlay using the DOLR abstraction, where the pointer random string is known.  in a tightly structured overlay using the DOLR abstraction, where the pointer random string is given.
1909    
1910  Each of these algorithms can locate Fenfire data in $O(\log{n})$ time at application level overlay:  Each of these algorithms can locate Fenfire blocks in $O(\log{n})$ time at application level overlay:
1911  $O(\log{n})$ time for query routing to pointer peer and constant time for  $O(\log{n})$ time for query routing to pointer peer and constant time for
1912  locating hosting peer with a given reference link.  locating hosting peer with a given reference link.
1913    
# Line 1935  locating hosting peer with a given refer Line 1929  locating hosting peer with a given refer
1929    
1930  \subsection{Problems}  \subsection{Problems}
1931    
1932  Perhaps the most biggest issue in Peer-to-Peer systems is non-maturity of  Perhaps the most biggest issue in Peer-to-Peer systems is the non-maturity of
1933  security technologies. For instance, online entities cannot be identified  security technologies. For instance, online entities cannot be identified
1934  safely (e.g., the Sybil attack \cite{douceur02sybil}). For Fenfire, one  safely (e.g., the Sybil attack \cite{douceur02sybil}). For the Fenfire system, one
1935  security related problem occurs when user wants to perform a global data lookup with a given  security related problem occurs when a user wants to perform a global data lookup with a given
1936  pointer random string; how can a user verify the correctness  pointer random string; how the user is able to verify the correctness
1937  of the search results ? Specifically, how she or he knows which one is the  of the search results, i.e.,  how she or he knows which one is the
1938  correct Storm scroll block ? The Spam attack \cite{naor03simpledht} is a variation of previously  correct Storm scroll block ? The Spam attack \cite{naor03simpledht} is a variation of previously
1939  mentioned problem; data lookup is performed by a user, but there is no reply  mentioned problem; data lookup is performed by a user, but there is no reply
1940  from the system. How are we able to know if this was a spam attack, or the  from the system. How are we able to know if this was a spam attack, or the
# Line 1950  and after a network disconnection, user Line 1944  and after a network disconnection, user
1944  authenticity of data. Obviously, optimal solution to all security issues would  authenticity of data. Obviously, optimal solution to all security issues would
1945  be that digital signatures are included to every message sent to the system therefore  be that digital signatures are included to every message sent to the system therefore
1946  enabling peers to authenticate other peers safely. However, these problems are not  enabling peers to authenticate other peers safely. However, these problems are not
1947  only limited to the Fenfire as it concerns all Peer-to-Peer based computer systems.  only limited to the Fenfire system as it concerns all Peer-to-Peer computer systems.
1948    
1949  As security technologies come more mature, we wish to apply these  As security technologies come more mature, we wish to apply these
1950  technologies with Fenfire, if applicable.  technologies with Fenfire, if applicable.
# Line 1958  technologies with Fenfire, if applicable Line 1952  technologies with Fenfire, if applicable
1952  \chapter{Conclusions and future work}  \chapter{Conclusions and future work}
1953    
1954  In this thesis, we have reviewed existing Peer-to-Peer approaches, algorithms and  In this thesis, we have reviewed existing Peer-to-Peer approaches, algorithms and
1955  their properties.  We have summarized open problems in Peer-to-Peer research domain.  their properties. Our insight is that despite the great amount of Peer-to-Peer systems,
1956  Specifically, we divided open problems into the three sub-categories: security related problems,  we are able to classify \emph{all} systems either to loosely or tightly structured systems.  
1957  performance related problems and miscellaneous problems. Each of these  We have summarized open problems in Peer-to-Peer research domain. Specifically, we divided open
1958    problems into the three sub-categories: security related problems,
1959    performance related problems and miscellaneous problems.  We point out that each of these
1960  sub-categories have number of open problems, in which there are no solutions  sub-categories have number of open problems, in which there are no solutions
1961  yet, or solutions are only partial. We point out that much research work is required to  yet, or solutions are only partial.
 solve these problems.  
1962    
1963  Then, we focused our attention to the Fenfire system. First, we gave a brief  Then, we focused our attention to the Fenfire system. First, we gave a brief
1964  overview of Fenfire and xanalogical storage model. We also described Storm software module.  overview of the Fenfire system and xanalogical storage model. We also described Storm software module.
1965    
1966  In the last chapter, we evaluated existing Peer-to-Peer approaches with regard  In the last chapter, we evaluated existing Peer-to-Peer approaches with regard
1967  to Fenfire's needs. We proposed that the tightly structured approach is the  to Fenfire's needs. We see that the tightly structured approach is the
1968  best alternative to Fenfire's needs for the following reasons. First, Storm, xanalogical  best alternative to Fenfire's needs for the following reasons. First, Storm, xanalogical
1969  storage model and tightly structured systems use global unique identifiers  storage model and tightly structured systems use global unique identifiers
1970  for identifying data. Second, our Storm design uses \emph{semantic-free references}  for identifying data. Second, our Storm design uses \emph{semantic-free references}
1971  (block identifiers and pointer random strings) for locating data in distributed  (block identifiers and pointer random strings) for locating data in distributed
1972  networks. As the authors of \cite{balakrishnan03semanticfree},  networks. As the authors of \cite{balakrishnan03semanticfree},
1973  we also agree that references should be semantically-free in the next-generation  we also agree that references should be semantically-free in next-generation
1974  reference resolution services. Third, by using  reference resolution services. Third, by using
1975  the DOLR abstraction of tightly structured overlay, we can minimize the lack  the DOLR abstraction of tightly structured overlay, we can minimize the lack
1976  of locality in the tightly structured approach. Finally, we believe that issues  of locality in the tightly structured approach. Finally, we believe that issues
1977  related to tightly structured overlays are solved in the near future, because of  related to tightly structured overlays are solved in the near future, because of
1978  wide and intensive co-operation among research groups.  wide and intensive co-operation among research groups.
1979    
1980  Then, we proposed system model for Fenfire and presented simple methods to perform data  Our future work includes a support for searching transclusions and xanalogical
1981  lookups in a Peer-to-Peer environment.  links in Peer-to-Peer environment. Preliminary analysis have shown
   
 Our future work includes support for searching transclusions and xanalogical  
 links in a Peer-to-Peer network. Specifically, we want to find transclusions  
 and xanalogical links in a global scale. Preliminary analysis have shown  
1982  that these questions are rather different than locating scroll or pointer  that these questions are rather different than locating scroll or pointer
1983  blocks \emph{directly} from the network. Techniques used in distributed  blocks from Peer-to-Peer environment. Techniques used in distributed
1984  database systems may prove to be useful. Some fundamental results  database systems may prove to be useful. Some fundamental results
1985  regarding Peer-to-Peer and database systems have already been  regarding Peer-to-Peer and database systems have already been
1986  presented in \cite{gribble01p2pdatabase}.    presented in \cite{gribble01p2pdatabase}.  

Legend:
Removed from v.1.166  
changed lines
  Added in v.1.167

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26