/[gzz]/gzz/Documentation/misc/hemppah-progradu/masterthesis.tex
ViewVC logotype

Diff of /gzz/Documentation/misc/hemppah-progradu/masterthesis.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.196 by hemppah, Wed Mar 26 12:49:20 2003 UTC revision 1.197 by hemppah, Wed Mar 26 15:00:22 2003 UTC
# Line 177  and maintained. In the following section Line 177  and maintained. In the following section
177  In the loosely structured approach the construction and the maintenance of the overlay is controlled  In the loosely structured approach the construction and the maintenance of the overlay is controlled
178  loosely. The placement of services and the topology of overlay is random. The data lookup model in loosely structured systems is  loosely. The placement of services and the topology of overlay is random. The data lookup model in loosely structured systems is
179  not very efficient, because of unstructured properties of the overlay. Data lookup model is a combination of methods which  not very efficient, because of unstructured properties of the overlay. Data lookup model is a combination of methods which
180  are used for finding data from the overlay.    are used for locatin data in the overlay.  
181    
182  \subsection{Definition}  \subsection{Definition}
183    
# Line 893  and that replicas should be located phys Line 893  and that replicas should be located phys
893    
894  According to \cite{aberer01trust}, mutual trust ''...allows agents to cooperate in a game-theoretic situation that corresponds  According to \cite{aberer01trust}, mutual trust ''...allows agents to cooperate in a game-theoretic situation that corresponds
895  to the repeated prisoners dilemma and leads in the long term to an increased aggregated utility for the participating agents''.  to the repeated prisoners dilemma and leads in the long term to an increased aggregated utility for the participating agents''.
896  They define \emph{trust management} as a mechanism that allows to establish mutual trust. Furthermore, \emph{reputation} is a measure  The authors of \cite{aberer01trust} define \emph{trust management} as a mechanism that allows to establish mutual trust. Furthermore, \emph{reputation} is a measure
897  that is derived from knowledge on interactions in the past \cite{aberer01trust}. In this subsection, we discuss mechanisms to maintain  that is derived from knowledge on interactions in the past \cite{aberer01trust}. In this subsection, we discuss mechanisms to maintain
898  trust in Peer-to-Peer systems.  trust in Peer-to-Peer systems.
899    
# Line 922  are efficient and reliable methods for i Line 922  are efficient and reliable methods for i
922  \subsection{Anonymity}  \subsection{Anonymity}
923    
924  According to \cite{dingledine00free}, there exist several kinds of anonymity: author-anonymity,  According to \cite{dingledine00free}, there exist several kinds of anonymity: author-anonymity,
925  publisher-anonymity, reader-anonymity, peer-anonymity and query-anonymity. Author-anonymity is a form  publisher-anonymity, reader-anonymity, peer-anonymity and query-ano-nymity. Author-anonymity is a form
926  of anonymity in which no one can link the author (who created the document) to a document.  of anonymity in which no one can link author (who created the document) to a document.
927  In publisher-anonymity system, no one is able to determine the publisher (how published the document into  Publisher-anonymity means that no one is able to determine the publisher (how published the document into
928  the system) of a document. Reader-anonymity means that a document cannot be linked to its readers.  the system) of a document. Reader-anonymity means that a document cannot be linked to its readers.
929  With peer-anonymity, no one is able to determine the peer, where the document was originally published.  With peer-anonymity, no one is able to determine the peer, where the document was originally published.
930  Document-anonymity means that a peer doesn't know which data it is currently hosting. Query-anonymity is a form  Document-anonymity means that a peer doesn't know which data it is currently hosting. Finally, query-anonymity is a form
931  of document-anonymity; when other peers perform data lookups, a peer doesn't know which data it serves  of document-anonymity; when other peers perform data lookups, a peer doesn't know which data it serves
932  to the data lookup originators. As the authors cite in \cite{dingledine00free}, some forms of anonymity  to the data lookup originators. As the authors of \cite{dingledine00free} cite, some forms of anonymity
933  may imply each other and possible issues raised by this property is one area of future work.  may imply each other and possible issues raised by this property is one area of future work.
934    
935  Obviously, existance of several types of anonymity often conflicts with other key properties of  Obviously, existance of several types of anonymity often conflicts with other key properties of
# Line 940  situations, such as pseudonymity which i Line 940  situations, such as pseudonymity which i
940  For instance, pseudonymity can be used for addressing peer-anonymity by providing anonymous-like identifiers to  For instance, pseudonymity can be used for addressing peer-anonymity by providing anonymous-like identifiers to
941  peers (e.g., peer identifiers of a tightly structured system).  peers (e.g., peer identifiers of a tightly structured system).
942    
943  Anonymity is widely used in a Peer-to-Peer system in which data publication and non-censorship are important. These include  Anonymity is widely used in a Peer-to-Peer system in which data publication and non-censorship are important. Forwarding
944  Forwarding proxies are used in Freenet \cite{clarke00freenet}, Crowds \cite{reiter98crowds} and Free Haven \cite{dingledine00free}  proxies are used in Freenet \cite{clarke00freenet}, Crowds \cite{reiter98crowds} and Free Haven \cite{dingledine00free}
945  in order to provide various types of anonymity. Tangler \cite{502002} and Publius \cite{pub00} use cryptographic sharing methods  in order to provide various types of anonymity. Tangler \cite{502002} and Publius \cite{pub00} use cryptographic sharing methods
946  to split data into fragments \cite{Shamir1979a}. Mix mailer networks (e.g., \cite{mixminionurl}) are commonly used in  to split data into fragments \cite{Shamir1979a}. Mix mailer networks (e.g., \cite{mixminionurl}) are commonly used in
947  distributed systems which are able to provide some level of anonymity (e.g., \cite{mneturl}).  distributed systems which are able to provide some level of anonymity (e.g., \cite{mneturl}).
948    
949  Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no  Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no
950  such a system which is able to provide complete anonymity. Specifically, the conflicts  such a system which is able to provide complete anonymity in all levels (see above). Specifically, the conflicts
951  between anonymity and other properties of Peer-to-Peer system require more research work.  between anonymity and other properties of Peer-to-Peer system require more research work.
952    
953    
# Line 964  control problem. They use Resource Descr Line 964  control problem. They use Resource Descr
964  schema policies to restrict access to certain data. Their current early prototype version only works in  schema policies to restrict access to certain data. Their current early prototype version only works in
965  loosely structured systems.  loosely structured systems.
966    
967    More research work is required in the development of \emph{working} access control scheme for Peer-to-Peer
968    systems.
969    
970    
971  \subsection{Hostile entities}  \subsection{Hostile entities}
972    
973  One serious problem in Peer-to-Peer systems is the inability to distinguish hostile entities from regular entities  One serious problem in Peer-to-Peer systems is the inability to distinguish hostile entities from regular entities
974  trustworthy.  trustworthy. Identification of hostile entities is essential in the tightly structured
975    approach, in which the fundamental (and implicit) assumption is that there is a random, uniform distribution
976    of peer identifiers that cannot be controlled by a hostile entity.
977    
978  One possible solution is to use a self-monitoring system, such as SOMO \cite{zhang03somo}, in which a self-monitoring overlay  One possible solution is to use a self-monitoring system, such as SOMO \cite{zhang03somo}, in which a self-monitoring overlay
979  constantly analyses the Peer-to-Peer overlay. Self-monitoring overlay is built on top of Peer-to-Peer overlay. Authors in  constantly analyses the Peer-to-Peer overlay. Self-monitoring overlay is built on top of Peer-to-Peer overlay. Authors in
980  \cite{sit02securitycons} suggest the use of system invariants. They emphasize that system invariants should be veriable, and if  \cite{sit02securitycons} suggest the use of system invariants. They emphasize that system invariants should be veriable, and if
981  system invariants fail the system must have a recovery mechanism. In distributed peer identifier assignment \cite{castro02securerouting, clarke00freenet},  system invariants fail the system must have a recovery mechanism. In distributed peer identifier assignment \cite{castro02securerouting, clarke00freenet},
982  multiple participating peers participate in a creation of peer identifier. Identification of hostile entities is essential in the tightly structured  multiple participating peers participate in a creation of peer identifier.
 approach, in which the fundamental (and implicit) assumption is that there is a random, uniform distribution  
 of peer identifiers that cannot be controlled by a hostile entity.  
983    
984  Naturally, centralized authorities could be used for the assignment of peer identifiers, but they may not be suitable  Centralized authorities could be used for the assignment of peer identifiers, but they may not be suitable
985  for ad hoc Peer-to-Peer environrment and have property of single point of failure. Moreover, distributed peer  for ad hoc Peer-to-Peer environrment and have property of single point of failure. Distributed peer
986  identification assignment can be problematic as long as the Sybil attack \cite{douceur02sybil} remains unsolved.  identification assignment can be problematic as long as the Sybil attack \cite{douceur02sybil} remains unsolved.
987  However, there are some partial solutions for controlling the \emph{rate} at which hostile entity is able to obtain peer  However, there are some partial solutions for controlling the \emph{rate} at which hostile entity is able to obtain peer
988  identifier, such as crypto-based puzzles \cite{juels99clientpuzzles}.  identifier, such as crypto-based puzzles \cite{juels99clientpuzzles}.
# Line 989  Secure query routing is essential to any Line 993  Secure query routing is essential to any
993  is able to deliver a network message thoughout the overlay to a correct destination efficiently.  is able to deliver a network message thoughout the overlay to a correct destination efficiently.
994    
995  Aspnes et al. in \cite{aspnes02faultrouting} and Kaashoek et al. in \cite{kaashoek03koorde} formally  Aspnes et al. in \cite{aspnes02faultrouting} and Kaashoek et al. in \cite{kaashoek03koorde} formally
996  prove the lower and upper bounds for the space requirements of locating a specific data item reliable in a  prove the lower and upper bounds for the space requirements of locating a specific data item reliably in a
997  Peer-to-Peer system. They show that to provide high degree of fault tolerance and efficiency in the system, each  Peer-to-Peer system. They show that to provide high degree of fault tolerance and efficiency in the system, each
998  participating peer must maintain average of $O(\log{n})$ neighbors. Fiat et al. in \cite{fiat02censorship, saia02dynamicfaultcontentnetwork}  participating peer must maintain average of $O(\log{n})$ neighbors.
999  and Datar in \cite{datar02butterflies} propose a tightly structured overlay with analytical results in the    
 presence of hostile entities. However, none of these proposals address a dynamic tightly structured  
 overlay with fault tolerance against multiple rounds  
 of hostile attacks. Also, above mentioned proposals are not very efficient. In \cite{fiat02censorship}, each peer  
 must maintain information of $O(\log^3{n})$ other peers, and in \cite{datar02butterflies}, $O(\log^2{n})$ is required.    
1000    
1001  Authors argue in \cite{castro02securitystructured} that with the combination of  Authors argue in \cite{castro02securitystructured} that with the combination of
1002  secure peer identifer assignment, secure routing table maintenance and secure message forwarding  secure peer identifer assignment, secure routing table maintenance and secure message forwarding
1003  the secure routing in tightly structured systems is possible. Additionally, authors cite in \cite{castro02securerouting}  secure query routing in tightly structured systems is possible. Additionally, authors cite in \cite{castro02securerouting}
1004  that the probability of routing successfully between to arbitrary  that the probability of routing successfully between to arbitrary
1005  correct peers is $(1-f)^{h-1}$, when a fraction $f$ of the other peers are faulty or hostile and where  correct peers is $(1-f)^{h-1}$, when a fraction $f$ of the other peers are faulty or hostile and where
1006  $h$ is the number of hops in the overlay. Sit and Morris \cite{sit02securitycons} discuss the possibility of  $h$ is the number of hops in the overlay. Sit and Morris \cite{sit02securitycons} discuss the possibility of
# Line 1011  in function i.e., it is unknown if this Line 1011  in function i.e., it is unknown if this
1011  Lynch et al. \cite{lynch02atomicdataaccess} propose a solution for secure routing table  Lynch et al. \cite{lynch02atomicdataaccess} propose a solution for secure routing table
1012  maintenance, but their solution seems to have two major problems according to \cite{castro02securitystructured}.  maintenance, but their solution seems to have two major problems according to \cite{castro02securitystructured}.
1013  First, the solution is very expensive even without faulty or hostile entities. Second, each group of replicas  First, the solution is very expensive even without faulty or hostile entities. Second, each group of replicas
1014  in their solution must have less than 1/3 of its peers faulty. Thus, this feature results in a low  in their solution must have less than 1/3 of its peers faulty. This feature results in a low
1015  probability of successful routing.  probability of successful routing.
1016    
1017  Finally, Gavoille \cite{gavoille01routing} lists open problems in general distributed systems  Fiat et al. in \cite{fiat02censorship, saia02dynamicfaultcontentnetwork}
1018  (not only in Peer-to-Peer domain).  and Datar in \cite{datar02butterflies} propose a tightly structured overlay with secure query routing in the
1019    presence of hostile entities. However, none of these proposals address a dynamic tightly structured
1020    overlay with fault tolerance against multiple rounds
1021    of hostile attacks. Also, above mentioned proposals are not very efficient. In \cite{fiat02censorship}, each peer
1022    must maintain information of $O(\log^3{n})$ other peers, and in \cite{datar02butterflies}, $O(\log^2{n})$ is required.
1023    
1024  \subsection{Other security threats}  \subsection{Other security threats}
1025    
# Line 1027  this kind of solution would be applicabl Line 1031  this kind of solution would be applicabl
1031    
1032  \subsection{Summary}  \subsection{Summary}
1033    
1034  In this subsection we list security problems in Peer-to-Peer systems in the table.  In this subsection we list security problems in Peer-to-Peer research. For each table entry,
1035    there is a brief description of the problem, possible solutions and comments.
1036    
1037    
1038  \scriptsize  \scriptsize
# Line 1157  searches and system management problems. Line 1162  searches and system management problems.
1162  \subsection{Efficient data lookup}  \subsection{Efficient data lookup}
1163    
1164  The most intensive research in Peer-to-Peer domain has been focused on efficient data lookup methods,  The most intensive research in Peer-to-Peer domain has been focused on efficient data lookup methods,
1165  especially with the loosely structured approach. In iterative deepening  especially with the loosely structured approach.
 \cite{yang02improvingsearch}, multiple BFS searches are initiated  
 with successively larger TTL depth limits, until either the query is satisfied,  
 or the maximum depth $D$ has been reached.  
1166    
1167  Expanding ring, proposed by Shenker et al. in \cite{lv02searchreplication},  In iterative deepening \cite{yang02improvingsearch}, multiple BFS searches are initiated
1168    with successively larger TTL depth limits, until either the query is satisfied,
1169    or the maximum depth $D$ has been reached. Expanding ring, proposed by Shenker et al. in \cite{lv02searchreplication},
1170  is similar to the iterative deepening technique. In this method, a peer starts a flood with small TTL, and  is similar to the iterative deepening technique. In this method, a peer starts a flood with small TTL, and
1171  waits to see if the search is successful. If it is, then the peer stops the data lookuo. Otherwise, the peer increases  waits to see if the search is successful. If it is, then the peer stops the data lookuo. Otherwise, the peer increases
1172  the TTL and starts another data lookup. With these techniques, searches  the TTL and starts another data lookup. With these techniques, searches
# Line 1186  randomly selected neighbor. The basic ra Line 1190  randomly selected neighbor. The basic ra
1190  has a poor response time but it doesn't generate as much network traffic as  has a poor response time but it doesn't generate as much network traffic as
1191  the original BFS. As suggested in \cite{lv02searchreplication}, the  the original BFS. As suggested in \cite{lv02searchreplication}, the
1192  random walk approach can be made more effective by introducing  random walk approach can be made more effective by introducing
1193  multiple simultaneously working ''walkers''. Freenet \cite{clarke00freenet} uses  multiple simultaneously working ''walkers''.
1194  random walk searches in data lookups. Freenet's data lookup model resembles  
1195    Freenet \cite{clarke00freenet} uses random walk searches in data lookups. Freenet's data lookup model resembles
1196  Depth-First-Search (DFS) and peers' routing tables are dynamically built  Depth-First-Search (DFS) and peers' routing tables are dynamically built
1197  using caching. This is an outcome of Freenet's main design principles, anonymity.  using caching. This is an outcome of Freenet's main design principles, anonymity.
1198  Another property of the Freenet's data lookup model is that  Another property of the Freenet's data lookup model is that
# Line 1211  lookups. Authors call this feature as a Line 1216  lookups. Authors call this feature as a
1216  To make Peer-to-Peer systems even more popular (and usable), these systems have to support flexible, efficient  To make Peer-to-Peer systems even more popular (and usable), these systems have to support flexible, efficient
1217  and simple search methods \cite{li03feasibility}. Currently, loosely structured systems are able to carry out the flexibility and  and simple search methods \cite{li03feasibility}. Currently, loosely structured systems are able to carry out the flexibility and
1218  simplicity requirements and tightly structured systems are able to fulfill the efficiency requirement.  simplicity requirements and tightly structured systems are able to fulfill the efficiency requirement.
1219  Research efforts have been focused to make security methods in tightly structured systems more usable since  Research efforts have been focused to make security methods in tightly structured systems more usable and flexible. However, studies
1220  the data lookup model of the loosely structured approach doesn't scale. However, studies  show that combining flexible search methods and tightly structured systems may not be trivial: tightly
1221  show that combining flexible search methods and tightly structured systems may not be trivial \cite{harren02complex,  structured algorithms perform data lookups based on a globally unique identifier thereby making efficient keyword searches hard to implement
1222  ansaryefficientbroadcast03}.  The main problem with tightly structured systems is the  \cite{harren02complex, ansaryefficientbroadcast03}.
 fact that tightly structured algorithms perform data lookups based on a globally unique identifier thereby  
 making efficient keyword searches hard to implement.  
1223    
1224  Some studies have been concentrated on SQL-like queries \cite{harren02complex}  Some studies have been concentrated on SQL-like queries \cite{harren02complex}
1225  in tightly structured overlays. It is unknown, however, if this approach is realizable to implement, since  in tightly structured overlays. It is unknown, however, if this approach is realizable to implement, since
1226  initial analysis have shown that this approach is rather complex. Other approaches include adaption of the data lookup model of the loosely  initial analysis have shown that this approach is rather complex. Other approaches include adaption of the data lookup model of the loosely
1227  structured approach into tightly structured systems \cite{ansaryefficientbroadcast03, chord:om_p-meng}.  structured approach into tightly structured systems \cite{ansaryefficientbroadcast03}.
1228  Work in \cite{ansaryefficientbroadcast03} seems quite promising. Authors' work is based on insight that  Authors' work is based on insight that
1229  performing data lookup in the overlay resembles regular tree-like search, where trees' data structure  performing a data lookup in the overlay resembles regular tree-like search, where trees' data structure
1230  is distributed throughout the overlay. Some studies suggest additional layer upon overlay network \cite{kronfol02fasdsearch,  is distributed throughout the overlay. Some studies suggest additional layer upon overlay network \cite{kronfol02fasdsearch,
1231  joseph02p2players}, which use metadata to implement search methods. The feasibility of implementing additional  joseph02p2players}, which use metadata to implement search methods. The feasibility of implementing additional
1232  search layer on top of the network layer is questionable, especially if the search layer and the network  search layer on top of the network layer is questionable, especially if the search layer and the network
# Line 1239  Peer-to-Peer Web-like search with certai Line 1242  Peer-to-Peer Web-like search with certai
1242  decrease the result quality in order to make searching more efficient. Second, Peer-to-Peer systems must  decrease the result quality in order to make searching more efficient. Second, Peer-to-Peer systems must
1243  consult the properties of underlying network for better performance.  consult the properties of underlying network for better performance.
1244    
1245  Many techniques have been developed in order to provide more efficient search indexing. As  Additionally, many techniques have been developed in order to provide more efficient search indexing. As
1246  several studies show, the popularity of queries in the Internet follow Zipf-like  several studies show, the popularity of queries in the Internet follow Zipf-like
1247  distributions\footnote{Zipf-distribution is a variant of power-law function.  distributions\footnote{Zipf-distribution is a variant of power-law function.
1248  Zipf-distribution can be used in observation of frequency of occurrence event $E$, as a function of the rank  Zipf-distribution can be used in observation of frequency of occurrence event $E$, as a function of the rank
1249  $i$ when the rank is determined by the frequency of occurrence, is a power-law function $E_i \sim \frac{1}{i^{a}}$,  $i$ when the rank is determined by the frequency of occurrence, is a power-law function $E_i \sim \frac{1}{i^{a}}$,
1250  where the exponent $a$ is close to unity.} (e.g., \cite{breslau98implications}).  where the exponent $a$ is close to unity.} (e.g., \cite{breslau98implications}).
1251  Therefore, according to \cite{li03feasibility}, caching and pre-computation can be done for optimizing search indices.  Therefore, according to \cite{li03feasibility}, caching and pre-computation can be done for optimizing search indices.
1252  Authors in \cite{li03feasibility} use Gap compression \cite{wittengigabytes}, Adaptive Set Intersection \cite{338634}    Authors in \cite{li03feasibility} use gap compression \cite{wittengigabytes}, adaptive set intersections \cite{338634}  
1253  and clustering with their search optimizations. Regular compression algorithms, Bloom filters \cite{362692}, vector  and clustering with their search optimizations. Regular compression algorithms, Bloom filters \cite{362692}, vector
1254  space models \cite{CuencaAcuna2002DSIWorkshop} and view trees \cite{Bhattacharjee03resultcache} can be used for even  space models \cite{CuencaAcuna2002DSIWorkshop} and view trees \cite{Bhattacharjee03resultcache} can be used for even
1255  better optimizations.  better optimizations.
# Line 1262  systems require less system management p Line 1265  systems require less system management p
1265  structured system, peers join and leave the system constantly without any restrictions \cite{saroiu02measurementstudyp2p}.  structured system, peers join and leave the system constantly without any restrictions \cite{saroiu02measurementstudyp2p}.
1266  On the other hand, however, peers in tightly structured system join and leave the system but have less freedom,  On the other hand, however, peers in tightly structured system join and leave the system but have less freedom,
1267  i.e., overlay chooses peer's neighbors on behalf of the peer itself and maps data items randomly  i.e., overlay chooses peer's neighbors on behalf of the peer itself and maps data items randomly
1268  throughout the overlay network \cite{balakrishanarticle03lookupp2p}. Almost all presented algorithms  throughout the overlay network \cite{balakrishanarticle03lookupp2p}.
1269    
1270    Almost all presented algorithms
1271  for the tightly structured systems have been analyzed under static simulation  for the tightly structured systems have been analyzed under static simulation
1272  environments \cite{libennowell01observations}. Furthermore, proposed tightly structured overlays are configured statically to achieve  environments \cite{libennowell01observations}. Furthermore, proposed tightly structured overlays are configured statically to achieve
1273  the desired reliability even in a uncommon and adverse environment \cite{rowston03controlloingreliability}.  the desired reliability even in a uncommon and adverse environment \cite{rowston03controlloingreliability}.
# Line 1274  distribution of peer and key identifiers Line 1279  distribution of peer and key identifiers
1279  computing power or network bandwidth, all data items are distributed uniformly. Clearly, this is  computing power or network bandwidth, all data items are distributed uniformly. Clearly, this is
1280  a serious problem of tightly structured overlays in face of performance and load balancing \cite{rao03loadbalancing}.  a serious problem of tightly structured overlays in face of performance and load balancing \cite{rao03loadbalancing}.
1281  Measurement study by Saroiu et al. show that there is a extreme heterogeneity among participating peers in already deployed Peer-to-Peer  Measurement study by Saroiu et al. show that there is a extreme heterogeneity among participating peers in already deployed Peer-to-Peer
1282  systems \cite{saroiu02measurementstudyp2p}. Symphony \cite{gurmeet03symphony} seems to be the first tightly structured overlay system  systems \cite{saroiu02measurementstudyp2p}.
 which supports heterogeneity. Zhao et al. have proposed a secondary layer on top of a structured overlay  
 to support heterogeneity better \cite{zhao02brocade}.  
1283    
1284  Some research has been done with regard to load balancing properties of tightly structured  Some research has been done with regard to load balancing properties of tightly structured
1285  overlays. Byers et al. suggest an idea of ''power of two choices'' whereby data item is stored at the less loaded  overlays. Byers et al. suggest an idea of ''power of two choices'' whereby data item is stored at the less loaded
# Line 1296  Also, query and routing hot spots may be Line 1299  Also, query and routing hot spots may be
1299  Hot spots happen, when a specific key is being requested extremely often in tightly structured overlays. Recent study  Hot spots happen, when a specific key is being requested extremely often in tightly structured overlays. Recent study
1300  by Freedman et al. tries to reduce hot spots in the system by performing \emph{sloppy} hashing  by Freedman et al. tries to reduce hot spots in the system by performing \emph{sloppy} hashing
1301  \cite{sloppy:iptps03}. Authors' technique is especially suitable for the DOLR abstraction of tightly structured overlays.  \cite{sloppy:iptps03}. Authors' technique is especially suitable for the DOLR abstraction of tightly structured overlays.
1302  They arque that with Sloppy hashing, the generation of query hot spots can be reduces and peers are able  They arque that with Sloppy hashing, the generation of query hot spots can be reduced and peers are able
1303  locate nearby data without looking up data from distant peers. Moreover, authors'  locate nearby data without looking up data from distant peers. Moreover, authors'
1304  proposal for self-organizing clusters using network diameters may be useful,  proposal for self-organizing clusters using network diameters.
 especially within small groups of working people.  
1305    
1306  The concept of ''half-life'' was introduced by Liben-Nowell \cite{libennowell01observations} since Peer-to-Peer  The concept of ''half-life'' was introduced by Liben-Nowell \cite{libennowell01observations} since Peer-to-Peer
1307  system is \emph{never} in the ''ideal'' state as Peer-to-Peer system is continiously evolving system. Half-life is defined  system is \emph{never} in the ''ideal'' state as Peer-to-Peer system is continiously evolving system. Half-life is defined
# Line 1319  unknown. Line 1321  unknown.
1321    
1322  \subsection{Summary}  \subsection{Summary}
1323    
1324  In this subsection we list performance and usability problems in Peer-to-Peer systems in the table.  In this subsection we list performance and usability problems in Peer-to-Peer research. For each table entry,
1325    there is a brief description of the problem, possible solutions and comments.
1326    
1327    
1328  \scriptsize  \scriptsize
# Line 1466  the efficiency of different algorithms e Line 1469  the efficiency of different algorithms e
1469  Recently, there have been few proposals towards common programming guidelines. Authors in  Recently, there have been few proposals towards common programming guidelines. Authors in
1470  \cite{zhao03api} propose a higher level abstracions for tightly structured overlays. Frise et al. suggest the use of  \cite{zhao03api} propose a higher level abstracions for tightly structured overlays. Frise et al. suggest the use of
1471  additional layer in Peer-to-Peer system to hide the structure of the overlay \cite{frise02p2pframework}.  additional layer in Peer-to-Peer system to hide the structure of the overlay \cite{frise02p2pframework}.
1472  Thus, with their method both the tightly structured and tightly structured approach can be used in the system.  With their abstraction, both the tightly structured and tightly structured approach can be used in the system.
1473  Montresor proposes a framework supporting developers and researchers in the design of Peer-to-Peer system  Montresor proposes a framework supporting developers and researchers in the design of Peer-to-Peer system
1474  \cite{babaoglu02anthill}.  \cite{babaoglu02anthill}.
1475    
# Line 1477  work focus on application-driven benchma Line 1480  work focus on application-driven benchma
1480    
1481  Frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate (e.g., \cite{shneidman03rationality}).  Frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate (e.g., \cite{shneidman03rationality}).
1482  Another belief is that all peers would behave equally, i.e., all peers both consume and contribute services \cite{saroiu02measurementstudyp2p}.  Another belief is that all peers would behave equally, i.e., all peers both consume and contribute services \cite{saroiu02measurementstudyp2p}.
1483  However, these assumptions are not true as several studies show \cite{saroiu02measurementstudyp2p,  However, these assumptions are not true as several publications show: peers rather consume than contribute and are
1484  oram01harnessingpower, hearn02mojonation}. Peers rather consume than contribute and peers are  unwilling to cooperate \cite{saroiu02measurementstudyp2p, oram01harnessingpower, hearn02mojonation}.
 unwilling to cooperate.  
1485    
1486  Somewhat surprisingly little research has been done in this area, especially when considering  Somewhat surprisingly little research has been done in this area, especially when considering
1487  the possible impact of \emph{unwanted social behavior} to performance of a Peer-to-Peer  the possible impact of \emph{unwanted social behavior} to performance of a Peer-to-Peer
1488  system. The problem is addressed by Golle et al. \cite{golle01incentivesp2p}, Ngan et al.  system. The problem is addressed by Golle et al. \cite{golle01incentivesp2p}, Ngan et al.
1489  \cite{ngan03enforcefile} and Shneidman et al. \cite{shneidman03rationality}.  \cite{ngan03enforcefile} and Shneidman et al. \cite{shneidman03rationality}.
1490  Some research has been focused on semantic properties of the overlay in order to increase  Some research has been focused on semantic properties of the overlay in order to increase
1491  cooperation among participating peers \cite{crespo02semanticoverlay}. Ramanathan et al.  co-operation among participating peers \cite{crespo02semanticoverlay}, i.e., peers'
1492  \cite{ramanathan02goodpeers} and Bernstein et al. \cite{bernstein03selection} use  neighbor connections are influenced by the content of data (e.g. music or movies).
1493    Ramanathan et al. \cite{ramanathan02goodpeers} and Bernstein et al. \cite{bernstein03selection} use
1494  empirical metrics and decision trees when teaching peers to make better decisions  empirical metrics and decision trees when teaching peers to make better decisions
1495  when contacting other peers in Peer-to-Peer system. Alpine \cite{alpineurl} is an example of  when contacting other peers in Peer-to-Peer system. Alpine \cite{alpineurl} is an example of
1496  Peer-to-Peer system, which uses empirical metrics for a peer selection.  Peer-to-Peer system, which uses empirical metrics for a peer selection.
# Line 1499  Very little research has been done on si Line 1502  Very little research has been done on si
1502  is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very  is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very
1503  difficult \cite{bhagwan03availability}. Floyd et al. have been studying the simulation of the Internet in \cite{504642}. Authors  difficult \cite{bhagwan03availability}. Floyd et al. have been studying the simulation of the Internet in \cite{504642}. Authors
1504  state that simulating the Internet is very challenging task, because of its heterogeneity  state that simulating the Internet is very challenging task, because of its heterogeneity
1505  and rapid change. Obviously, these factors exist also in Peer-to-Peer systems even with higher  and rapid change. These factors exist also in Peer-to-Peer systems even with higher
1506  rates.  rates.
1507    
1508  As long as comprehensive simulations of a Peer-to-Peer systems are lacking, we cannot make any detailed  As long as comprehensive simulations of a Peer-to-Peer systems are lacking, we cannot make any detailed
1509  analysis on general properties of a Peer-to-Peer system, such as usage patterns. However, we can assume  analysis on general properties of a Peer-to-Peer system, such as usage patterns of participating peers.
 that, e.g., query keywords follow the Zipf-like distribution \cite{breslau98implications} both in the  
 Internet and in Peer-to-Peer systems.  
1510                                    
1511  \subsection{Summary}  \subsection{Summary}
1512    
1513  In this subsection we list security problems in Peer-to-Peer systems in the table.  In this subsection we list miscellaneous problems in Peer-to-Peer research. For each table entry,
1514    there is a brief description of the problem, possible solutions and comments.
1515    
1516  \scriptsize  \scriptsize
1517  \begin{longtable}{|l|l|l|l|}  \begin{longtable}{|l|l|l|l|}
# Line 1599  which is an essential part of Fenfire's Line 1601  which is an essential part of Fenfire's
1601  \section{Overview}  \section{Overview}
1602    
1603  The Fenfire project \cite{fenfireurl} is an effort to build a location transparent, hyperstructured desktop  The Fenfire project \cite{fenfireurl} is an effort to build a location transparent, hyperstructured desktop
1604  environment. By location transparent, we mean concealing the heterogeneous and distributed nature of the system  environment. By location transparent, we mean hiding the heterogeneous and distributed nature of the system
1605  so that it appears to the end user like one system and by hyperstructured system  so that it appears to the end user like one system and by hyperstructured system
1606  a system in which data can be associated with other data arbitrarly. Fenfire uses xanalogical storage model  a system in which data can be associated with other data arbitrarly. Fenfire uses xanalogical storage model
1607  \cite{ted-xu-model} as a basis for hyperstructured media. Each data item in the Fenfire system has a globally unique  \cite{ted-xu-model} as a basis for hyperstructured media. Each data item in the Fenfire system has a globally unique
1608  identifier. This property should allow making references between \emph{any}  identifier. This property should allow making references between \emph{any}
1609  data easier and more seamlessly interoperating than in other systems. For location transparency in the Fenfire system,  data easier and more seamlessly interoperating than in other systems. For location transparency in the Fenfire system,
1610  we are currently analysing the applicably of Peer-to-Peer infrastructure for locating and fetching blocks in a  we are currently analysing the applicability of Peer-to-Peer infrastructure.
1611  distributed environment. Fenfire was formerly also a partial implementation  
1612    Fenfire was formerly also a partial implementation
1613  of the ZigZag\texttrademark -- structure, which has been originally invented  of the ZigZag\texttrademark -- structure, which has been originally invented
1614  by Ted Nelson. Now, however, Fenfire uses Resource Description Framework (RDF) \cite{w3rdfurl}  by Ted Nelson. Now, however, Fenfire uses Resource Description Framework (RDF) \cite{w3rdfurl}
1615  for representing internal data structures and their relationships.  for representing internal data structures and their relationships.
# Line 1622  Fenfire is a free software and it is lic Line 1625  Fenfire is a free software and it is lic
1625  \item \textbf{LibVob}: a graphic library used for creating navigation interfaces in complex data views.  \item \textbf{LibVob}: a graphic library used for creating navigation interfaces in complex data views.
1626  \end{itemize}  \end{itemize}
1627    
1628  In this thesis, we focus on Storm and Alph modules as they are a foundation for Peer-to-Peer functionality  In this thesis, we focus on Storm and Alph modules as they are the foundation for Peer-to-Peer functionality
1629  in the Fenfire system.  in the Fenfire system.
1630    
1631  \section{Xanalogical storage model}  \section{Xanalogical storage model}
1632    
1633  Xanalogical storage model \cite{nelson99xanalogicalneeded} is a different kind of model for  Xanalogical storage model \cite{nelson99xanalogicalneeded} is a different kind of model for
1634  presenting data and relationships between data, e.g., while in the World Wide Web links are  presenting data and relationships between data, e.g., while in the World Wide Web links are
1635  between \emph{documents}, in xanalogical storage model links are between individual  between documents, in the xanalogical storage model links are between individual
1636  \emph{characters}. \emph{Enfilade},can be considered as a mutable ''virtual file'' (or part of one), which is a list  characters\footnote{Xanalogical storage model
1637  of fluid media content. Fluid media is the smallest units of data in xanalogical storage  is not limited to text. It can support arbitrary data, e.g., pixels of picture or
1638  model. \emph{Transclusion} is an inclusion in  frames of video.}. \emph{Enfilade} is a mutable ''virtual file'' (or part of one), which is a list
1639    of fluid media content. Fluid media is the smallest units of data in the xanalogical storage
1640    model (e.g., a character). \emph{Transclusion} is an inclusion in
1641  enfilade of contents already used in another enfilade. With the transclusion, a system  enfilade of contents already used in another enfilade. With the transclusion, a system
1642  implementing xanalogical storage model is able to show \emph{all} data content that share the same  implementing the xanalogical storage model is able to show \emph{all} data content that share the same
1643  fluid media with current data content (e.g., all documents in a system containing document's text).  fluid media with current data content (e.g., all documents in a system containing document's text).
1644  Figure \ref{fig:xanalogical_model}  Figure \ref{fig:xanalogical_model}
1645  illustrates xanalogical storage model with documents, text and characters.  illustrates the xanalogical storage model with documents, text and characters.
1646    
1647  In xanalogical storage model, links between data are external  In the xanalogical storage model, links between data are external
1648  and bidirectional. A link is shown between any two data contents  and bidirectional. A link is shown between any two data contents
1649  containing a specific \emph{fluid media unit} (e.g., a character) that the link connects.  containing a specific fluid media unit that the link connects.
1650  Each fluid media unit in xanalogical storage model has a  Each fluid media unit in the xanalogical storage model has a
1651  permanent, globally unique identifier\footnote{Xanalogical storage model  permanent, globally unique identifier. For instance, let's consider the following
 is not limited to text. It can support arbitrary data, e.g., pixels of picture or  
 frames of video.}. For instance, let's consider the following  
1652  example, presented first time in \cite{lukka02freenetguids}: ''the character 'D'  example, presented first time in \cite{lukka02freenetguids}: ''the character 'D'
1653  typed by Janne Kujala on 10/8/97 8:37:18''. When character  typed by Janne Kujala on 10/8/97 8:37:18''. When character
1654  'D' is first typed in, xanalogical storage model  'D' is first typed in, the xanalogical storage model
1655  creates a permanent globally identifier for that character  creates a permanent globally identifier for that character
1656  and retains it when the character is copied to different document. In practice, xanalogical  and retains it when the character is copied to different document. In practice, the xanalogical
1657  storage model uses \emph{spans}, ranges of consecutive  storage model uses \emph{spans}, ranges of consecutive
1658  fluid media units to perform storage operations.  fluid media units to perform storage operations.
1659    
1660  \begin{figure}  \begin{figure}
1661  \centering  \centering
1662  \includegraphics[width=14cm, height=12cm]{xanadu_model.eps}  \includegraphics[width=14cm, height=12cm]{xanadu_model.eps}
1663  \caption{Xanalogical storage model.}  \caption{Xanalogical storage model with documents, text and characters.}
1664  \label{fig:xanalogical_model}  \label{fig:xanalogical_model}
1665  \end{figure}  \end{figure}
1666    
# Line 1665  fluid media units to perform storage ope Line 1668  fluid media units to perform storage ope
1668  \section{Storm}  \section{Storm}
1669    
1670  In this section, we will give a brief overview of Storm design. More information can be found  In this section, we will give a brief overview of Storm design. More information can be found
1671  from recent publications: for general discussion about Fenfire in Peer-to-Peer environment,  from recent publications. For general discussion about Fenfire in Peer-to-Peer environment,
1672  see \cite{lukka02freenetguids}, and for detailed Storm design, see \cite{fallenstein03storm}.  see \cite{lukka02freenetguids}, and for detailed Storm design, see \cite{fallenstein03storm}.
1673    
1674  Storm (for \emph{STORage Module}) stores all data as \emph{blocks}, which  Storm (for \emph{STORage Module}) stores all data as data \emph{blocks}, which
1675  are immutable byte sequences. Storm \emph{assigns} a globally unique identifier to each  are immutable byte sequences. Storm \emph{assigns} a globally unique identifier to each
1676  data block\footnote{This resembles the process how tightly structured overlay assigns a subset of keys  data block\footnote{This resembles the process how tightly structured overlay assigns a subset of keys
1677  to each participating peer: a user has no control over the assignment process.}. SHA-1 cryptographic  to each participating peer: a user has no control over the assignment process.}. SHA-1 cryptographic
1678  content hash function\footnote{SHA-1 is  content hash function\footnote{SHA-1 is
1679  considered as a collision free hash function. Therefore, it is very unlikely that two different Storm data blocks  considered as a collision free hash function. Therefore, it is very unlikely that two different Storm data blocks
1680  would have same identifier.} \cite{fips-sha-1} is used  would have same identifier.} \cite{fips-sha-1} is used
1681  for creating semantic-free, globally unique identifiers for blocks. Because of SHA-1  for creating unstructured and semantic-free, globally unique identifiers for blocks. Because of SHA-1
1682  content hash, all identifiers are directly the data verifiers as well. The uniquess of blocks creates  content hash, all identifiers are directly the data verifiers as well. The uniquess of blocks creates
1683  a basis for implementing xanalogical storage model in the Fenfire system. Storm blocks have much in common with regular files as they  a basis for implementing the xanalogical storage model in the Fenfire system. Storm blocks have in common with regular files as they
1684  both contain the data. The main difference is that Storm blocks are \emph{immutable} since any  both contain the data. The main difference is that Storm blocks are \emph{immutable} since any
1685  change to the byte sequence would change block's hash value (i.e., globally unique identifier).    change to the byte sequence would change block's hash value (i.e., globally unique identifier).  
1686    
# Line 1690  More information about diffs can be foun Line 1693  More information about diffs can be foun
1693  \emph{Pointer} \cite{benja02urn5, fallenstein03storm} is a semantic-free updatable reference to  \emph{Pointer} \cite{benja02urn5, fallenstein03storm} is a semantic-free updatable reference to
1694  Storm data block. Pointer is a unique reference to the data and it is usually  Storm data block. Pointer is a unique reference to the data and it is usually
1695  represented as a random string. Storm pointers are rather a \emph{concept} of data (e.g., ''The front page of the most recent  represented as a random string. Storm pointers are rather a \emph{concept} of data (e.g., ''The front page of the most recent
1696  version of New York Times newspaper'') whereas scroll blocks \emph{contain} the data  version of New York Times newspaper'') whereas blocks \emph{contain} the data
1697  (''New York Times newspaper, 10.10.2002, version 1.0'').  (''New York Times newspaper, 10.10.2002, version 1.0'').
1698  Figure \ref{fig:storm_model} illustrates Storm storage model with pointers.  Figure \ref{fig:storm_model} illustrates Storm storage model with pointers.
1699    
1700  Each pointer is \emph{linked} to a collection of \emph{pointer blocks}.  Each pointer is \emph{linked} to a collection of \emph{pointer blocks}.
1701  Pointers can be created by a user, before the creation of scroll blocks. Pointer blocks  Pointers can be created by a user, before the creation of a data block. Pointer blocks
1702  are created automatically by Storm when a scroll block is created and associated with a pointer  are created automatically by Storm when a data block is created and associated with a pointer
1703  (e.g., a user creates a scroll block associated with the concept ''The front page of the most recent  (e.g., a user creates a data block associated with the concept ''The front page of the most recent
1704  version of New York Times newspaper''). Pointer block has always a single target (i.e., a scroll block)  version of New York Times newspaper''). Pointer block has always a single target (i.e., a data block),
1705  for the pointer, saying that pointer $P$ targets block $B$. In addition to this, pointer block  saying that pointer $P$ targets data block $B$. In addition to this, pointer block
1706  may contain a list of zero or more obsoleted pointer blocks: when a new version of pointer  may contain a list of zero or more obsoleted pointer blocks: when a new version of pointer
1707  block is created, it supersedes one older version which has been created in the past using the Storm indexing  block is created, it supersedes one older version which has been created in the past using the Storm indexing
1708  mechanisms. For details, see \cite{fallenstein03storm}. Next  mechanisms. For details, see \cite{fallenstein03storm}. Next
1709  time, when the pointer is used for referring to a specific scroll block, only  time, when the pointer is used for referring to a specific data block, only
1710  the most recent pointer's block target is loaded. However, the pointer blocks pointing  the most recent pointer's block target is loaded. However, the pointer blocks pointing
1711  to the previous versions of scroll blocks remains accessible, if needed. In figure \ref{fig:storm_pointercreation},  to the previous versions of data blocks remains accessible, if needed. In figure \ref{fig:storm_pointercreation},
1712  we show the overall pointer creation process.  we show the overall pointer creation process.
1713    
1714  \begin{figure}  \begin{figure}
1715  \centering  \centering
1716  \includegraphics[width=10cm, height=10cm]{storm_uml.eps}  \includegraphics[width=10cm, height=10cm]{storm_uml.eps}
1717  \caption{Implementation of xanalogical storage model on Storm. Storm storage model is based on  \caption{Implementation of the xanalogical storage model on Storm. Storm storage model is based on
1718  fluid media units, i.e., fluid media units are smallest units of data. Currently, Storm provides a support  fluid media units, i.e., fluid media units are smallest units of data. Currently, Storm provides a support
1719  for textual fluid media units (characters) only, but a support for arbitrary data (e.g., video or music) is  for textual fluid media units (characters) only, but a support for arbitrary data (e.g., video or music) is
1720  planned in future versions of Storm.}  planned in future versions of Storm.}
# Line 1741  in Peer-to-Peer environment. Line 1744  in Peer-to-Peer environment.
1744    
1745  Some research regarding to Peer-to-Peer technologies and hypermedia systems have been made by Lukka et al.  Some research regarding to Peer-to-Peer technologies and hypermedia systems have been made by Lukka et al.
1746  \cite{lukka02freenetguids}. Authors' work is mainly based on the insight of implementing    \cite{lukka02freenetguids}. Authors' work is mainly based on the insight of implementing  
1747  xanalogical storage model in Peer-to-Peer environment with globally unique identifiers. Lukka et al.  the xanalogical storage model in Peer-to-Peer environment with globally unique identifiers. Lukka et al.
1748  use Freenet \cite{clarke00freenet} as an example Peer-to-Peer system supporting  use Freenet \cite{clarke00freenet} as an example Peer-to-Peer system supporting
1749  globally unique identifiers. The work presented in this thesis extends their work by  globally unique identifiers. The work presented in this thesis extends their work by
1750  evaluating different Peer-to-Peer systems more extensively to Fenfire's needs.  evaluating different Peer-to-Peer systems more extensively to Fenfire's needs.
# Line 1758  In Peer-to-Peer environment, our objecti Line 1761  In Peer-to-Peer environment, our objecti
1761  First, as discussed in chapter 4, xanalogical document is a ''virtual  First, as discussed in chapter 4, xanalogical document is a ''virtual
1762  file'', in which parts of the document are fetched from a  file'', in which parts of the document are fetched from a
1763  \emph{global} data repository\footnote{Global repository is not a requirement. Locally constructed xanalogical  \emph{global} data repository\footnote{Global repository is not a requirement. Locally constructed xanalogical
1764  documents are feasible and they can be assembled without global data repository.}. Thus, system implementing xanalogical storage model \emph{must}  documents are feasible and they can be assembled without global data repository.}. System implementing the xanalogical storage model \emph{must}
1765  support global data lookups order to assemble the ''virtual file'' from fragments of data.  support global scale data lookups, i.e., if a data item exists in the system it can be located and fetched.
1766  Specifically, our task is to locate and fetch (i.e., obtain) \emph{all} Storm blocks\footnote{These blocks are called \emph{scroll} blocks.},  Specifically, our task is to locate and fetch (i.e., obtain) all Storm blocks\footnote{We call these blocks as \emph{scroll} blocks.},
1767  associated to a specific ''virtual file'' from the Peer-to-Peer once the construction of ''virtual'' file  associated to a specific ''virtual file'' from the Peer-to-Peer network. Also, in addition to the
 is resolved (i.e., we know what blocks are required to assemble the ''virtual file''). Also, in addition to the  
1768  \emph{direct} block obtaining using globally unique identifier of Storm block,  \emph{direct} block obtaining using globally unique identifier of Storm block,
1769  we also must support the \emph{indirect} obtaining of Storm block using the pointer mechanism.  we also must support the \emph{indirect} obtaining of Storm block using the pointer mechanism.
1770  Second, we want that users' operations in Fenfire  Second, we want that users' operations in Fenfire
# Line 1779  In this section we focus on locating the Line 1781  In this section we focus on locating the
1781  respond to fetching of Storm blocks as fetching can be performed easily once  respond to fetching of Storm blocks as fetching can be performed easily once
1782  Storm block is located.  Storm block is located.
1783    
1784  In chapter 2, we discussed main differences between the loosely and the tightly structured  In chapter 2, we discussed main the differences between the loosely and the tightly structured
1785  approach. As stated, the most significant difference is that the tightly structured  approach. As stated, the most significant difference is that the tightly structured
1786  approach has logarithmical properties in all internal operations, while the loosely  approach has at least poly-logarithmical properties in all internal operations, while the loosely
1787  structured approach doesn't always have even linear properties. Furthermore, the  structured approach doesn't always have even linear properties. Furthermore, the
1788  data lookup model of the tightly structured overlay scales much better than in loosely  data lookup model of the tightly structured overlay scales much better than in loosely
1789  structured overlays; the tightly structured overlay supports global data lookups  structured overlays; the tightly structured overlay supports global data lookups
# Line 1792  is more efficient and scalable than the Line 1794  is more efficient and scalable than the
1794    
1795  Since both Storm and tightly structured overlays use globally unique identifiers for each data item,  Since both Storm and tightly structured overlays use globally unique identifiers for each data item,
1796  it is feasible to use tightly structured overlays for \emph{locating} Storm blocks efficiently.  it is feasible to use tightly structured overlays for \emph{locating} Storm blocks efficiently.
1797  Another key feature of tightly structured overlays is that they are able  Additionally, the unstructured and semantic-free properties of Storm identifiers enables
1798  to provide general purpose \emph{interface} for Reference Resolution Services (RRS)  the use of general purpose Reference Resolution Services (RRS) \cite{balakrishnan03semanticfree} on
1799   \cite{balakrishnan03semanticfree}. Authors argue that next generation RRS must be  top of the tightly structured overlay.
 application-independent and references itself should be \emph{unstructured} and  
 \emph{semantically free}.  Thus, we see the tightly structured approach as the best alternative to  
 locate data in Peer-to-Peer environment.  
1800    
1801  Again, there are research challenges with tightly structured systems which have to be  Again, there are research challenges with tightly structured systems which have to be
1802  addressed, as described in chapter 3. The main concerns include decreased performance and fault  addressed, as described in chapter 3. The main concerns include decreased performance and fault
1803  tolerance in presence of system flux, non-optimal distance functions in identifier space,  tolerance in presence of system flux, non-optimal distance functions in identifier space,
1804  proximity routing, hostile entities and flexible search \cite{balakrishanarticle03lookupp2p}.  proximity routing, hostile entities and flexible search \cite{balakrishanarticle03lookupp2p}.
1805  Additionally, there is only little real world experiments yet with tightly structured systems  Additionally, there is only little real world experiments with tightly structured systems
1806  (e.g., \cite{overneturl, edonkey2kurl}). Therefore, we can't say for sure, how well these  (e.g., \cite{overneturl, edonkey2kurl}). Therefore, we can't say for sure, how well these
1807  systems would perform in real Peer-to-Peer environment. However, we believe that these issues are  systems would perform in real Peer-to-Peer environment. However, we believe that these issues will be
1808  solved in near future, since there is a strong and wide research community towards to the tightly structured  solved in the near future, since there is a strong and wide research community towards tightly structured
1809  overlays \cite{projectirisurl}.  overlays \cite{projectirisurl}.
1810    
1811                
1812  \section{Fenfire system model in Peer-to-Peer environment}  \section{Fenfire system model in Peer-to-Peer environment}
1813    
1814  In this section we give a proposal for Fenfire Peer-to-Peer system, which consists  In this section we give a proposal for the Fenfire Peer-to-Peer system, which consists
1815  of several technologies reviewed in this thesis.  Then, we introduce methods for  of several technologies reviewed in this thesis.  Then, we introduce methods for
1816  obtaining Fenfire data from a Peer-to-Peer network.  obtaining Fenfire data from a Peer-to-Peer network.
1817    
# Line 1821  obtaining Fenfire data from a Peer-to-Pe Line 1820  obtaining Fenfire data from a Peer-to-Pe
1820  We emphasize that we prefer \emph{abstraction}  We emphasize that we prefer \emph{abstraction}
1821  level analysis as very recently better and better tightly structured algorithms have been proposed.  level analysis as very recently better and better tightly structured algorithms have been proposed.
1822  Thus, we don't want to bind our system proposal to a specific algorithm definitively as we expect  Thus, we don't want to bind our system proposal to a specific algorithm definitively as we expect
1823  that this development continues. Currently, we see Kademlia \cite{maymounkov02kademlia} as the best algorithm for  that this development continues.
1824    
1825    Currently, we see Kademlia \cite{maymounkov02kademlia} as the best algorithm for
1826  locating data efficiently in the Peer-to-Peer overlay. There are two  locating data efficiently in the Peer-to-Peer overlay. There are two
1827  reasons for this. First, Kademlia's XOR-based distance function is superior  reasons for this. First, Kademlia's XOR-based distance function is superior
1828  over distance functions of other systems (see section 2.3.2). Secondly, Kademlia  over distance functions of other systems (see section 2.3.2). Secondly, Kademlia
# Line 1829  is one of the only tightly structured sy Line 1830  is one of the only tightly structured sy
1830  (e.g., \cite{overneturl, edonkey2kurl, kashmirurl,kato02gisp}), which means that  (e.g., \cite{overneturl, edonkey2kurl, kashmirurl,kato02gisp}), which means that
1831  Kademlia's algorithm is simple and easy to implement.  Kademlia's algorithm is simple and easy to implement.
1832    
1833  On top of Kademlia, we propose the usage of Sloppy hashing \cite{sloppy:iptps03} which  On top of Kademlia, we propose the use of Sloppy hashing \cite{sloppy:iptps03} which
1834  is optimized for the DOLR abstraction of tightly structured overlays. With Sloppy hashing  is optimized for the DOLR abstraction of tightly structured overlays. With Sloppy hashing
1835  we can provide locality properties for the Fenfire system.  we can provide locality properties for the Fenfire system which may be useful
1836    within a small group of working people.
1837    
1838  For better fault tolerance and self-monitoring for Fenfire, we propose techniques  For better fault tolerance and self-monitoring for Fenfire, we propose techniques
1839  presented by Rowston et al. \cite{rowston03controlloingreliability}.  With these  presented by Rowston et al. \cite{rowston03controlloingreliability}.  With these
1840  techniques, we can ensure the performance of the Fenfire system in a highly adverse conditions, such  techniques, we can ensure the performance of the Fenfire system in a highly adverse conditions, such
1841  as sudden network partition, or highly dynamic and heterogeneous environment.  as sudden network partition, or highly dynamic and heterogeneous environment.
1842    
1843  Finally, for more efficient data transfer, we can use variable techniques for this purpose.  Additionally, for more efficient data transfer, we can use variable techniques for this purpose.
1844  For small amounts of data, HTTP can be used \cite{rfc2068}. For large amounts of data, we can use  For small amounts of data, HTTP can be used \cite{rfc2068}. For large amounts of data, we can use
1845  multisource downloads for better efficiency and reliability. Specifically, the technology based  multisource downloads for better efficiency and reliability. Specifically, the technology based
1846  on rateless erasure codes \cite{maymounkov03ratelesscodes} seems very promising.  on rateless erasure codes \cite{maymounkov03ratelesscodes} seems very promising.
1847  Furthermore, multisource downloads can be used for decreasing load of a certain peer, thus avoiding query  Furthermore, multisource downloads can be used for decreasing load of a certain peer, thus avoiding query
1848  hot spots in the system \cite{ratnasamy02routing}. Current client-server implementation of Fenfire uses  hot spots in the system \cite{ratnasamy02routing}. Current client-server implementation of Fenfire uses
1849  standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content  standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content
1850  hash for verifying the integrity of data by recomputing the content hash  hash for verifying the integrity of data. In face of multisource downloads, Fenfire must support
 for Storm block. In face of multisource downloads, Fenfire must support  
1851  tree-based hashes\footnote{With multisource downloads, tree-based hash functions can be used  tree-based hashes\footnote{With multisource downloads, tree-based hash functions can be used
1852  to verify fixed length segments of data. If hash value of data segment is incorrect,  to verify fixed length segments of data. If hash value of data segment is incorrect,
1853  we need only to fetch \emph{segment} of data (instead of whole data) from  we need only to fetch \emph{segment} of data (instead of whole data) from
# Line 1855  data validation. Line 1856  data validation.
1856    
1857  \subsection{Methods}  \subsection{Methods}
1858    
1859  We use the DOLR abstraction of the tightly structured approach since DOLR systems locate data without  In our data lookup methods, we use the DOLR abstraction of the tightly structured approach since DOLR systems locate data without
1860  specifying a storage policy explicitly \cite{rhea03benchmarks}, i.e., each participating peer hosts  specifying a storage policy explicitly \cite{rhea03benchmarks}, i.e., each participating peer hosts
1861  the data they are offering and the overlay maintains only the \emph{pointers} to the data.  the data they are offering and the overlay maintains the \emph{pointers} to the data.
1862  DHT-based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have  Storage systems based on the DHT abstraction, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have
1863  severe problems with load balancing in a highly heterogeneous environment \cite{rao03loadbalancing}. The problem is caused by peers  severe problems with load balancing in a highly heterogeneous environment \cite{rao03loadbalancing}. The problem is caused by peers
1864  which may not be able to store relatively large blocks, assigned randomly by the mapping function of the overlay.  which may not be able to store relatively large blocks, assigned randomly by the mapping function of the overlay.
1865    
1866  For simplicity, we assume that we have resolved the construction of the ''virtual file'' before locating any Storm blocks, i.e.,  For simplicity, we assume that we have resolved the construction of the ''virtual file'' before locating any Storm blocks, i.e.,
1867  when assembling the ''virtual file'' we know all the Storm blocks, which are required to complete the ''virtual file''.  when assembling the ''virtual file'' we know all the Storm blocks, which are required to complete the ''virtual file''.
1868  Also, we don't respond to the security issues related to Peer-to-Peer systems, since there is no working solution  Also, we don't respond to the security issues related to Peer-to-Peer systems, since there is no working solution
1869  available yet. Thus, we either assume that Fenfire has a reliable technique for identifying individual entities, or  available yet. We either assume that Fenfire has a reliable technique for identifying individual entities, or
1870  there are no hostile entities among participating peers, i.e.,  Storm blocks can be identified correctly (e.g., when  there are no hostile entities among participating peers, i.e.,  Storm blocks can be identified correctly (e.g., when
1871  performing searches). In the next subsection, we discuss security problems in more detail.  performing searches). In the next subsection, we discuss security problems in more detail.
1872    
# Line 1919  process of obtaining Storm block from th Line 1920  process of obtaining Storm block from th
1920  Perhaps the most biggest issue in Peer-to-Peer systems is the non-maturity of  Perhaps the most biggest issue in Peer-to-Peer systems is the non-maturity of
1921  security technologies. For the Fenfire system, one security related problem occurs when a user wants to  security technologies. For the Fenfire system, one security related problem occurs when a user wants to
1922  perform a global data lookup with a given pointer; how the user is able to verify  perform a global data lookup with a given pointer; how the user is able to verify
1923  the correctness of the search results, i.e.,  how she or he knows which one is the  the correctness of the search results, e.g., how she or he knows which one is the
1924  correct Storm block ? Another problem related to the Fenfire's  correct Storm block ? Another problem related to the Fenfire's
1925  security is that if a user downloads data from the network to local computer  security is that if a user downloads data from the network to local computer
1926  and after a network disconnection, user wants to verify \emph{off line} the  and after a network disconnection, user wants to verify \emph{off line} the
1927  authenticity of data. Finally, if a data lookup is performed by a user, but there is no reply  authenticity of data. Finally, if a data lookup is performed by a user, but there is no reply
1928  from the Fenfire system, how are we able to know if this was the Spam attack \cite{naor03simpledht},  from the Fenfire system, how are we able to know if this was the Spam attack \cite{naor03simpledht},
1929  or the data really doesn't exist in the system ?    or the data really doesn't exist in the system ?  
1930  However, these problems are not only limited to the Fenfire system as it  These problems, however, are not only limited to the Fenfire system as it
1931  concerns all Peer-to-Peer computer systems.    concerns all Peer-to-Peer computer systems.  
1932    
1933  Obviously, optimal solutions to all security issues would be that digital  Optimal solutions to all security issues would be that digital
1934  signatures are included to every message which are sent to the system or the use of working PKI-based  signatures are included to every message which are sent to the system or the use of working PKI-based
1935  certificate distribution. As security technologies come more mature, we wish to apply these  certificate distribution. As security technologies become more mature, we wish to apply these
1936  technologies with Fenfire, if applicable.  technologies with Fenfire, if applicable.
1937    
1938  \chapter{Conclusions and future work}  \chapter{Conclusions and future work}
# Line 1943  We have summarized open problems in Peer Line 1944  We have summarized open problems in Peer
1944  problems into the three sub-categories: security related problems,  problems into the three sub-categories: security related problems,
1945  performance related problems and miscellaneous problems.  performance related problems and miscellaneous problems.
1946    
1947  Then we gave a brief overview of the Fenfire system and xanalogical storage model. We also  Then we gave a brief overview of the Fenfire system and the xanalogical storage model. We also
1948  described Storm software module.  described Storm software module.
1949    
1950  In the last chapter, we evaluated existing Peer-to-Peer approaches with regard  In the last chapter, we evaluated existing Peer-to-Peer approaches with regard
1951  to Fenfire's needs. We see that the tightly structured approach is the  to Fenfire's needs. Currently, we see that the tightly structured approach as the
1952  best alternative to Fenfire's needs for the following reasons. First, our Storm design uses \emph{semantic-free references}  best alternative to Fenfire's needs for the following reasons.
1953  (block identifiers and pointers) for locating data in distributed  First, both Storm and tightly structured overlays use globally unique identifiers for each data item. Therefore,
1954  networks. As the authors of \cite{balakrishnan03semanticfree},  it is feasible to use tightly structured overlays for \emph{locating} Storm blocks efficiently.
1955  we also agree that references should be semantically-free in next-generation  Second, the unstructured and semantic-free properties of Storm identifiers enables
1956  reference resolution services. Second, by using  the use of general purpose Reference Resolution Services (RRS) \cite{balakrishnan03semanticfree} on
1957  the DOLR abstraction of tightly structured overlay and Sloppy hashing , we can  top of the tightly structured overlay. As the authors of \cite{balakrishnan03semanticfree},
1958  minimize the lack of locality in the tightly structured approach, i.e., we are able to locate nearby  we also agree that references should be semantically-free in next-generation RRS systems.
1959  data without looking up data from distant peers. Third, we believe that issues  Third, we believe that issues related to tightly structured overlays will be solved in
1960  related to tightly structured overlays are solved in the near future, because of  the near future, because of wide and intensive co-operation among research groups.
 wide and intensive co-operation among research groups.  
1961    
1962  Our future work includes a support for searching transclusions and xanalogical  Our future work includes a support for searching transclusions and xanalogical
1963  links in Peer-to-Peer environment. Preliminary analysis have shown  links in Peer-to-Peer environment. Preliminary analysis have shown
# Line 1967  database systems may prove to be useful. Line 1967  database systems may prove to be useful.
1967  regarding Peer-to-Peer and database systems have already been  regarding Peer-to-Peer and database systems have already been
1968  presented in \cite{gribble01p2pdatabase}.    presented in \cite{gribble01p2pdatabase}.  
1969    
 We will implement a Fenfire Peer-to-Peer prototype in the near future.  
   
1970  \bibliographystyle{gradu}  \bibliographystyle{gradu}
1971  \bibliography{progradu}  \bibliography{progradu}
1972  \end{document}  \end{document}

Legend:
Removed from v.1.196  
changed lines
  Added in v.1.197

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26