860 |
information from multiple entities and trust on the majority's opinion. This method requires more messages to be |
information from multiple entities and trust on the majority's opinion. This method requires more messages to be |
861 |
sent to the network while increasing the system load. However, if the Spam attack is combined with the Sybil attack, obviously |
sent to the network while increasing the system load. However, if the Spam attack is combined with the Sybil attack, obviously |
862 |
the previously mentioned solution doesn't work. Naor et al. \cite{naor03simpledht} have proposed a partial solution against Spam attack |
the previously mentioned solution doesn't work. Naor et al. \cite{naor03simpledht} have proposed a partial solution against Spam attack |
863 |
in \emph{faulty} peer environment (not hostile). |
in a \emph{faulty} peer environment (not hostile). |
864 |
|
|
865 |
Traditional overloading of targeted peers is the best known form of distributed Denial of Service attack (DDoS) (see, e.g., \cite{372148}). |
Traditional overloading of targeted peers is the best known form of distributed Denial of Service attack (DDoS) (see, e.g., \cite{372148}). |
866 |
For example, a hostile entity can attempt to burden targeted peers with garbage network packets. As an implication, peers may act |
For example, a hostile entity can attempt to burden targeted peers with garbage network packets. As an implication, peers may act |
889 |
systems \cite{rivest96sdsi}, \cite{spkiworkinggroup}. PKI is a reliable technology for securing |
systems \cite{rivest96sdsi}, \cite{spkiworkinggroup}. PKI is a reliable technology for securing |
890 |
data in rather \emph{static} computing systems, such as the Internet. However, in Peer-to-Peer |
data in rather \emph{static} computing systems, such as the Internet. However, in Peer-to-Peer |
891 |
networks, the problem of key-based security mechanism is the maintenance of the keys as participating |
networks, the problem of key-based security mechanism is the maintenance of the keys as participating |
892 |
peers constantly join and leave the system. These include revocation of keys and new key distribution in hostile |
peers constantly join and leave the system. These include the revocation of keys and the distribution of |
893 |
environment. |
new keys in a hostile environment. |
894 |
|
|
895 |
ConChord \cite{ajmani02conchord} is the first Peer-to-Peer system which has a support for PKI based |
ConChord \cite{ajmani02conchord} is the first Peer-to-Peer system which has a support for PKI based |
896 |
security infrastructure. Still, however, ConChord \cite{ajmani02conchord} is in early phase of development and lacks |
security infrastructure. Still, however, ConChord \cite{ajmani02conchord} is in early phase of development and lacks |
899 |
Peer-to-Peer systems, in which hierarchy is intentionally missing. |
Peer-to-Peer systems, in which hierarchy is intentionally missing. |
900 |
|
|
901 |
For data integrity, on the other hand, there are few working solutions. Cryptographic content hashes |
For data integrity, on the other hand, there are few working solutions. Cryptographic content hashes |
902 |
\cite{fips-sha-1}, their variations \cite{merkle87hashtree} and implementation techniques \cite{mohr02thex}, |
\cite{fips-sha-1}, their variations \cite{merkle87hashtree} and implementation techniques \cite{mohr02thex} |
903 |
are efficient and reliable methods for identifying the integrity of data in Peer-to-Peer systems. One |
are efficient and reliable methods for identifying the integrity of data in Peer-to-Peer systems. One |
904 |
possible application of cryptographic content hashes may be in peer identifier creation process, in which |
possible application of cryptographic content hashes may be in the creation process of peer identifier, in which |
905 |
the IP address of a peer can be verified by the other peer. This is one form of \emph{self-certifying data}. |
the IP address of a peer can be verified by the other peer. This is one form of \emph{self-certifying data}. |
906 |
|
|
907 |
|
|
925 |
Obviously, existance of several types of anonymity often conflicts with other key properties of |
Obviously, existance of several types of anonymity often conflicts with other key properties of |
926 |
Peer-to-Peer systems. Let us consider anonymity and efficient data lookup. In efficient data lookup, we must know |
Peer-to-Peer systems. Let us consider anonymity and efficient data lookup. In efficient data lookup, we must know |
927 |
the peers responsible for given data. Of course, when we know the peers responsible |
the peers responsible for given data. Of course, when we know the peers responsible |
928 |
for the data, the anonymity of peer is lost. Fortunately, there are partial solutions to previously |
for the data, the anonymity of peer is lost. Fortunately, there are partial solutions to these kinds of |
929 |
mentioned situations, such as pseudonymity which is a partial form of anonymity. For instance, pseudonymity can be used for |
situations, such as pseudonymity which is a partial form of anonymity. For instance, pseudonymity can be used for |
930 |
addressing peer-anonymity by providing anonymous-like identifiers to peers (e.g., peer identifiers of a tightly |
addressing peer-anonymity by providing anonymous-like identifiers to peers (e.g., peer identifiers of a tightly |
931 |
structured system). |
structured system). |
932 |
|
|
933 |
Anonymity is widely used in Peer-to-Peer system in which data publication and non-censorship are important properties |
Anonymity is widely used in a Peer-to-Peer system in which data publication and non-censorship are important properties |
934 |
of the system. These include |
of the system. These include |
935 |
Freenet \cite{clarke00freenet}, Publius \cite{pub00}, Free Haven \cite{dingledine00free}, Crowds \cite{reiter98crowds}, |
Freenet \cite{clarke00freenet}, Publius \cite{pub00}, Free Haven \cite{dingledine00free}, Crowds \cite{reiter98crowds}, |
936 |
Tangler \cite{502002} and upcoming Mnet \cite{mneturl}. Forwarding proxies are used in Freenet, Crowds and |
Tangler \cite{502002} and upcoming Mnet \cite{mneturl}. Forwarding proxies are used in Freenet, Crowds and |
940 |
of anonymity. |
of anonymity. |
941 |
|
|
942 |
Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no |
Even if many existing Peer-to-Peer systems are able to provide some of the types of anonymity, there is no |
943 |
such system which is able to provide all kinds of anonymity as listed above. Specifically, the conflicts |
such a system which is able to provide all kinds of anonymity as listed above. Specifically, the conflicts |
944 |
between anonymity and other Peer-to-Peer system properties require more research work. |
between anonymity and other Peer-to-Peer system properties require more research work. |
945 |
|
|
946 |
|
|
970 |
|
|
971 |
Naturally, centralized authorities could be used for the assignment of peer identifiers, but they may not be suitable |
Naturally, centralized authorities could be used for the assignment of peer identifiers, but they may not be suitable |
972 |
for ad hoc Peer-to-Peer environrment and have property of single point of failure. Moreover, distributed peer |
for ad hoc Peer-to-Peer environrment and have property of single point of failure. Moreover, distributed peer |
973 |
identification assignment can be problematic as long as Sybil attack \cite{douceur02sybil} remains unsolved. |
identification assignment can be problematic as long as the Sybil attack \cite{douceur02sybil} remains unsolved. |
974 |
However, there are some partial solutions for controlling the rate at which hostile entity is able to obtain peer |
However, there are some partial solutions for controlling the \emph{rate} at which hostile entity is able to obtain peer |
975 |
identifier, such as crypto-based puzzles \cite{juels99clientpuzzles}. |
identifier, such as crypto-based puzzles \cite{juels99clientpuzzles}. |
976 |
|
|
|
In the end, none of these problems solutions are able to identify hostile entities safely. |
|
|
|
|
|
|
|
977 |
\subsection{Secure query routing} |
\subsection{Secure query routing} |
978 |
|
|
979 |
Much work has been done on secure routing, especially related to tightly structured systems. In |
Much work has been done on secure routing, especially related to tightly structured systems. In |
980 |
\cite{castro02securitystructured} and \cite{castro02securerouting}, authors suggest the usage |
\cite{castro02securitystructured} and \cite{castro02securerouting}, authors suggest the use |
981 |
of constrained routing tables and diverse routes, and the detection of faults during query routing. |
of constrained routing tables and diverse routes, and the detection of faults during data lookup routing. |
982 |
Additionally, authors present in \cite{castro02securerouting} an important aspect of the tightly structured approach with regard |
Additionally, authors present in \cite{castro02securerouting} an important aspect of the tightly structured approach with regard |
983 |
to fault-tolerant query routing: the probability of routing successfully between to arbitrary |
to fault tolerant query routing: the probability of routing successfully between to arbitrary |
984 |
correct peers, when a fraction $f$ of the other peers are faulty or hostile, is only $(1-f)^{h-1}$, where |
correct peers, when a fraction $f$ of the other peers are faulty or hostile, is only $(1-f)^{h-1}$, where |
985 |
$h$ is the number of hops in the overlay. |
$h$ is the number of hops in the overlay. |
986 |
|
|
987 |
Sit and Morris \cite{sit02securitycons} discuss the possibility of allowing query originator |
Sit and Morris \cite{sit02securitycons} discuss the possibility of allowing the query originator |
988 |
to observe lookup progress and cross-check routing tables using random queries. However, their |
to observe lookup progress and cross-check routing tables using random queries. However, their |
989 |
approach is not very efficient, since proposals create a lot of additional network traffic when |
approach is not very efficient, since this method creates lot of additional network traffic when |
990 |
in function. |
in function. |
991 |
|
|
992 |
Additionally, Lynch et al. \cite{lynch02atomicdataaccess} propose a solution to secure routing table |
Additionally, Lynch et al. \cite{lynch02atomicdataaccess} propose a solution for secure routing table |
993 |
maintenance, but their solution seems to have two major problems \cite{castro02securitystructured}. First, |
maintenance, but their solution seems to have two major problems \cite{castro02securitystructured}. First, |
994 |
the solution is very expensive even without faulty or hostile entities. Second, each group of replicas |
the solution is very expensive even without faulty or hostile entities. Second, each group of replicas |
995 |
in their solution must have less than 1/3 of its peers faulty. Thus, this feature results in a low |
in their solution must have less than 1/3 of its peers faulty. Thus, this feature results in a low |
996 |
probability of successful routing. |
probability of successful routing. |
997 |
|
|
998 |
Aspnes et al. in \cite{aspnes02faultrouting} and Kaashoek et al. in \cite{kaashoek03koorde} formally |
Aspnes et al. in \cite{aspnes02faultrouting} and Kaashoek et al. in \cite{kaashoek03koorde} formally |
999 |
prove the lower and upper bounds for space requirements of locating a specific data item in a |
prove the lower and upper bounds for the space requirements of locating a specific data item in a |
1000 |
Peer-to-Peer system. They show that to provide high degree of fault tolerance and efficiency in the system, each |
Peer-to-Peer system. They show that to provide high degree of fault tolerance and efficiency in the system, each |
1001 |
participating peer must maintain average of $O(\log{n})$ neighbors. |
participating peer must maintain average of $O(\log{n})$ neighbors. |
1002 |
|
|
1008 |
|
|
1009 |
Finally, Ratnasamy and Gavoille \cite{ratnasamy02routing, gavoille01routing} list several open problems |
Finally, Ratnasamy and Gavoille \cite{ratnasamy02routing, gavoille01routing} list several open problems |
1010 |
regarding routing in distributed networks. Obviously, more research is required in order to provide secure |
regarding routing in distributed networks. Obviously, more research is required in order to provide secure |
1011 |
data lookup routing possible in Peer-to-Peer networks. |
data lookup routing in Peer-to-Peer networks. |
1012 |
|
|
1013 |
|
|
1014 |
\subsection{Other security threats} |
\subsection{Other security threats} |
1015 |
|
|
1016 |
Ross Lee Graham lists several external threats against Peer-to-Peer networks \cite{grahamp2psecurity}. Most important, |
Ross Lee Graham lists several external threats against Peer-to-Peer networks \cite{grahamp2psecurity}. Most important, |
1017 |
the list includes viruses and trojans. Currently, there are not even partial solutions |
the list includes viruses and trojans. Currently, there are not even partial solutions |
1018 |
to the problems mentioned above. General robustness properties of a Peer-to-Peer system is able to |
to the problems mentioned above. The reason for this is that there are no experience about these kinds of |
1019 |
deal with software failures and hostile attacks, but fault tolerance against external threats is unknown. |
attacks. Possible solution would be a distributed anti-virus software, but much more intensive research is required until |
|
The reason for this is that there are no experience on these kinds of attacks. Possible solution |
|
|
would be distributed anti-virus software, but much more intensive research is required until |
|
1020 |
this kind of solution would be applicable. |
this kind of solution would be applicable. |
1021 |
|
|
1022 |
|
|
1027 |
\subsection{Efficient data lookup} |
\subsection{Efficient data lookup} |
1028 |
|
|
1029 |
The most intensive research in Peer-to-Peer domain has been focused on efficient data lookup methods, |
The most intensive research in Peer-to-Peer domain has been focused on efficient data lookup methods, |
1030 |
especially with the loosely structured approach. In addition to the ''super-peer'' method presented in chapter |
especially with the loosely structured approach. In iterative deepening |
|
2, there has been other improvements also. In iterative deepening |
|
1031 |
\cite{yang02improvingsearch}, multiple BFS searches are initiated |
\cite{yang02improvingsearch}, multiple BFS searches are initiated |
1032 |
with successively larger TTL depth limits, until either the query is satisfied, |
with successively larger TTL depth limits, until either the query is satisfied, |
1033 |
or the maximum depth $D$ has been reached. To perform a data lookup, query |
or the maximum depth $D$ has been reached. To perform a data lookup, the query |
1034 |
originator starts the data lookup with a small TTL value. If the search is not successful, |
originator starts the data lookup with a small TTL value. If the search is not successful, |
1035 |
the query originator increases the TTL value and performs another data lookup. This |
the query originator increases the TTL value and performs another data lookup. This |
1036 |
process is repeated until the desired data is found or the maximum depth $D$ |
process is repeated until the desired data is found or the maximum depth $D$ |
1037 |
has been reached. Expanding ring, proposed by Shenker et al. in \cite{lv02searchreplication}, |
has been reached. Expanding ring, proposed by Shenker et al. in \cite{lv02searchreplication}, |
1038 |
is similar to iterative deepening technique. With these techniques, search |
is similar to the iterative deepening technique. With these techniques, searches |
1039 |
may not be fast when desired data item requires many consecutive flooding rounds. |
may not be fast when desired data item requires several consecutive flooding rounds. |
1040 |
|
|
1041 |
Directed BFS \cite{yang02improvingsearch} optimizes the original |
Directed BFS \cite{yang02improvingsearch} optimizes the original |
1042 |
BFS in a way that a peer selects neighbors with many quality results are reached in the past, |
BFS in a way that a peer selects the neighbors which have provided many quality results in the past, |
1043 |
thereby maintaining the quality of costs and decreasing the amount |
thereby maintaining the quality of costs and decreasing the amount |
1044 |
of messages sent to network. Alpine \cite{alpineurl} and NeuroGrid \cite{joseph02neurogrid} |
of messages sent to network. Alpine \cite{alpineurl} and NeuroGrid \cite{joseph02neurogrid} |
1045 |
are Peer-to-Peer systems which use somewhat similar method when performing data lookups. |
are Peer-to-Peer systems which use somewhat similar method when performing data lookups. |
1047 |
Local indices \cite{yang02improvingsearch} is a variation of active caching. |
Local indices \cite{yang02improvingsearch} is a variation of active caching. |
1048 |
In this scheme, each peer maintains an index over the data of all peers within |
In this scheme, each peer maintains an index over the data of all peers within |
1049 |
$h$ hops of itself, where $h$ is a system-wide variable, called radius of the |
$h$ hops of itself, where $h$ is a system-wide variable, called radius of the |
1050 |
index\footnote{In normal BFS case, the value of $h$ is 0, as peer only has index |
index\footnote{In the normal BFS case, the value of $h$ is 0, as a peer only has index |
1051 |
over its local content.}. Mutual index caching architecture, as proposed in |
over its local content.}. Mutual index caching architecture, as proposed in |
1052 |
\cite{osokine02distnetworks}, is one variation of local indices technique. |
\cite{osokine02distnetworks}, is a variation of local indices technique. |
1053 |
|
|
1054 |
In random walk approach \cite{lv02searchreplication}, a peer forwards query to a |
In the random walk approach \cite{lv02searchreplication}, a peer forwards query to a |
1055 |
randomly selected neighbor. The basic random walk approach |
randomly selected neighbor. The basic random walk approach |
1056 |
has a poor response time but it doesn't generate as much network traffic as |
has a poor response time but it doesn't generate as much network traffic as |
1057 |
the original BFS. As suggested in \cite{lv02searchreplication}, the |
the original BFS. As suggested in \cite{lv02searchreplication}, the |
1058 |
random walk approach can be made more effective by introducing |
random walk approach can be made more effective by introducing |
1059 |
multiple ''walkers''. Freenet \cite{clarke00freenet} uses |
multiple ''walkers''. Freenet \cite{clarke00freenet} uses |
1060 |
random walk searches in query lookups. Indeed, Freenet's query resembles |
random walk searches in data lookups. Freenet's data lookup model resembles |
1061 |
Depth-First-Search (DFS) and peers' routing tables are dynamically built |
Depth-First-Search (DFS) and peers' routing tables are dynamically built |
1062 |
using caching. This is an outcome of Freenet's main design principles, anonymity. |
using caching. This is an outcome of Freenet's main design principles, anonymity. |
1063 |
Another property of the Freenet's data lookup model is that |
Another property of the Freenet's data lookup model is that |
1064 |
it adapts well with varying usage patterns. Improvements to Freenet's data lookup using |
it adapts well with varying usage patterns. Improvements to Freenet's data lookup using |
1065 |
the ''small-world phenomenon'' has been proposed by Zhang et al. \cite{zhang02using}. |
the ''small-world phenomenon'' have been proposed by Zhang et al. \cite{zhang02using}. |
1066 |
|
|
1067 |
Since tightly structured systems have an efficient data lookup at the application level overlay, |
Since tightly structured systems have an efficient data lookup at the application level overlay, |
1068 |
current research efforts are focused on a proximity based data lookup. In the proximity based data lookup, |
current research efforts are focused on the proximity-based data lookup. In the proximity-based data lookup, |
1069 |
peers try to choose entries of routing-tables referring to other peers that are \emph{nearby} in the |
peers try to choose entries of routing-tables referring to other peers that are \emph{nearby} in the |
1070 |
underlying network. In this way, tightly structured systems are able to decrease actual |
underlying network. In this way, tightly structured systems are able to decrease actual |
1071 |
lookup \emph{latency}. CAN \cite{ratnasamy01can}, Kademlia \cite{maymounkov02kademlia}, |
lookup \emph{latency}. CAN \cite{ratnasamy01can}, Kademlia \cite{maymounkov02kademlia}, |
1072 |
Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry} have advanced heuristics for |
Pastry \cite{rowston01pastry} and Tapestry \cite{zhao01tapestry} have advanced heuristics for |
1073 |
the proximity based routing. Additionally, most recent version of Chord uses proximity based |
the proximity-based routing. Additionally, most recent version of Chord uses proximity-based |
1074 |
routing inspired by Karger and Ruhl \cite{karger02findingnearest}. SkipNet \cite{harvey03skipnet1} |
routing, inspired by Karger and Ruhl \cite{karger02findingnearest}. SkipNet \cite{harvey03skipnet1} |
1075 |
uses a combination of proximity and application level overlay routing when performing data |
uses a combination of proximity and application level overlay routing when performing data |
1076 |
lookups. Authors call this feature as a \emph{constrained load balancing}. |
lookups. Authors call this feature as a \emph{constrained load balancing}. |
1077 |
|
|
1078 |
Additional research related to proximity based routing include \cite{karger02findingnearest, hildrum02distributedobject, |
Additional research related to proximity-based routing include \cite{karger02findingnearest, hildrum02distributedobject, |
1079 |
brinkmann02compactplacement, rhea02probabilistic, castro02networkproximity, ng02predicting, pias03lighthouse}. |
brinkmann02compactplacement, rhea02probabilistic, castro02networkproximity, ng02predicting, pias03lighthouse}. |
1080 |
|
|
1081 |
\subsection{Fast and usable search} |
\subsection{Fast and usable search} |
1085 |
is the ability to perform keyword searches (e.g., Google \cite{googleurl}). Currently, only loosely |
is the ability to perform keyword searches (e.g., Google \cite{googleurl}). Currently, only loosely |
1086 |
structured systems are able to carry out this requirement. Unfortunately, as discussed in this text, |
structured systems are able to carry out this requirement. Unfortunately, as discussed in this text, |
1087 |
the data lookup model of the loosely structured approach doesn't scale. Thus, research efforts have |
the data lookup model of the loosely structured approach doesn't scale. Thus, research efforts have |
1088 |
been focused on tightly structured systems. The main problem with tightly structured systems is the |
been focused towards tightly structured systems. The main problem with tightly structured systems is the |
1089 |
fact that tightly structured algorithms perform data lookups based on a globally unique identifier (key). |
fact that tightly structured algorithms perform data lookups based on a globally unique identifier (key). |
1090 |
|
|
1091 |
Recent study has been focused on the feasibility of Peer-to-Peer Web-like indexing and searching |
Recent study has been focused on the feasibility of Peer-to-Peer Web-like indexing and searching |
1092 |
on top of tightly structured overlays \cite{li03feasibility} . Authors argue, that it is possible to implement |
on top of tightly structured overlays \cite{li03feasibility} . Authors argue, that it is possible to implement |
1093 |
Peer-to-Peer Web-like search with certain compromises. First, Peer-to-Peer search engine may need to |
Peer-to-Peer Web-like search with certain compromises. First, Peer-to-Peer search engine may need to |
1094 |
decrease the result quality in order to make searching more efficient. Second, Peer-to-Peer systems must |
decrease the result quality in order to make searching more efficient. Second, Peer-to-Peer systems must |
1095 |
observe the properties of underlying network for better performance. |
consult the properties of underlying network for better performance. |
1096 |
|
|
1097 |
Some studies have been concentrated on SQL-like queries \cite{harren02complex} |
Some studies have been concentrated on SQL-like queries \cite{harren02complex} |
1098 |
in tightly structured overlays. Other approaches include adaption of the data lookup model of the loosely |
in tightly structured overlays. Other approaches include adaption of the data lookup model of the loosely |
1112 |
in \cite{li03feasibility} use Gap compression \cite{wittengigabytes}, Adaptive Set Intersection \cite{338634} |
in \cite{li03feasibility} use Gap compression \cite{wittengigabytes}, Adaptive Set Intersection \cite{338634} |
1113 |
and clustering with their search optimizations. |
and clustering with their search optimizations. |
1114 |
|
|
|
|
|
1115 |
While it is expected that web-like searches can be layered on a top of tightly structured overlay, much |
While it is expected that web-like searches can be layered on a top of tightly structured overlay, much |
1116 |
more research is required to make indexing and searching more efficient. |
more research is required to make indexing and searching more efficient. |
1117 |
|
|
1130 |
environments. Furthermore, proposed tightly structured overlays are configured statically to achieve |
environments. Furthermore, proposed tightly structured overlays are configured statically to achieve |
1131 |
the desired reliability even in a uncommon and adverse environment \cite{rowston03controlloingreliability}. |
the desired reliability even in a uncommon and adverse environment \cite{rowston03controlloingreliability}. |
1132 |
The most important factor for future research is to get real-life experiences from tightly structured |
The most important factor for future research is to get real-life experiences from tightly structured |
1133 |
systems, when there are frequent joins and leaves in the system. Some research has been done already in this area. |
systems, when there are frequent joins and leaves in the system. |
1134 |
|
|
1135 |
A concept of ''half-life'' was introduced by Liben-Nowell \cite{libennowell01observations} since Peer-to-Peer |
The concept of ''half-life'' was introduced by Liben-Nowell \cite{libennowell01observations} since Peer-to-Peer |
1136 |
system is \emph{never} in the ''ideal'' state as it is continiously evolving system. Half-life is defined |
system is \emph{never} in the ''ideal'' state as Peer-to-Peer system is continiously evolving system. Half-life is defined |
1137 |
as follows: let there be $N$ live peers at time $t$. The doubling from time $t$ is the time that pass before |
as follows: let there be $N$ live peers at time $t$. The doubling from time $t$ is the time that pass before |
1138 |
$N$ new additional peers arrive into the system. The halving time from time $t$ is the time |
$N$ new additional peers arrive into the system. The halving time from time $t$ is the time |
1139 |
required for half of the living peers at time $t$ to leave the system. The half-life from |
required for half of the living peers at time $t$ to leave the system. The half-life from |
1159 |
a serious problem of tightly structured overlays in face of performance and load balancing. Measurement study |
a serious problem of tightly structured overlays in face of performance and load balancing. Measurement study |
1160 |
by Saroiu et al. show that there is a extreme heterogeneity among participating peers in already deployed Peer-to-Peer |
by Saroiu et al. show that there is a extreme heterogeneity among participating peers in already deployed Peer-to-Peer |
1161 |
systems \cite{saroiu02measurementstudyp2p}. Symphony \cite{gurmeet03symphony} seems to be the first tightly structured overlay system |
systems \cite{saroiu02measurementstudyp2p}. Symphony \cite{gurmeet03symphony} seems to be the first tightly structured overlay system |
1162 |
which supports heterogeneity. Zhao et al. have proposed a secondary layer on top of structured overlay |
which supports heterogeneity. Zhao et al. have proposed a secondary layer on top of a structured overlay |
1163 |
to support heterogeneity better \cite{zhao02brocade}. |
to support heterogeneity better \cite{zhao02brocade}. |
1164 |
|
|
1165 |
Research has been done on self-organization. Ledlie et al. propose techniques for forming and maintaining |
Research has been done on self-organization. Ledlie et al. propose techniques for forming and maintaining |
1166 |
groups in highly dynamic environment \cite{ledlie02selfp2p}. Unfortunately their work relies on the idea that |
groups in a highly dynamic environment \cite{ledlie02selfp2p}. Unfortunately their work relies on the idea that |
1167 |
participating peers would create multiple hierarchical groups. It is not clear whether this approach |
participating peers would create multiple hierarchical groups. It is not clear whether this approach |
1168 |
is fault tolerant and suitable for a Peer-to-Peer environment. More promising work has been done by Rowston et al. |
is fault tolerant and suitable for Peer-to-Peer environment. More promising work has been done by Rowston et al. |
1169 |
in \cite{rowston03controlloingreliability}. Authors propose techniques for self-tuning, dealing with |
in \cite{rowston03controlloingreliability}. Authors propose techniques for self-tuning, dealing with |
1170 |
uncommon conditions (e.g., network partition and high failure rates). Moreover, authors argue that |
uncommon conditions (e.g., network partition and high failure rates). Moreover, authors argue that |
1171 |
with these techniques, the concerns over tightly structured overlay maintenance costs are no more |
with these techniques, the concerns over tightly structured overlay maintenance costs are no more |
1172 |
an open issue. |
an open issue. |
1173 |
|
|
1174 |
Finally, little research has been done regarding self-monitoring and data availability. Zhang et al. |
Finally, little research has been done regarding self-monitoring and data availability. Zhang et al. |
1175 |
describe an arbitrary data structure on top of a tightly structured overlay \cite{zhang03somo}. They |
describe an arbitrary data structure on top of a tightly structured overlay \cite{zhang03somo}. Authors |
1176 |
call their proposal as a \emph{data overlay}, since it supports several fundamental data structures. |
call their technique as a \emph{data overlay}, since it supports several fundamental data structures. |
1177 |
Authors use this data overlay to build Self-Organized Meta data Overlay (SOMO), which can be used |
Authors have used this data overlay when building a Self-Organized Meta data Overlay (SOMO), which can be used |
1178 |
for monitoring the health of a tightly structured overlay. The fault tolerance of SOMO itself is currently |
for monitoring the health of a tightly structured overlay. The fault tolerance of SOMO itself is currently |
1179 |
unknown. |
unknown. |
1180 |
|
|
1188 |
All existing Peer-to-Peer systems have rather different interfaces even though they have common properties and |
All existing Peer-to-Peer systems have rather different interfaces even though they have common properties and |
1189 |
components. More important, all existing Peer-to-Peer systems are incompatible with each other. One |
components. More important, all existing Peer-to-Peer systems are incompatible with each other. One |
1190 |
of the most important area of future research is to create common programming abstractions, i.e., |
of the most important area of future research is to create common programming abstractions, i.e., |
1191 |
interfaces, design patters and frameworks. Also, equal benchmarks are needed for comparing |
interfaces, design patters and frameworks. Also, benchmarks are needed for comparing |
1192 |
different algorithms. Recently, there have been few proposals towards common programming |
different algorithms equally. Recently, there have been few proposals towards common programming |
1193 |
guidelines. This list includes \cite{zhao03api, frise02p2pframework, babaoglu02anthill}. |
guidelines. This list includes \cite{zhao03api, frise02p2pframework, babaoglu02anthill}. |
1194 |
Early experiments with Peer-to-Peer benchmarking include \cite{ratnasamy02routing, rhea03benchmarks}. |
Early experiments with Peer-to-Peer benchmarking include \cite{ratnasamy02routing, rhea03benchmarks}. |
1195 |
|
|
1196 |
\subsection{Social behavior} |
\subsection{Social behavior} |
1197 |
|
|
1198 |
Frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate. Another belief |
Frequent assumption in Peer-to-Peer systems is that peers are willing to cooperate. Another belief |
1199 |
is that all peers would behave equally, i.e., all peers both consume and contribute resources. |
is that all peers would behave equally, i.e., all peers both consume and contribute services. |
1200 |
However, these assumptions are not true as several studies show \cite{saroiu02measurementstudyp2p, |
However, these assumptions are not true as several studies show \cite{saroiu02measurementstudyp2p, |
1201 |
oram01harnessingpower, hearn02mojonation}. Peers rather consume than contribute and peers are |
oram01harnessingpower, hearn02mojonation}. Peers rather consume than contribute and peers are |
1202 |
unwilling to cooperate. |
unwilling to cooperate. |
1217 |
|
|
1218 |
Very little research has been done on simulating a Peer-to-Peer system. Presumably, this |
Very little research has been done on simulating a Peer-to-Peer system. Presumably, this |
1219 |
is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very |
is due to complex nature of Peer-to-Peer system, which makes comprehensive simulations very |
1220 |
difficult. Floyd et al. has been studying the simulation of the Internet in \cite{504642}. Authors |
difficult. Floyd et al. have been studying the simulation of the Internet in \cite{504642}. Authors |
1221 |
state that simulating the Internet is very challenging task, because of its heterogeneity |
state that simulating the Internet is very challenging task, because of its heterogeneity |
1222 |
and rapid change. Obviously, these factors exist also in Peer-to-Peer systems even with higher |
and rapid change. Obviously, these factors exist also in Peer-to-Peer systems even with higher |
1223 |
rates. |
rates. |
1224 |
|
|
1225 |
As long as comprehensive simulations of a Peer-to-Peer systems are lacking, we cannot make any detailed |
As long as comprehensive simulations of a Peer-to-Peer systems are lacking, we cannot make any detailed |
1226 |
analysis on general properties of Peer-to-Peer system such as usage patterns. However, we can assume |
analysis on general properties of a Peer-to-Peer system, such as usage patterns. However, we can assume |
1227 |
that, e.g., query keywords follow the Zipf-like distribution \cite{breslau98implications} both in the |
that, e.g., query keywords follow the Zipf-like distribution \cite{breslau98implications} both in the |
1228 |
Internet and in Peer-to-Peer systems. |
Internet and in Peer-to-Peer systems. |
1229 |
|
|
1552 |
\parbox{90pt}{Locating Peer-to-Peer network} & |
\parbox{90pt}{Locating Peer-to-Peer network} & |
1553 |
\parbox{110pt}{How old peers or new peers are able to locate Peer-to-Peer network, if it exists} & |
\parbox{110pt}{How old peers or new peers are able to locate Peer-to-Peer network, if it exists} & |
1554 |
\parbox{110pt}{Servers maintaining online peers (e.g. gnutellahosts.com), peer's history information} & |
\parbox{110pt}{Servers maintaining online peers (e.g. gnutellahosts.com), peer's history information} & |
1555 |
\parbox{110pt}{Depends on implementation and purpose of the system, for desktop based system there are working solutions, for mobile ad hoc networks more research is needed (Mobile ad hoc |
\parbox{110pt}{Depends on implementation and purpose of the system, for a desktop system there are working solutions, for mobile ad hoc networks more research is needed (Mobile ad hoc |
1556 |
networks (MANETs) can be only connected through radio resource interface, i.e., peers which are in same geographical area)} |
networks (MANETs) can be only connected through radio resource interface, i.e., peers which are in same geographical area)} |
1557 |
\\ \hline |
\\ \hline |
1558 |
|
|
1575 |
The Fenfire project \cite{fenfireurl} is an effort to build a location transparent, hyperstructured desktop |
The Fenfire project \cite{fenfireurl} is an effort to build a location transparent, hyperstructured desktop |
1576 |
environment. Fenfire uses xanalogical storage model \cite{ted-xu-model} as a basis for hyperstructured |
environment. Fenfire uses xanalogical storage model \cite{ted-xu-model} as a basis for hyperstructured |
1577 |
media. Fenfire deploys innovative user interfaces for displaying data to the end users. All data in the Fenfire |
media. Fenfire deploys innovative user interfaces for displaying data to the end users. All data in the Fenfire |
1578 |
is stored in a unified format, blocks. This should allow making references between data easier and more |
is stored in a unified format, i.e., blocks. This features should allow making references between \emph{any} |
1579 |
seamlessly interoperating than in other systems. For location transparency in a distributed system, Fenfire |
data easier and more seamlessly interoperating than in other systems. For location transparency in a distributed system, Fenfire |
1580 |
uses Peer-to-Peer network for locating and fetching blocks. |
uses Peer-to-Peer network for locating and fetching blocks. |
1581 |
|
|
1582 |
Fenfire is a free software and it is licensed under GNU LGPL. Fenfire was formerly also a partial implementation |
Fenfire is a free software and it is licensed under GNU LGPL. Fenfire was formerly also a partial implementation |
1583 |
of the ZigZag\texttrademark --structure, which was originally invented |
of the ZigZag\texttrademark -- structure, which has been originally invented |
1584 |
by Ted Nelson. Now, however, Fenfire uses Resource Description Framework (RDF) \cite{w3rdfurl} |
by Ted Nelson. Now, however, Fenfire uses Resource Description Framework (RDF) \cite{w3rdfurl} |
1585 |
for representing internal data structures and their relationships. |
for representing internal data structures and their relationships. |
1586 |
|
|
1588 |
|
|
1589 |
\begin{itemize} |
\begin{itemize} |
1590 |
\item \textbf{Storm}: a distributed storage module for storing arbitrary data items |
\item \textbf{Storm}: a distributed storage module for storing arbitrary data items |
1591 |
\item \textbf{Navidoc}: an UML based tool for generating software documentation |
\item \textbf{Navidoc}: an UML-based tool for generating software documentation |
1592 |
\item \textbf{Alph}: a xanalogical hypertext built upon Storm storage model |
\item \textbf{Alph}: a xanalogical hypertext built upon Storm storage model |
1593 |
\item \textbf{GLMosaicText}: a flexible OpenGL interface for font manipulation |
\item \textbf{GLMosaicText}: a flexible OpenGL interface for font manipulation |
1594 |
\item \textbf{CallGL}: a wrapping library used for OpenGL calls |
\item \textbf{CallGL}: a wrapping library used for OpenGL calls |
1595 |
\item \textbf{LibVob}: a graphic library used for creating navigation interfaces in complex data views |
\item \textbf{LibVob}: a graphic library used for creating navigation interfaces in complex data views |
1596 |
\end{itemize} |
\end{itemize} |
1597 |
|
|
1598 |
In this thesis, we focus on Storm and Alph modules, since they are the foundation of Fenfire's |
|
1599 |
Peer-to-Peer functionality. If not otherwise mentioned, we use term 'Storm' when referring to both |
For location transparency in the Fenfire system, Storm software module |
|
Storm and Alph software modules. For location transparency in the Fenfire system, Storm software module |
|
1600 |
must have a support for Peer-to-Peer functionality as it provides low-level data storage operations |
must have a support for Peer-to-Peer functionality as it provides low-level data storage operations |
1601 |
in the Fenfire system. |
in the Fenfire system. Therefore, we focus on Storm and Alph modules, since they are the foundation of Fenfire's |
1602 |
|
Peer-to-Peer functionality. If not otherwise mentioned, we use term 'Storm' when referring to both |
1603 |
|
Storm and Alph software modules. |
1604 |
|
|
1605 |
|
|
1606 |
\section{Xanalogical storage model} |
\section{Xanalogical storage model} |
1612 |
permanent, globally unique identifier. For instance, let's consider the following |
permanent, globally unique identifier. For instance, let's consider the following |
1613 |
scenario: ''the character 'D' typed by Janne Kujala on 10/8/97 8:37:18''. In this |
scenario: ''the character 'D' typed by Janne Kujala on 10/8/97 8:37:18''. In this |
1614 |
example, when character 'D' is first typed in, xanalogical storage model |
example, when character 'D' is first typed in, xanalogical storage model |
1615 |
acquires a permanent identifier for that character and retains it when character |
acquires a permanent identifier for that character and retains it when the character |
1616 |
is copied to different document. Thus, the identifier distinguishes the character from |
is copied to different document. Thus, the identifier distinguishes the character from |
1617 |
all similar characters typed in independently\footnote{Xanalogical storage model |
all similar characters typed in independently\footnote{Xanalogical storage model |
1618 |
is not limited to text. It can support arbitrary data, e.g., pixels of picture or |
is not limited to text. It can support arbitrary data, e.g., pixels of picture or |
1619 |
frames of video.}. The connectivity in the xanalogical storage model between data content |
frames of video.}. The connectivity in xanalogical storage model between data content |
1620 |
is more substantial than in other models; a link is shown between any two data contents |
is more substantial than in other models; a link is shown between any two data contents |
1621 |
containing a specific \emph{fluid media unit} (e.g., a character) that the link connects. |
containing a specific \emph{fluid media unit} (e.g., a character) that the link connects. |
1622 |
In practice, however, xanalogical storage model uses \emph{spans}, ranges of consecutive |
In practice, however, xanalogical storage model uses \emph{spans}, ranges of consecutive |
1623 |
fluid media units to perform storage operations. This is done for better performance as |
fluid media units to perform storage operations. This is done for better performance as |
1624 |
doing expensive operations for every fluid media unit is not efficient. As a implication, |
doing expensive operations for every fluid media unit is not efficient. Xanalogical |
1625 |
the xanalogical storage model stores fluid media units to append-only \emph{scrolls}. |
storage model stores fluid media units to append-only \emph{scrolls}. |
1626 |
|
|
1627 |
\emph{Enfilade} can be considered as a ''virtual file'' (or part of one), which is a list |
An \emph{enfilade} can be considered as a ''virtual file'' (or part of one), which is a list |
1628 |
of fluid media content. In the xanalogical storage model, links between content are external |
of fluid media content. In xanalogical storage model, links between content are external |
1629 |
and bidirectional. Xanalogical link is an \emph{association} of two enfilades, such as an |
and bidirectional. Xanalogical link is an \emph{association} of two enfilades, such as an |
1630 |
annotation to a specific part of a another document. \emph{Transclusion} is an inclusion in |
annotation to a specific part of a another document. \emph{Transclusion} is an inclusion in |
1631 |
enfilade of contents already used in another enfilade, i.e., current fluid media is copied into |
enfilade of contents already used in another enfilade, i.e., current fluid media is copied into |
1632 |
different data contents. By using this mechanism, a system implementing the xanalogical storage model |
different data content. By using this mechanism, a system implementing xanalogical storage model |
1633 |
is able to show all data content that share the same fluid media with current data content |
is able to show all data content that share the same fluid media with current data content |
1634 |
(e.g., all documents containing current document's text). Figure \ref{fig:xanalogical_model} |
(e.g., all documents containing current document's text). Figure \ref{fig:xanalogical_model} |
1635 |
illustrates xanalogical storage model with documents, text and characters. |
illustrates xanalogical storage model with documents, text and characters. |
1658 |
SHA-1 \cite{fips-sha-1} is used for verifying the integrity of Storm data blocks. Storm |
SHA-1 \cite{fips-sha-1} is used for verifying the integrity of Storm data blocks. Storm |
1659 |
blocks have much in common with regular files, except that Storm blocks are \emph{immutable} as |
blocks have much in common with regular files, except that Storm blocks are \emph{immutable} as |
1660 |
any change to the byte sequence would change block's hash value (globally unique |
any change to the byte sequence would change block's hash value (globally unique |
1661 |
identifier). This mechanism creates a basis for implementing xanalogical storage model in |
identifier). This mechanism creates a basis for implementing xanalogical storage model |
1662 |
in Fenfire system. Figure \ref{fig:storm_model} illustrates simplified Storm storage model. |
in the Fenfire system. Figure \ref{fig:storm_model} illustrates simplified Storm storage model. |
1663 |
|
|
1664 |
\begin{figure} |
\begin{figure} |
1665 |
\centering |
\centering |
1680 |
(URN) \cite{rfc2396}. Pointer itself doesn't contain any data, it is rather a \emph{concept} of |
(URN) \cite{rfc2396}. Pointer itself doesn't contain any data, it is rather a \emph{concept} of |
1681 |
data. Pointers are created automatically by Storm and each pointer is |
data. Pointers are created automatically by Storm and each pointer is |
1682 |
associated with a collection of \emph{pointer blocks}. Pointer block has a single |
associated with a collection of \emph{pointer blocks}. Pointer block has a single |
1683 |
target for the pointer. In figure \ref{fig:storm_model}, we present the overall |
target for the pointer. In figure \ref{fig:storm_model}, we show the overall |
1684 |
pointer creation process. Pointer block may contain zero or more obsoleted |
pointer creation process. Pointer block may contain zero or more obsoleted |
1685 |
pointer blocks, i.e., when a new version of scroll block is created, it supersedes |
pointer blocks, i.e., when a new version of scroll block is created, it supersedes |
1686 |
one older version which has been created in the past. The most current pointer |
one older version which has been created in the past. The most current pointer |
1698 |
|
|
1699 |
\chapter{Evaluation of Peer-to-Peer for Fenfire} |
\chapter{Evaluation of Peer-to-Peer for Fenfire} |
1700 |
|
|
1701 |
In this chapter we evaluate Fenfire in a Peer-to-Peer environment. |
In this chapter we evaluate Fenfire in Peer-to-Peer environment. |
1702 |
We start by giving a problem overview when considering Fenfire in a Peer-to-Peer |
We start by giving a problem overview when considering Fenfire in Peer-to-Peer |
1703 |
environment. We define Fenfire's special needs and evaluate existing |
environment. We define Fenfire's special needs and evaluate existing |
1704 |
Peer-to-Peer approaches in light of these requirements. After that, we propose a system |
Peer-to-Peer approaches in light of these requirements. After that, we propose a system |
1705 |
model for Fenfire and present simple methods to perform data |
model for Fenfire and present simple methods to perform data |
1706 |
lookups in a Peer-to-Peer environment. In the end of this chapter, we discuss possible problems of using Fenfire |
lookups in Peer-to-Peer environment. In the end of this chapter, we discuss possible problems of using Fenfire |
1707 |
in a Peer-to-Peer environment. |
in Peer-to-Peer environment. |
1708 |
|
|
1709 |
|
|
1710 |
\section{Problem overview} |
\section{Problem overview} |
1711 |
|
|
1712 |
As already mentioned in chapter 4, a xanalogical document is a ''virtual |
As already mentioned in chapter 4, xanalogical document is a ''virtual |
1713 |
file'', in which parts of the document are fetched from a |
file'', in which parts of the document are fetched from a |
1714 |
\emph{global} data repository. Thus, system implementing the xanalogical storage model \emph{must} |
\emph{global} data repository. Thus, system implementing xanalogical storage model \emph{must} |
1715 |
support global data lookups efficiently in order to assemble the ''virtual file'' |
support global data lookups efficiently in order to assemble the ''virtual file'' |
1716 |
from fragments of data. |
from fragments of data. |
1717 |
|
|
1718 |
In the xanalogical storage model, each fragment of data is identified by a globally |
In xanalogical storage model, each fragment of data is identified by a globally |
1719 |
unique identifier. In the Fenfire system, data fragments are scroll blocks generated by Storm storage module. |
unique identifier. In the Fenfire system, data fragments are scroll blocks generated by Storm storage module. |
1720 |
As we discussed already in chapter 4, Fenfire's Storm design |
As we discussed already in chapter 4, Fenfire's Storm design |
1721 |
uses SHA-1 \cite{fips-sha-1} hash over the contents of a scroll block for creating globally unique |
uses SHA-1 \cite{fips-sha-1} hash over the contents of a scroll block for creating globally unique |
1722 |
identifiers for each scroll block. In our scenario, fragments of data is distributed |
identifiers for each scroll block. In our scenario, fragments of data is distributed |
1723 |
throughout the Peer-to-Peer overlay. We want that user operations in Fenfire are location transparent. |
throughout the Peer-to-Peer overlay network. We want that user operations in Fenfire are location transparent. |
1724 |
Therefore, our task is to locate and fetch (i.e. obtain) \emph{all} Storm scroll blocks, associated to a specific ''virtual |
Therefore, our task is to locate and fetch (i.e. obtain) \emph{all} Storm scroll blocks, associated to a specific ''virtual |
1725 |
file'' from the Peer-to-Peer overlay as efficiently as possible. In addition to the |
file'' from the Peer-to-Peer overlay as efficiently as possible. In addition to the |
1726 |
\emph{direct} scroll block obtaining using globally unique identifier of Storm scroll block, |
\emph{direct} scroll block obtaining using globally unique identifier of Storm scroll block, |
1727 |
we also must support the \emph{indirect} obtaining of Storm scroll block using pointer blocks. |
we also must support the \emph{indirect} obtaining of Storm scroll block using the pointer blocks. |
1728 |
|
|
1729 |
Obviously, our objectives are simple but yet hard to fulfill. First, as a prerequisite |
Our objectives are simple but yet hard to fulfill. First, as a prerequisite |
1730 |
to implementing xanalogical storage model in a Peer-to-Peer environment, a system |
to implementing xanalogical storage model in Peer-to-Peer environment, a system |
1731 |
supporting data lookups must be able to perform \emph{global} scale lookups. Thus, |
supporting data lookups must be able to perform \emph{global} scale lookups. Thus, |
1732 |
we must be able to locate and fetch Storm block, if it exists in the |
we must be able to obtain the Storm block, if it exists in the |
1733 |
Peer-to-Peer overlay. Second, data lookups have to be efficient, since constructing |
Peer-to-Peer overlay. Second, data lookups have to be efficient, since constructing |
1734 |
one ''virtual file'' may need obtaining several Storm blocks, which are distributed |
one ''virtual file'' may need obtaining several Storm blocks, which are distributed |
1735 |
randomly throughout the overlay; if not efficient, construction of the ''virtual file'' |
randomly throughout the overlay; if not efficient, construction of the ''virtual file'' |
1736 |
may take reasonable amount of time while rendering system very unusable. Third, Peer-to-Peer |
may take reasonable amount of time while rendering system very unusable. Third, Peer-to-Peer |
1737 |
infrastructure has to be scalable and robust against hostile attacks. |
infrastructure has to be scalable and fault tolerant against hostile attacks. |
1738 |
|
|
1739 |
Some research regarding to these problem has been made by Lukka et al. |
Some research regarding to these problems have been made by Lukka et al. |
1740 |
\cite{lukka02freenetguids}. Authors' work is mainly based on the insight of implementing the |
\cite{lukka02freenetguids}. Authors' work is mainly based on the insight of implementing |
1741 |
xanalogical storage model in a Peer-to-Peer environment with globally unique identifiers. Lukka et al. |
xanalogical storage model in Peer-to-Peer environment with globally unique identifiers. Lukka et al. |
1742 |
use Freenet \cite{clarke00freenet} as an example Peer-to-Peer system supporting |
use Freenet \cite{clarke00freenet} as an example Peer-to-Peer system supporting |
1743 |
globally unique identifiers. The work presented in this thesis extends their work by |
globally unique identifiers. The work presented in this thesis extends their work by |
1744 |
evaluating different Peer-to-Peer systems more extensively to Fenfire's needs. |
evaluating different Peer-to-Peer systems more extensively to Fenfire's needs. |
1745 |
|
|
1746 |
Additionally, related to non-xanalogical hypermedia systems, Bouving |
Additionally, related to non-xanalogical hypermedia systems, Bouving |
1747 |
\cite{bouvin02openhypermedia} has done initial work regarding ways in which |
\cite{bouvin02openhypermedia} has done initial work regarding ways in which |
1748 |
Peer-to-Peer can be used in non-xanalogical hypermedia systems. Thompson and de Roure |
Peer-to-Peer can be used in non-xanalogical hypermedia systems. Thompson and de Roure |
1749 |
\cite{thompson01hypermedia} have studied locating documents and links in Peer-to-Peer |
\cite{thompson01hypermedia} have studied locating documents and links in Peer-to-Peer |
1750 |
environment. At the Hypertext '02 panel, moderated by Wiil \cite{wiil02p2phypertext}, |
environment. At the Hypertext '02 panel, moderated by Wiil \cite{wiil02p2phypertext}, |
1753 |
|
|
1754 |
\section{Evaluation of Peer-to-Peer approaches with regard to Fenfire} |
\section{Evaluation of Peer-to-Peer approaches with regard to Fenfire} |
1755 |
|
|
1756 |
In chapter 2, we discussed main differences between loosely and tightly structured |
In chapter 2, we discussed main differences between the loosely and the tightly structured |
1757 |
approaches. As stated, the most significant difference is that the tightly structured |
approach. As stated, the most significant difference is that the tightly structured |
1758 |
approach has logarithmical properties in all internal operations, while the loosely |
approach has logarithmical properties in all internal operations, while the loosely |
1759 |
structured approach doesn't always have even linear properties. Furthermore, the |
structured approach doesn't always have even linear properties. Furthermore, the |
1760 |
data lookup model of tightly structured overlay scales much better than loosely |
data lookup model of the tightly structured overlay scales much better than in loosely |
1761 |
structured overlays; tightly structured overlay supports global data lookups |
structured overlays; the tightly structured overlay supports global data lookups |
1762 |
in the overlay, whereas the data lookup model of the loosely structured approach |
in the overlay, whereas the data lookup model of the loosely structured approach |
1763 |
is limited to certain area of the overlay\footnote{The area depends on where the query |
is limited to a certain area of the overlay\footnote{The area depends on where the query |
1764 |
originator is located in the overlay.}. |
originator is located in the overlay.}. |
1765 |
|
|
1766 |
For Fenfire's special needs for \emph{locating} data, an important advantage of the |
For Fenfire's needs for \emph{locating} data, an important advantage of the |
1767 |
tightly structured approach over the loosely structured approach is that tightly |
tightly structured approach over the loosely structured approach is that tightly |
1768 |
structured systems use location-independent, globally unique identifiers for |
structured systems use location-independent, globally unique identifiers for |
1769 |
identifying data in the system. Indeed, this |
identifying data in the system. Indeed, this |
1770 |
feature is almost analogical to Fenfire's (and xanalogical storage model's) way of |
feature is similar to Fenfire's (and xanalogical storage model's) way of |
1771 |
handling data. Another key feature of tightly structured overlays is that they are able |
handling data. Another key feature of tightly structured overlays is that they are able |
1772 |
to provide general purpose \emph{interface} for Reference Resolution Services (RRS)\footnote{ |
to provide general purpose \emph{interface} for Reference Resolution Services (RRS)\footnote{ |
1773 |
Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system in the Internet.} |
Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system in the Internet.} |
1774 |
\cite{balakrishnan03semanticfree}. Authors argue that next generation RRS must be |
\cite{balakrishnan03semanticfree}. Authors argue that next generation RRS must be |
1775 |
application-independent and references itself should be \emph{unstructured} and |
application-independent and references itself should be \emph{unstructured} and |
1776 |
\emph{semantically free}. Finally, as said, with tightly structured systems, it is feasible to |
\emph{semantically free}. Finally, as said, with tightly structured systems it is feasible to |
1777 |
perform \emph{global} data lookups in the overlay. To summarize, these aspects may be the most important features |
perform \emph{global} data lookups in the overlay. To summarize, these aspects may be the most important features |
1778 |
of Peer-to-Peer infrastructure with regard to Fenfire as a distributed, location transparent hypermedia system. |
of Peer-to-Peer infrastructure with regard to Fenfire as a distributed, location transparent hypermedia system. |
1779 |
Thus, we see the tightly structured approach as the best alternative to locate data in a Peer-to-Peer |
Thus, we see the tightly structured approach as the best alternative to locate data in Peer-to-Peer |
1780 |
environment. |
environment. |
1781 |
|
|
1782 |
Once located, we can use regular TCP/IP-protocols, such as Hypertext Transfer protocol (HTTP) |
Once located, we can use regular TCP/IP-protocols, such as Hypertext Transfer protocol (HTTP) |
1783 |
\cite{rfc2068} for \emph{fetching} Storm blocks from the overlay. However, HTTP-protocol may |
\cite{rfc2068} for \emph{fetching} Storm blocks from the overlay. However, HTTP-protocol may |
1784 |
not be optimal when obtaining large amounts of data from a Peer-to-Peer network (e.g., |
not be a optimal solution when obtaining large amounts of data from a Peer-to-Peer network (e.g., |
1785 |
videos, images or music). In this case, multisource downloads can be very useful |
videos, images or music). In this case, multisource downloads can be very useful |
1786 |
for better efficiency \cite{maymounkov03ratelesscodes, bittorrenturl}. Furthermore, |
for better efficiency \cite{maymounkov03ratelesscodes, bittorrenturl}. Furthermore, |
1787 |
multisource downloads can be used for decreasing load of a certain peer, thus avoiding query |
multisource downloads can be used for decreasing load of a certain peer, thus avoiding query |
1789 |
standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content |
standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content |
1790 |
hash for verifying the integrity of data by recomputing the content hash |
hash for verifying the integrity of data by recomputing the content hash |
1791 |
for a scroll block. In face of multisource downloads, Fenfire must support |
for a scroll block. In face of multisource downloads, Fenfire must support |
1792 |
tree-based hashes\footnote{With multisource downloads, tree based hash functions can be used |
tree-based hashes\footnote{With multisource downloads, tree-based hash functions can be used |
1793 |
to verify fixed length segments of data. If hash value of data segment is incorrect, |
to verify fixed length segments of data. If hash value of data segment is incorrect, |
1794 |
we need only to fetch \emph{segment} of data (instead of whole data) from |
we need only to fetch \emph{segment} of data (instead of whole data) from |
1795 |
an other source.}, such as \cite{merkle87hashtree, mohr02thex} for reliable and efficient |
an other source.}, such as \cite{merkle87hashtree, mohr02thex} for reliable and efficient |
1802 |
Additionally, there is only little real world experiments yet with tightly structured systems |
Additionally, there is only little real world experiments yet with tightly structured systems |
1803 |
(e.g., \cite{overneturl, edonkey2kurl}). Therefore, we can't say for sure, how well these |
(e.g., \cite{overneturl, edonkey2kurl}). Therefore, we can't say for sure, how well these |
1804 |
systems would perform in real Peer-to-Peer environment. However, we believe that these issues are |
systems would perform in real Peer-to-Peer environment. However, we believe that these issues are |
1805 |
solved, since there is a strong and wide research community towards to tightly structured |
solved, since there is a strong and wide research community towards to the tightly structured |
1806 |
overlays \cite{projectirisurl}. |
overlays \cite{projectirisurl}. |
1807 |
|
|
1808 |
|
|
1809 |
\section{Fenfire system model in Peer-to-Peer environment} |
\section{Fenfire system model in Peer-to-Peer environment} |
1810 |
|
|
1811 |
In this section we give a proposal for Fenfire Peer-to-Peer system, which consists |
In this section we give a proposal for Fenfire Peer-to-Peer system, which consists |
1812 |
of several technologies reviewed in this thesis. Then, we introduce simple but |
of several technologies reviewed in this thesis. Then, we introduce methods for |
1813 |
yet effective methods for obtaining Fenfire data from a Peer-to-Peer environment. |
obtaining Fenfire data from a Peer-to-Peer network. |
1814 |
|
|
1815 |
\subsection{System proposal} |
\subsection{System proposal} |
1816 |
|
|
1827 |
locate nearby data without looking up data from distant peers. Moreover, authors' |
locate nearby data without looking up data from distant peers. Moreover, authors' |
1828 |
proposal for self-organizing clusters using network diameters may be useful, |
proposal for self-organizing clusters using network diameters may be useful, |
1829 |
especially within small groups of working people. Thus, with Sloppy hashing |
especially within small groups of working people. Thus, with Sloppy hashing |
1830 |
we can provide locality properties for Fenfire. |
we can provide locality properties for the Fenfire system. |
1831 |
|
|
1832 |
For better fault tolerance and self-monitoring for Fenfire, we propose techniques |
For better fault tolerance and self-monitoring for Fenfire, we propose techniques |
1833 |
presented by Rowston et al. \cite{rowston03controlloingreliability}. With these |
presented by Rowston et al. \cite{rowston03controlloingreliability}. With these |
1834 |
techniques, we can ensure the performance of Fenfire in a highly adverse conditions, such |
techniques, we can ensure the performance of the Fenfire system in a highly adverse conditions, such |
1835 |
as sudden network partition, or highly dynamic and heterogeneous environment. |
as sudden network partition, or highly dynamic and heterogeneous environment. |
1836 |
|
|
1837 |
Finally, for more efficient data transfer, we can use variable techniques for this purpose. |
Finally, for more efficient data transfer, we can use variable techniques for this purpose. |
1838 |
For small amounts of data, HTTP can be used \cite{rfc2068}. For big amounts of data, we can use |
For small amounts of data, HTTP can be used \cite{rfc2068}. For big amounts of data, we can use |
1839 |
multisource downloads for better efficiency and reliability. Specifically, technology based |
multisource downloads for better efficiency and reliability. Specifically, the technology based |
1840 |
on rateless erasure codes \cite{maymounkov03ratelesscodes} seems very promising. |
on rateless erasure codes \cite{maymounkov03ratelesscodes} seems very promising. |
1841 |
|
|
1842 |
\subsection{Methods} |
\subsection{Methods} |
1844 |
We use the DOLR abstraction of the tightly structured approach, i.e., each participating peer hosts |
We use the DOLR abstraction of the tightly structured approach, i.e., each participating peer hosts |
1845 |
the data and the overlay maintains only the \emph{pointers} to the data. We decided to use the DOLR |
the data and the overlay maintains only the \emph{pointers} to the data. We decided to use the DOLR |
1846 |
abstraction in our model, since DOLR systems locate data without specifying a storage policy explicitly \cite{rhea03benchmarks}. |
abstraction in our model, since DOLR systems locate data without specifying a storage policy explicitly \cite{rhea03benchmarks}. |
1847 |
DHT based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have |
DHT-based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have |
1848 |
critical problems with load balancing in a highly heterogeneous environment. This problem is caused by peers |
severe problems with load balancing in a highly heterogeneous environment. The problem is caused by peers |
1849 |
which may not be able to store relatively large amount of data with key-value pair, assigned randomly by |
which may not be able to store relatively large amount of data with a key-value pair, assigned randomly by |
1850 |
the mapping function of the overlay. These systems waste both storage and bandwidth, and |
the mapping function of the overlay. These systems waste both storage and bandwidth, and |
1851 |
are sensitive to certain attacks (e.g., DDoS attack). Additionally, we emphasize that we prefer \emph{abstraction} |
are sensitive to certain attacks (e.g., the DDoS attack). Additionally, we emphasize that we prefer \emph{abstraction} |
1852 |
level analysis as very recently better and better tightly structured algorithms have been proposed. |
level analysis as very recently better and better tightly structured algorithms have been proposed. |
1853 |
Thus, we don't want to bind our system proposal to a specific algorithm definitively as we expect |
Thus, we don't want to bind our system proposal to a specific algorithm definitively as we expect |
1854 |
that this development continues. |
that this development continues. In this model, we use Kademlia's \cite{maymounkov02kademlia} algorithm for |
1855 |
|
locating data in the overlay. |
1856 |
|
|
1857 |
In the following subsections we assume that we know the structure of |
In the following subsections we assume that we know the structure of |
1858 |
the ''virtual file'' before hand, i.e., when assembling the ''virtual file'', we know all Storm |
the enfilade before hand, i.e., when assembling the ''virtual file'' we know all the Storm |
1859 |
blocks, which are required when building the ''virtual file''. Also, we don't |
blocks, which are required to complete the enfilade. Also, we don't |
1860 |
respond to security issues related to Peer-to-Peer systems, since there is no working solution |
respond to the security issues related to Peer-to-Peer systems, since there is no working solution |
1861 |
available yet; we either assume that Fenfire has a reliable technique for identifying individual entities, or |
available yet; we either assume that Fenfire has a reliable technique for identifying individual entities, or |
1862 |
there are no hostile entities among participating peers. |
there are no hostile entities among participating peers. |
1863 |
|
|
1864 |
In our model, each peer maintains following data structures for local operations: a data structure for listing all |
In our method, each peer maintains the following data structures for local operations: a data structure for listing all |
1865 |
key-value pairs which peer maintains; a data structure for listing all key-value pairs in the |
key-value pairs which peer maintains; a data structure for listing all key-value pairs in the |
1866 |
chronological order (the most recent block is topmost) which peer maintains. We use Storm blocks' identifiers |
chronological order (the most recent block is topmost) which peer maintains. We use Storm blocks' identifiers |
1867 |
as \emph{keys} of the overlay. Every key-value pair consists of either a hash of pointer random string |
as \emph{keys} of the overlay. Every key-value pair consists of either a hash of pointer random string |
1868 |
(pointer blocks), or a hash of block's content (scroll blocks) as a key. Value is always a reference to a hosting |
(pointer blocks), or a hash of block's content (scroll blocks) as a key. The value is always a reference to a hosting |
1869 |
peer (e.g., IP address). We use Kademlia's \cite{maymounkov02kademlia} algorithm for locating data in the overlay. |
peer (e.g., IP address). Finally, we assume that all local operations can be done in a constant time. |
|
Finally, we assume that all local operations can be done in a constant time. |
|
1870 |
|
|
1871 |
|
|
1872 |
\begin{itemize} |
\begin{itemize} |
1873 |
\item Data lookup with a given identifier of Storm scroll block. |
\item Data lookup with a given identifier of Storm scroll block. |
1874 |
\begin{enumerate} |
\begin{enumerate} |
1875 |
\item Submit data lookup using scroll block's identifier. |
\item Submit the data lookup using scroll block's identifier. |
1876 |
\item Repeat until the pointer peer is found: each peer forwards the data lookup to a closer peer which hosts the given scroll block identifier. |
\item Repeat until the pointer peer is found: each peer forwards the data lookup to a closer peer which hosts the given scroll block identifier. |
1877 |
\item The pointer peer returns value (e.g., the IP address of provider peer) to the query originator. |
\item The pointer peer returns value (e.g., the IP address of provider peer) to the query originator. |
1878 |
\item The query originator requests the provider peer to return the scroll block. |
\item The query originator requests the provider peer to return the scroll block. |
1879 |
\end{enumerate} |
\end{enumerate} |
1880 |
\end{itemize} |
\end{itemize} |
1881 |
|
|
1882 |
Figure \ref{fig:storm_query_blockid} illustrates how a Storm scroll block is located |
Figure \ref{fig:storm_query_blockid} illustrates how Storm scroll block is located |
1883 |
in tightly structured overlay using the DOLR abstraction, where the identifier of Storm scroll |
in a tightly structured overlay using the DOLR abstraction, where the identifier of Storm scroll |
1884 |
block is known. |
block is given. |
1885 |
|
|
1886 |
|
|
1887 |
\begin{itemize} |
\begin{itemize} |
1888 |
\item Data lookup with a given pointer random string returning most recent scroll block. |
\item Data lookup with a given pointer random string returning most recent scroll block. |
1889 |
\begin{enumerate} |
\begin{enumerate} |
1890 |
\item The query originator locally computes a hash for given pointer random string. |
\item The query originator locally computes a hash over given pointer random string. |
1891 |
\item Repeat until the pointer peer is found: each peer forwards the data lookup to a closer peer which hosts the given hash of pointer random string. |
\item Repeat until the pointer peer is found: each peer forwards the data lookup to a closer peer which hosts the given hash of pointer random string. |
1892 |
\item The pointer peer returns most recent pointer block's key-value pair (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes. |
\item The pointer peer returns most recent pointer block's key-value pair (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes. |
1893 |
\item The query originator requests the provider peer to return the scroll block. |
\item The query originator requests the provider peer to return the scroll block. |
1897 |
\begin{itemize} |
\begin{itemize} |
1898 |
\item Data lookup with a given pointer random string returning scroll block(s) for a given date or time range. |
\item Data lookup with a given pointer random string returning scroll block(s) for a given date or time range. |
1899 |
\begin{enumerate} |
\begin{enumerate} |
1900 |
\item The query originator locally computes a hash for given pointer random string. |
\item The query originator locally computes a hash over given pointer random string. |
1901 |
\item Repeat until the pointer peer is found: each peer forwards the data lookup to a closer peer which hosts the given hash of pointer random string. |
\item Repeat until the pointer peer is found: each peer forwards the data lookup to a closer peer which hosts the given hash of pointer random string. |
1902 |
\item Pointer peer returns pointer block's key-value pair(s) (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes. |
\item Pointer peer returns pointer block's key-value pair(s) (e.g., the IP address of provider peer) to the query originator, using pointer block's own indexing schemes. |
1903 |
\item The query originator requests the provider peer to return the scroll block. |
\item The query originator requests the provider peer to return the scroll block. |
1905 |
\end{itemize} |
\end{itemize} |
1906 |
|
|
1907 |
Figure \ref{fig:storm_query_urn5} illustrates how Storm scroll block is located |
Figure \ref{fig:storm_query_urn5} illustrates how Storm scroll block is located |
1908 |
in tightly structured overlay using the DOLR abstraction, where the pointer random string is known. |
in a tightly structured overlay using the DOLR abstraction, where the pointer random string is given. |
1909 |
|
|
1910 |
Each of these algorithms can locate Fenfire data in $O(\log{n})$ time at application level overlay: |
Each of these algorithms can locate Fenfire blocks in $O(\log{n})$ time at application level overlay: |
1911 |
$O(\log{n})$ time for query routing to pointer peer and constant time for |
$O(\log{n})$ time for query routing to pointer peer and constant time for |
1912 |
locating hosting peer with a given reference link. |
locating hosting peer with a given reference link. |
1913 |
|
|
1929 |
|
|
1930 |
\subsection{Problems} |
\subsection{Problems} |
1931 |
|
|
1932 |
Perhaps the most biggest issue in Peer-to-Peer systems is non-maturity of |
Perhaps the most biggest issue in Peer-to-Peer systems is the non-maturity of |
1933 |
security technologies. For instance, online entities cannot be identified |
security technologies. For instance, online entities cannot be identified |
1934 |
safely (e.g., the Sybil attack \cite{douceur02sybil}). For Fenfire, one |
safely (e.g., the Sybil attack \cite{douceur02sybil}). For the Fenfire system, one |
1935 |
security related problem occurs when user wants to perform a global data lookup with a given |
security related problem occurs when a user wants to perform a global data lookup with a given |
1936 |
pointer random string; how can a user verify the correctness |
pointer random string; how the user is able to verify the correctness |
1937 |
of the search results ? Specifically, how she or he knows which one is the |
of the search results, i.e., how she or he knows which one is the |
1938 |
correct Storm scroll block ? The Spam attack \cite{naor03simpledht} is a variation of previously |
correct Storm scroll block ? The Spam attack \cite{naor03simpledht} is a variation of previously |
1939 |
mentioned problem; data lookup is performed by a user, but there is no reply |
mentioned problem; data lookup is performed by a user, but there is no reply |
1940 |
from the system. How are we able to know if this was a spam attack, or the |
from the system. How are we able to know if this was a spam attack, or the |
1944 |
authenticity of data. Obviously, optimal solution to all security issues would |
authenticity of data. Obviously, optimal solution to all security issues would |
1945 |
be that digital signatures are included to every message sent to the system therefore |
be that digital signatures are included to every message sent to the system therefore |
1946 |
enabling peers to authenticate other peers safely. However, these problems are not |
enabling peers to authenticate other peers safely. However, these problems are not |
1947 |
only limited to the Fenfire as it concerns all Peer-to-Peer based computer systems. |
only limited to the Fenfire system as it concerns all Peer-to-Peer computer systems. |
1948 |
|
|
1949 |
As security technologies come more mature, we wish to apply these |
As security technologies come more mature, we wish to apply these |
1950 |
technologies with Fenfire, if applicable. |
technologies with Fenfire, if applicable. |
1952 |
\chapter{Conclusions and future work} |
\chapter{Conclusions and future work} |
1953 |
|
|
1954 |
In this thesis, we have reviewed existing Peer-to-Peer approaches, algorithms and |
In this thesis, we have reviewed existing Peer-to-Peer approaches, algorithms and |
1955 |
their properties. We have summarized open problems in Peer-to-Peer research domain. |
their properties. Our insight is that despite the great amount of Peer-to-Peer systems, |
1956 |
Specifically, we divided open problems into the three sub-categories: security related problems, |
we are able to classify \emph{all} systems either to loosely or tightly structured systems. |
1957 |
performance related problems and miscellaneous problems. Each of these |
We have summarized open problems in Peer-to-Peer research domain. Specifically, we divided open |
1958 |
|
problems into the three sub-categories: security related problems, |
1959 |
|
performance related problems and miscellaneous problems. We point out that each of these |
1960 |
sub-categories have number of open problems, in which there are no solutions |
sub-categories have number of open problems, in which there are no solutions |
1961 |
yet, or solutions are only partial. We point out that much research work is required to |
yet, or solutions are only partial. |
|
solve these problems. |
|
1962 |
|
|
1963 |
Then, we focused our attention to the Fenfire system. First, we gave a brief |
Then, we focused our attention to the Fenfire system. First, we gave a brief |
1964 |
overview of Fenfire and xanalogical storage model. We also described Storm software module. |
overview of the Fenfire system and xanalogical storage model. We also described Storm software module. |
1965 |
|
|
1966 |
In the last chapter, we evaluated existing Peer-to-Peer approaches with regard |
In the last chapter, we evaluated existing Peer-to-Peer approaches with regard |
1967 |
to Fenfire's needs. We proposed that the tightly structured approach is the |
to Fenfire's needs. We see that the tightly structured approach is the |
1968 |
best alternative to Fenfire's needs for the following reasons. First, Storm, xanalogical |
best alternative to Fenfire's needs for the following reasons. First, Storm, xanalogical |
1969 |
storage model and tightly structured systems use global unique identifiers |
storage model and tightly structured systems use global unique identifiers |
1970 |
for identifying data. Second, our Storm design uses \emph{semantic-free references} |
for identifying data. Second, our Storm design uses \emph{semantic-free references} |
1971 |
(block identifiers and pointer random strings) for locating data in distributed |
(block identifiers and pointer random strings) for locating data in distributed |
1972 |
networks. As the authors of \cite{balakrishnan03semanticfree}, |
networks. As the authors of \cite{balakrishnan03semanticfree}, |
1973 |
we also agree that references should be semantically-free in the next-generation |
we also agree that references should be semantically-free in next-generation |
1974 |
reference resolution services. Third, by using |
reference resolution services. Third, by using |
1975 |
the DOLR abstraction of tightly structured overlay, we can minimize the lack |
the DOLR abstraction of tightly structured overlay, we can minimize the lack |
1976 |
of locality in the tightly structured approach. Finally, we believe that issues |
of locality in the tightly structured approach. Finally, we believe that issues |
1977 |
related to tightly structured overlays are solved in the near future, because of |
related to tightly structured overlays are solved in the near future, because of |
1978 |
wide and intensive co-operation among research groups. |
wide and intensive co-operation among research groups. |
1979 |
|
|
1980 |
Then, we proposed system model for Fenfire and presented simple methods to perform data |
Our future work includes a support for searching transclusions and xanalogical |
1981 |
lookups in a Peer-to-Peer environment. |
links in Peer-to-Peer environment. Preliminary analysis have shown |
|
|
|
|
Our future work includes support for searching transclusions and xanalogical |
|
|
links in a Peer-to-Peer network. Specifically, we want to find transclusions |
|
|
and xanalogical links in a global scale. Preliminary analysis have shown |
|
1982 |
that these questions are rather different than locating scroll or pointer |
that these questions are rather different than locating scroll or pointer |
1983 |
blocks \emph{directly} from the network. Techniques used in distributed |
blocks from Peer-to-Peer environment. Techniques used in distributed |
1984 |
database systems may prove to be useful. Some fundamental results |
database systems may prove to be useful. Some fundamental results |
1985 |
regarding Peer-to-Peer and database systems have already been |
regarding Peer-to-Peer and database systems have already been |
1986 |
presented in \cite{gribble01p2pdatabase}. |
presented in \cite{gribble01p2pdatabase}. |