491 |
overlays, i.e., $O(\log{n})$ space required for maintaining information about other peers in |
overlays, i.e., $O(\log{n})$ space required for maintaining information about other peers in |
492 |
the system and $O(\log{n})$ data lookup efficiency. |
the system and $O(\log{n})$ data lookup efficiency. |
493 |
|
|
494 |
|
Additionally, authors argue in \cite{balakrishnan03semanticfree} that tightly structured systems |
495 |
|
are suitable for next generation Reference Resolutions Services (RRS)\footnote{ |
496 |
|
Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system in the Internet.}. They present |
497 |
|
two requirements about the nature of reference resolution. First, there should be a general-purpose |
498 |
|
and application-indepedent substrate for reference resolution. Second, the references themselves |
499 |
|
should be unstructured and semantic-free. Authors emphasize that tightly structured systems (specifically |
500 |
|
DHT-based) provide an elegant platform for RRS. In this context, we define unstructured reference |
501 |
|
as a reference that it doesn't expose the target in any way and semantic-free reference as a reference |
502 |
|
that there are no directives in the reference itself which would expose how the reference should be processed. |
503 |
|
|
504 |
|
|
505 |
\section{Differences} |
\section{Differences} |
506 |
|
|
1296 |
Also, query and routing hot spots may be an issue in tightly structured overlays \cite{ratnasamy02routing}. |
Also, query and routing hot spots may be an issue in tightly structured overlays \cite{ratnasamy02routing}. |
1297 |
Hot spots happen, when a specific key is being requested extremely often in tightly structured overlays. Recent study |
Hot spots happen, when a specific key is being requested extremely often in tightly structured overlays. Recent study |
1298 |
by Freedman et al. tries to reduce hot spots in the system by performing \emph{sloppy} hashing |
by Freedman et al. tries to reduce hot spots in the system by performing \emph{sloppy} hashing |
1299 |
\cite{sloppy:iptps03}. Another key feature of their work is that peers self-organize into clusters, |
\cite{sloppy:iptps03}. Authors' technique is especially suitable for the DOLR abstraction of tightly structured overlays. |
1300 |
therefore enabling peers to find nearby data without looking up data from distant peers. |
With Sloppy hashing, we are able to reduce the generation of query hot spots. Sloppy hashing enables to |
1301 |
|
locate nearby data without looking up data from distant peers. Moreover, authors' |
1302 |
|
proposal for self-organizing clusters using network diameters may be useful, |
1303 |
|
especially within small groups of working people. Thus, with Sloppy hashing |
1304 |
|
we can provide locality properties the system. |
1305 |
|
|
1306 |
|
|
1307 |
|
|
1308 |
As mentioned before, an implicit assumption of almost every tightly structured system is that there is a random, uniform |
As mentioned before, an implicit assumption of almost every tightly structured system is that there is a random, uniform |
1309 |
distribution of peer and key identifiers. Even if participating peers are extremely heterogeneous, e.g., in |
distribution of peer and key identifiers. Even if participating peers are extremely heterogeneous, e.g., in |
1777 |
one ''virtual file'' may need obtaining several Storm blocks, which are distributed |
one ''virtual file'' may need obtaining several Storm blocks, which are distributed |
1778 |
randomly throughout the overlay. If not efficient, construction of the ''virtual file'' |
randomly throughout the overlay. If not efficient, construction of the ''virtual file'' |
1779 |
may take reasonable amount of time while rendering system very unusable. Third, Peer-to-Peer |
may take reasonable amount of time while rendering system very unusable. Third, Peer-to-Peer |
1780 |
infrastructure has to be scalable and fault tolerant against hostile attacks. |
infrastructure has to be scalable, fault tolerant against hostile attacks and resilience in |
1781 |
|
adverse conditions (e.g., a network partition). |
1782 |
|
|
1783 |
\section{Evaluation of Peer-to-Peer approaches with regard to Fenfire} |
\section{Evaluation of Peer-to-Peer approaches with regard to Fenfire} |
1784 |
|
|
1785 |
|
In this section we focus on locating the Storm blocks in Peer-to-Peer environment. We don't |
1786 |
|
respond to fetching of Storm blocks as fetching of Storm block can be performed easily once |
1787 |
|
Storm block is located. |
1788 |
|
|
1789 |
In chapter 2, we discussed main differences between the loosely and the tightly structured |
In chapter 2, we discussed main differences between the loosely and the tightly structured |
1790 |
approach. As stated, the most significant difference is that the tightly structured |
approach. As stated, the most significant difference is that the tightly structured |
1791 |
approach has logarithmical properties in all internal operations, while the loosely |
approach has logarithmical properties in all internal operations, while the loosely |
1801 |
structured systems and Fenfire use similar methods for identifying data in the |
structured systems and Fenfire use similar methods for identifying data in the |
1802 |
system, i.e., globally unique identifiers. |
system, i.e., globally unique identifiers. |
1803 |
Another key feature of tightly structured overlays is that they are able |
Another key feature of tightly structured overlays is that they are able |
1804 |
to provide general purpose \emph{interface} for Reference Resolution Services (RRS)\footnote{ |
to provide general purpose \emph{interface} for Reference Resolution Services (RRS) |
|
Domain Name System (DNS) \cite{rfc1101} is a widely used RRS system in the Internet.} |
|
1805 |
\cite{balakrishnan03semanticfree}. Authors argue that next generation RRS must be |
\cite{balakrishnan03semanticfree}. Authors argue that next generation RRS must be |
1806 |
application-independent and references itself should be \emph{unstructured} and |
application-independent and references itself should be \emph{unstructured} and |
1807 |
\emph{semantically free}. Finally, as said, with tightly structured systems it is feasible to |
\emph{semantically free}. Thus, we see the tightly structured approach as the best alternative to |
1808 |
perform \emph{global} data lookups in the overlay. To summarize, these aspects may be the most important features |
locate data in Peer-to-Peer environment. |
|
of Peer-to-Peer infrastructure with regard to Fenfire as a distributed, location transparent hypermedia system. |
|
|
Thus, we see the tightly structured approach as the best alternative to locate data in Peer-to-Peer |
|
|
environment. |
|
|
|
|
|
Once located, we can use regular TCP/IP-protocols, such as Hypertext Transfer protocol (HTTP) |
|
|
\cite{rfc2068} for \emph{fetching} Storm blocks from the overlay. However, HTTP-protocol may |
|
|
not be a optimal solution when obtaining large amounts of data from a Peer-to-Peer network (e.g., |
|
|
videos, images or music). In this case, multisource downloads can be very useful |
|
|
for better efficiency \cite{maymounkov03ratelesscodes, bittorrenturl}. Furthermore, |
|
|
multisource downloads can be used for decreasing load of a certain peer, thus avoiding query |
|
|
hot spots in the system \cite{ratnasamy02routing}. Current implementation of Fenfire uses |
|
|
standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content |
|
|
hash for verifying the integrity of data by recomputing the content hash |
|
|
for a scroll block. In face of multisource downloads, Fenfire must support |
|
|
tree-based hashes\footnote{With multisource downloads, tree-based hash functions can be used |
|
|
to verify fixed length segments of data. If hash value of data segment is incorrect, |
|
|
we need only to fetch \emph{segment} of data (instead of whole data) from |
|
|
an other source.}, such as \cite{merkle87hashtree, mohr02thex} for reliable and efficient |
|
|
data validation. |
|
1809 |
|
|
1810 |
Again, there are research challenges with tightly structured systems which have to be |
Again, there are research challenges with tightly structured systems which have to be |
1811 |
addressed, as described in chapter 3. The main concerns include decreased performance and fault |
addressed, as described in chapter 3. The main concerns include decreased performance and fault |
1814 |
Additionally, there is only little real world experiments yet with tightly structured systems |
Additionally, there is only little real world experiments yet with tightly structured systems |
1815 |
(e.g., \cite{overneturl, edonkey2kurl}). Therefore, we can't say for sure, how well these |
(e.g., \cite{overneturl, edonkey2kurl}). Therefore, we can't say for sure, how well these |
1816 |
systems would perform in real Peer-to-Peer environment. However, we believe that these issues are |
systems would perform in real Peer-to-Peer environment. However, we believe that these issues are |
1817 |
solved, since there is a strong and wide research community towards to the tightly structured |
solved in near future, since there is a strong and wide research community towards to the tightly structured |
1818 |
overlays \cite{projectirisurl}. |
overlays \cite{projectirisurl}. |
1819 |
|
|
1820 |
|
|
1826 |
|
|
1827 |
\subsection{System proposal} |
\subsection{System proposal} |
1828 |
|
|
1829 |
Currently, we see Kademlia \cite{maymounkov02kademlia} as the best algorithm for |
We emphasize that we prefer \emph{abstraction} |
1830 |
|
level analysis as very recently better and better tightly structured algorithms have been proposed. |
1831 |
|
Thus, we don't want to bind our system proposal to a specific algorithm definitively as we expect |
1832 |
|
that this development continues. Currently, we see Kademlia \cite{maymounkov02kademlia} as the best algorithm for |
1833 |
locating data efficiently in the Peer-to-Peer overlay. There are two |
locating data efficiently in the Peer-to-Peer overlay. There are two |
1834 |
reasons for this. First, Kademlia's XOR-based distance function is superior |
reasons for this. First, Kademlia's XOR-based distance function is superior |
1835 |
over distance functions of other systems (see section 2.4). Second, there exist already |
over distance functions of other systems (see section 2.3.2). Secondly, Kademlia |
1836 |
deployed real-life systems using Kademlia (e.g., \cite{overneturl, edonkey2kurl, kashmirurl, |
is one of the only tightly structured systems that has been deployed in real life |
1837 |
kato02gisp}), which means that Kademlia's algorithm is simple and easy to implement. |
(e.g., \cite{overneturl, edonkey2kurl, kashmirurl,kato02gisp}), which means that |
1838 |
|
Kademlia's algorithm is simple and easy to implement. |
1839 |
|
|
1840 |
On top of Kademlia, we propose the usage of Sloppy hashing \cite{sloppy:iptps03} which |
On top of Kademlia, we propose the usage of Sloppy hashing \cite{sloppy:iptps03} which |
1841 |
is optimized for the DOLR abstraction of tightly structured overlays. With the Sloppy hashing, |
is optimized for the DOLR abstraction of tightly structured overlays. With Sloppy hashing |
|
we are able to reduce the generation of query hot spots. Sloppy hashing enables to |
|
|
locate nearby data without looking up data from distant peers. Moreover, authors' |
|
|
proposal for self-organizing clusters using network diameters may be useful, |
|
|
especially within small groups of working people. Thus, with Sloppy hashing |
|
1842 |
we can provide locality properties for the Fenfire system. |
we can provide locality properties for the Fenfire system. |
1843 |
|
|
1844 |
For better fault tolerance and self-monitoring for Fenfire, we propose techniques |
For better fault tolerance and self-monitoring for Fenfire, we propose techniques |
1847 |
as sudden network partition, or highly dynamic and heterogeneous environment. |
as sudden network partition, or highly dynamic and heterogeneous environment. |
1848 |
|
|
1849 |
Finally, for more efficient data transfer, we can use variable techniques for this purpose. |
Finally, for more efficient data transfer, we can use variable techniques for this purpose. |
1850 |
For small amounts of data, HTTP can be used \cite{rfc2068}. For big amounts of data, we can use |
For small amounts of data, HTTP can be used \cite{rfc2068}. For large amounts of data, we can use |
1851 |
multisource downloads for better efficiency and reliability. Specifically, the technology based |
multisource downloads for better efficiency and reliability. Specifically, the technology based |
1852 |
on rateless erasure codes \cite{maymounkov03ratelesscodes} seems very promising. |
on rateless erasure codes \cite{maymounkov03ratelesscodes} seems very promising. |
1853 |
|
Furthermore, multisource downloads can be used for decreasing load of a certain peer, thus avoiding query |
1854 |
|
hot spots in the system \cite{ratnasamy02routing}. Current client-server implementation of Fenfire uses |
1855 |
|
standard single source downloads (HTTP) and SHA-1 \cite{fips-sha-1} cryptographic content |
1856 |
|
hash for verifying the integrity of data by recomputing the content hash |
1857 |
|
for block. In face of multisource downloads, Fenfire must support |
1858 |
|
tree-based hashes\footnote{With multisource downloads, tree-based hash functions can be used |
1859 |
|
to verify fixed length segments of data. If hash value of data segment is incorrect, |
1860 |
|
we need only to fetch \emph{segment} of data (instead of whole data) from |
1861 |
|
an other source.}, such as \cite{merkle87hashtree, mohr02thex} for reliable and efficient |
1862 |
|
data validation. |
1863 |
|
|
1864 |
\subsection{Methods} |
\subsection{Methods} |
1865 |
|
|
1866 |
We use the DOLR abstraction of the tightly structured approach, i.e., each participating peer hosts |
We use the DOLR abstraction of the tightly structured approach since DOLR systems locate data without |
1867 |
the data and the overlay maintains only the \emph{pointers} to the data. We decided to use the DOLR |
specifying a storage policy explicitly \cite{rhea03benchmarks}, i.e., each participating peer hosts |
1868 |
abstraction in our model, since DOLR systems locate data without specifying a storage policy explicitly \cite{rhea03benchmarks}. |
the data they are offering and the overlay maintains only the \emph{pointers} to the data. |
1869 |
DHT-based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have |
DHT-based storage systems, such as CFS \cite{dabek01widearea} and PAST \cite{rowstron01storage}, may have |
1870 |
severe problems with load balancing in a highly heterogeneous environment. The problem is caused by peers |
severe problems with load balancing in a highly heterogeneous environment \cite{rao03loadbalancing}. The problem is caused by peers |
1871 |
which may not be able to store relatively large amount of data with a key-value pair, assigned randomly by |
which may not be able to store relatively large blocks, assigned randomly by the mapping function of the overlay. |
|
the mapping function of the overlay. These systems waste both storage and bandwidth, and |
|
|
are sensitive to certain attacks (e.g., the DDoS attack). Additionally, we emphasize that we prefer \emph{abstraction} |
|
|
level analysis as very recently better and better tightly structured algorithms have been proposed. |
|
|
Thus, we don't want to bind our system proposal to a specific algorithm definitively as we expect |
|
|
that this development continues. In this model, we use Kademlia's \cite{maymounkov02kademlia} algorithm for |
|
|
locating data in the overlay. |
|
|
|
|
|
In the following subsections we assume that we know the structure of |
|
|
the enfilade before hand, i.e., when assembling the ''virtual file'' we know all the Storm |
|
|
blocks, which are required to complete the enfilade. Also, we don't |
|
|
respond to the security issues related to Peer-to-Peer systems, since there is no working solution |
|
|
available yet; we either assume that Fenfire has a reliable technique for identifying individual entities, or |
|
|
there are no hostile entities among participating peers. |
|
1872 |
|
|
1873 |
In our method, each peer maintains the following data structures for local operations: a data structure for listing all |
In our method, each peer maintains the following data structures for local operations: a data structure for listing all |
1874 |
key-value pairs which peer maintains; a data structure for listing all key-value pairs in the |
key-value pairs which peer maintains; a data structure for listing all key-value pairs in the |
1877 |
(pointer blocks), or a hash of block's content (scroll blocks) as a key. The value is always a reference to a hosting |
(pointer blocks), or a hash of block's content (scroll blocks) as a key. The value is always a reference to a hosting |
1878 |
peer (e.g., IP address). Finally, we assume that all local operations can be done in a constant time. |
peer (e.g., IP address). Finally, we assume that all local operations can be done in a constant time. |
1879 |
|
|
1880 |
|
We assume that we have resolved the construction of the ''virtual file'' before locating any Storm blocks, i.e., |
1881 |
|
when assembling the ''virtual file'' we know all the Storm blocks, which are required to complete the ''virtual file''. |
1882 |
|
Also, we don't respond to the security issues related to Peer-to-Peer systems, since there is no working solution |
1883 |
|
available yet. Thus, we either assume that Fenfire has a reliable technique for identifying individual entities, or |
1884 |
|
there are no hostile entities among participating peers, i.e., Storm blocks can be identified correctly (e.g., when |
1885 |
|
performing searches). In the next section, we discuss security problems in more detail. |
1886 |
|
|
1887 |
|
|
1888 |
\begin{itemize} |
\begin{itemize} |
1889 |
\item Data lookup with a given identifier of Storm scroll block. |
\item Data lookup with a given identifier of Storm scroll block. |
1924 |
in a tightly structured overlay using the DOLR abstraction, where the pointer random string is given. |
in a tightly structured overlay using the DOLR abstraction, where the pointer random string is given. |
1925 |
|
|
1926 |
Each of these algorithms can locate Fenfire blocks in $O(\log{n})$ time at application level overlay: |
Each of these algorithms can locate Fenfire blocks in $O(\log{n})$ time at application level overlay: |
1927 |
$O(\log{n})$ time for query routing to pointer peer and constant time for |
$O(\log{n})$ time for query routing to pointer peer (Kademlia \cite{maymounkov02kademlia}) and constant time for |
1928 |
locating hosting peer with a given reference link. |
locating hosting peer with a given reference link. |
1929 |
|
|
1930 |
|
|