Monday, June 26, 2006

The Three Laws of File Sharing

My ideas for a fast private p2p network are based on the three laws of file sharing.

1st Law of P2P File Sharing:

Every bit of data downloaded must have been uploaded.

So overall no network is faster than any other as uploads must equal downloads on any network.

Multi-source or swarming by itself does not increase download speed.

Networks that have high download speed have a greater number of users that upload more than they download.

For example, the majority of Bit Torrent users feel that they should at the very least maintain an upload/download ratio of one for every file they download even if they can download faster than they can upload.

This leads to faster downloads for everyone.

2nd Law of P2P File Sharing:

The size of a network's user base is indirectly and inversely related to download speed. Networks with a large user base have slower downloads than those with a small one.

The majority of files on a P2P network are provided a small minority of enthusiastic users who often join networks early on.

As the network becomes more popular it attracts a greater proportion of users who want to download files but who don't have the technical skill or motivation to provide files to the network.

Law one states that uploads have to equal downloads, therefore downloads slow as there are more users demanding files without a corresponding increase in the number of users providing files.

For example, the ed2k network, a very large network, is often slower than say a DC hub which has a smaller user base of enthusiasts.

BT is fast as it is many thousands of small networks controlled by a tracker, with all downloaders uploading, rather than one enormous network.

3rd Law of P2P File Sharing:

The more upload bandwidth donated by users of a network then the faster that network.

It is often social pressure or a technical feature of the p2p file sharing application that ensures that users donates a high proportion of their available upload bandwidth to the network and that they don't leach.

For example, leachers on Winmx often receive messages telling them to mend their ways.

Many DC hubs have rules to discourage leachers from joining.

To download on Bit Torrent you have to upload.

Proxy Networks (such as kerjodando p2p):

It should be noted that due to laws 1, 2 and 3 proxy networks have slower download speeds than direct connect** networks.

This is due to proxy data having to be downloaded and uploaded more than once (law 1).

kerjodando reduces this by have many small networks controller by a tracker rather than one big network.

As file providers also proxy data their upload bandwidth is used up even faster than on direct connect networks (law 2).

kerjodando reduces proxy traffic by clustering users according to their interests.

Also, proxy networks need users to donate much more upload bandwidth than direct connect networks making them even slower (law 3).

kerjodando when operational will provide dedicated servers with extra upload bandwidth.

Fast proxy networks need an efficient node layout (topology) so fewer hops(less bandwidth) used for each download(law 1); to be small(law 2) and to attract users who want to share rather than download (law 3).

A prime example of this is "Share" an off-shoot of Winny (Japanese).

kerjodando does not cache files but does cluster users according to interest like winny so attracting users who want to share content that they want to promote like their favourite band, movie or hobby.

The price for users wanting to share anonymously is slower downloads.

However, the Japanese have shown that networks like kerjodando can speed-up downloads while still hiding their users' identity and keeping file transfers private.

**Direct connect networks are ordinary p2p networks such as edonkey and winmx as oppose to proxy networks that don't connect users directly"