Optimal piece size for torrents?

[CodeGeek]

Hello everyone,

can anyone tell me what the optimal piece size for torrents is (in relation to their overall size)?

If you create a new torrent in BitTorrent it will offer the following piece sizes:

auto-detect
16 kB
32 kB
64 kB
128 kB
256 kB
512 kB
1024 kB
2048 kB
4096 kB
8192 kB
16384 kB

I try to look up how the auto-detect algorithm works, but I didn't find any information about it.

In the Internet I found the following article: Torrent Piece Size - VuzeWiki

File size - piece size
350 MB - 256 or 512 kB
1400 MB - 1024 kB

And the following post in a forum: Piece size of torrents

* Files up to 50MiB: 32KiB piece size
* Files 50MiB to 150MiB: 64KiB piece size
* Files 150MiB to 350MiB: 128KiB piece size
* Files 350MiB to 512MiB: 256KiB piece size
* Files 512MiB to 1.0GiB: 512KiB piece size
* Files 1.0GiB to 2.0GiB: 1024KiB piece size
* Files 2.0GiB and up: 2048KiB piece size
* Piece sizes above 2048KiB are not recommended.

What do you think about these recommendations mentioned in the article as well as in the post?
What are your recommendations (if auto-detect is / would be not available)?

 

[IdolFun]

512 kB 1GB an under
1024 kB 1GB to 5GB
2048 kB 5GB and over

 

[CodeGeek]

512 kB 1GB an under
1024 kB 1GB to 5GB
2048 kB 5GB and over

Thank you very much for your recommendation, IdolFun. :wahey:

Any other opinions / recommendations?

 

[SamKook]

Personally, I'd go with if you don't know enough about it, leave the default on, they're the default for a reason.

The way I understand it, lower pieces means you will waste less bandwidth since if it can't complete a block or has to re-dl it due to corruption, it has less to dl to complete it, but bigger pieces means less time wasted managing them since the software spend less time/effort asking for a new piece because there are less of them.

The only reason I wouldn't use auto is if I could make sure the pieces would end exactly at the end of the files if there are more than one, but that's pretty much impossible unless you split or archive them to make them fit.

 

[CodeGeek]

As you said - if the piece size is smaller corrupt parts can be downloaded faster because they are smaller. Also the files will be available faster because clients have complete parts earlier.

The biggest disadvantage is the overhead: Smaller piece size means more pieces for one file. And more pieces means more hashes which results in a bigger torrent file size. Also the overhead when sharing the files is bigger as more often parts have to be requested instead of loading one big part after another.

Okay, but if you don't have any auto option - what size would you prefer?

 

[SamKook]

I'd go with a suggestion from a source I trust like the following article from torrentfreak:

After a number of years toying around, the optimum number of pieces seems to be between 1200 and 2200. Most torrent creators will only allow piece-sizes in multiples of 16kb, so you should, with few exceptions, find a size that fits in that range. A 700Mb torrent should be 512Kb pieces (giving 1400 total) and similarly, 350Mb would be better with 256kb. A 4.5Gb torrent would have 2,250 pieces, roughly, with a 2Mb piece-size. Or 1,125 with 4Mb. Either way would be fine, but 256kb pieces would mean 17,500+ pieces, and is too many.

 

[CodeGeek]

Thanks, SamKook, for positing.

 

[CodeGeek]

I found a wiki called Theory.org Wiki. They have wiki articles about BitTorrent.
In the Notes of one of this wiki articles they write about the piece size.

The piece length specifies the nominal piece size, and is usually a power of 2. The piece size is typically chosen based on the total amount of file data in the torrent, and is constrained by the fact that too-large piece sizes cause inefficiency, and too-small piece sizes cause large .torrent metadata file. Historically, piece size was chosen to result in a .torrent file no greater than approx. 50 - 75 kB (presumably to ease the load on the server hosting the torrent files).

• Current best-practice is to keep the piece size to 512KB or less, for torrents around 8-10GB, even if that results in a larger .torrent file. This results in a more efficient swarm for sharing files. The most common sizes are 256 kB, 512 kB, and 1 MB.

That will result in something like that (of course that number of pieces is not 100% accurate as I simply divided the whole torrent meta file size by 20 - 20 bytes for each SHA-1/SHA-160. But a torrent meta file contains also some other information as well as some overhead from the bencoded data):

This result in the following list:

• Files up to 50MiB: 16 or 32KiB piece size
• Files 50MiB to 150MiB: 64KiB piece size
• Files 150MiB to 350MiB: 128 or 256KiB piece size
• Files 350MiB to 512MiB: 256KiB piece size
• Files 512MiB to 1.0GiB: 512KiB piece size
• Files 1.0GiB to 2.0GiB: 1024KiB piece size
• Files 2.0GiB and up: 2048KiB piece size

So it's pretty much the same like that forum post Piece size of torrents mentioned in my first post.

 

[SamKook]

Why didn't you follow the current best practice from the article and not go over 512KB per pieces?

 

[CodeGeek]

Why didn't you follow the current best practice from the article and not go over 512KB per pieces?

My table was based on the paragraph above the "best-practice". But you're right: 512KiB should be the maximum.

• 1 GiB @ 512KiB piece size: ~41 KiB
• 2 GiB @ 512KiB piece size: ~82 KiB
• 4 GiB @ 512KiB piece size: ~164 KiB
• 8 GiB @ 512KiB piece size: ~328 KiB
• 64 GiB @ 512KiB piece size: ~2.7 MiB
• 128 GiB @ 512KiB piece size: ~5.3 MiB

For torrents up to 8 GiB it's no problem. But for bigger torrents... Especially as AO limits torrent attachments to 2.86 MiB.
On the other hand: There aren't many torrents which are sooo big.