Off Topic
Report problems with the forum itself, or any thing else not specific to Tixati
<<  Back To Forum

[feature request] multi source downloading

by Guest on 2019/10/24 09:08:40 PM    
sometimes different groups put out exactly the same torrent with only a few bits extra files added but since the main file is exactly the same between all the torrents it should be possible to merge them all into one download.
right now pointing three torrents at the same location sadly makes them eventually fail

would be awesome if that could be supported

it would also make sense to have it automaticall detect if two torrents contain files with the same hash and notify the user if he wants to merge them.
by Guest on 2019/11/07 08:32:27 PM    
up to this. i had 3~tb in many hds in 2009~ and seeded during the nights/idle. when i did develop a hash program and stuff to check duplicates and everything like that i notice that 4 torrents had the exact same file of 500mb. i just maded a hardlink . the problem is that all the 4 files is 1 filestream and any of them are modified i lose the file.

also to implement the idea. download from many torrent sources and many https sources as well.
by notaLamer on 2020/04/25 08:02:57 AM    
It's not easy the way Bittorrent is designed.
If you know how .tar format works it's like that: Bittorrent is a continuous stream of data split into pieces.[1] That's why Vuze (Azureus now BiglyBT) swarm-merging mode is so problematic. It's a very opportunistic algorithm.
For the purposes of the other keys, the multi-file case is treated as only having a single file by concatenating the files in the order they appear in the files list. The files list is the value files maps to, and is a list of dictionaries containing the following keys:
[1] - from Bittorrent specification, BEP-3

Examples:
Torrent1
- File1.zip
- File2.zip

Torrent2
- File1.zip
- File2.zip
- Readme.txt

Torrent3
- Readme-by-John.txt
- File1.zip
- File2.zip

Torrent 1 & 2 have fully compatibly pieces to each other. The only difference is the .txt file appended at the end. Torrent3's pieces are NOT compatible with neither 1 NOR 2 because the Readme.txt at the beginning shifted all following data because it was prepended to the data stream.
Whether or not the torrents are 'easily compatible' (1&2) depends on the order the seeder's torrent client was adding (hashing) the files to the torrent. Theoretically there's no excuse for clients not to support this type of torrent merging (if the order is equal).

While Torrent3 has almost equal data, but has pieces that are totally different. If you are downloading 1/2 and wanted to merge in pieces from 3, you would need at least 2 pieces (don't mind different piece sizes for this example) from 3 to assemble one full block for 1/2. Same math going the other way around. At this point you still don't know if the data is indeed the same - you have to verify the hash of torrent 1/2 to see if your frankenstein piece from 3 is indeed correct. If it is then you can save it to HDD.

If after over 15 years the clients still have not adopted this approach there is little chance they will.

That's why torrent trackers should STOP adding their advertisements/readme to already working torrents otherwise they are breaking the global bittorrent swarm. And you should stop being part of such trackers. No one profits from dead torrents.

The only legitimate use case of merging torrents is the one shown on Vuze wiki: Libreoffice torrent 1 only has Libreoffice as part of it. Libreoffice torrent 2 has Libreoffice and the extension pack for it added. It makes sense there to merge torrents. Ideally though the person creating the torrent would add the extension pack at the very beginning and create only one and not two separate torrents. The people downloading will then choose to not download the extension pack file.
Of course it's not always possible to act this way for us. The technology where each separate file is a separate 'building block' is IPFS: there you can have 'downloads' that each consist of separately seeded files that are fetched from the global network. But Bittorrent isn't built this way.




This web site is powered by Super Simple Server