Log In     Register    

Help and Support
Ask a question, report a problem, request a feature...
<<  Back To Forum

Downloads while checking a complete torrent

by Guest on 2013/12/02 12:25:19 AM    
I decided to try Tixati today, abandoning Deluge for good (its daemon had a tendency to hang without crashing, and the only way to notice it was to see if the current transfer speed varied or stayed fixed).

So when I was importing my entire library of torrent files, and painstakingly show Tixati where the files for each torrent was, it started checking the file integrity. So far so good, the speed was OK and everything seems to work. But then I started noticing my DL speed peaking, and this was on torrents Tixati was currently checking.

Is this a bug, or is this how Tixati is programmed to behave? I don't understand where the data goes either, because the check completes and download stops, and all files work. So why does it start downloading a torrent that still isn't done being checked? Seems like a ridiculous waste of bandwith, especially considering that all the 150 torrents I added was complete.

A "workaround" for this was stopping all torrents, and checking one at a time. This only allowed for the current torrent being checked to download, which finishes before a substantial amount has been transferred.

Edit:
Here's a screenshot of it happening: http://i.imgur.com/jRhOR3v.png
This is NOT good for any sites that tracks UL/DL ratio.

Windows 7, x64. The data for these torrents are located on both an internal 5400rpm drive and an external 7200rpm USB3.0 drive, and I don't use a page file.
by VigilanteP on 2013/12/02 01:58:09 AM    
I noticed the same thing as well after files were relocated outside of Tixati and were later corrected within it.  

Just to add a bit more information: When it finished checking a torrent, after having downloaded what it could during that process, the progress would be set to 99% and some pieces would be incomplete.  So that seems to be where the data is going... it is overwriting some of the completed pieces, at least until the check finishes and it finishes downloading the missing bits.
by Pete on 2013/12/02 11:22:46 AM    
Yes, for now when adding a complete torrent it is better to Stop that torrent on Loading Transfer window and Force Check manually when not running (right click a torrent > Local Files > Force Check). For large amount of torrents you can first load them stopped and later Force Check all of them.
by Guest on 2013/12/02 01:13:08 PM    
I'm guessing "checking" is a torrent state, so a simple check for it before starting download shouldn't be hard to implement.

Luckily this is far from an app-breaking bug, because I'm really starting to like Tixati. I drooled when I first opened the settings window and seeing all the things I wanted to personalize, good job guys!
by Guest on 2024/10/01 07:55:27 PM    
WHY IS THIS STILL NOT AN OPTION?
by Guest on 2024/10/03 04:11:06 PM    
About the 99% thing... Some people may not be aware of how torrents implement the hash checks internally.

Many years ago, I was into newsgroups - back then the standard verification procedure there was a hash/repair mechanism called PAR2 (since updated to PAR3). Basically, the files are split into equal-sized chunks and each piece is hashed with both an MD5 and a CRC. The CRC is a fast hash so you can sanity-check the piece roughly, then if that confirms a data match the MD5 is much slower but more definitive (far less likely than the CRC to produce false hash positives). Much like a torrent file, you'd download the separate PAR2 file for the set from the newsgroup which would list the filenames and sizes and order in the set and give the hash checks for every piece.

PAR2 also allowed repair, as there was a stupendously funky piece of maths doing much the same thing as RAID5/6 does with XOR when a disk in a RAID set goes offline, which allowed you to pre-create repair blocks from the original fileset which you could then use to fix small errors ANYWHERE in the set! I corresponded with the writer of PAR2 and wrote my own program for fun (in Visual Basic LOL! With Assembler routines added...) to work with Agent to automatically repair newsgroup filesets (before that capability was added to Agent) so I guess I know a little about it... Even if you don't use newsgroups, it's still a very useful archiving tool for data that's unlikely to change because of the check/repair mechanism - check out quickpar.org.uk

Torrents work exactly the same way. You put all the files end-to-end like a big long data sausage and chop that up into equal-sized pieces, then hash the pieces so you can check the validity when they are downloaded. I don't know the technical details of bittorrent (how they are hashed etc.) but the principle is identical to PAR2. But both systems have to deal with a small problem - files are never conveniently an exact multiple of the size of the chunk/piece you are splitting to. So the chunk at the end of a file will contain data from the start of the next. And the chunk at the start of a file will contain data from the previous one.

If you are downloading ALL the files in a torrent, that's not a problem as you'll get ALL the data. But if you skip a file, however small, then the hashed piece at the end of the previous file is not fully downloaded EVEN IF you've downloaded the whole file and it's complete as far as your file system is concerned. So when Tixati or any other torrent program checks the file, although it's got ALL the data for the file, it hasn't got that little piece of the chunk at the end (from the start of the missing file), so it can't verify that piece as it can't create the hash without it. So the file will be marked as 99% complete and the chunk will be marked missing. And the only way to fix that is to download the WHOLE missing piece, you can't just get the missing end data. All this discussion is exactly the same for chunks at the START of files too, where the previous file was skipped.

Now, actually PAR2 contained hashes for the complete files in the set as well as for the pieces, so it COULD verify whole files without needing to do it at the piece level and deal with those data over-runs in the end pieces. That may not be the case with bittorrent (it may have been removed from the spec to keep the torrent files smaller) so it'll probably always have to redownload the starts/ends of some files to complete a set even if you've previously checked it in another client - but that should only need to happen if you skipped a file and that's why some data is missing.
by Guest on 2024/10/04 08:38:18 PM    
DOWNLOADS SHOULD NEVER HAPPEN WHEN FILECHECK IS ON. THIS IS VERY BAD DESIGN




This web site is powered by Super Simple Server