Help and Support
Ask a question, report a problem, request a feature...
<<  Back To Forum

Piece completion priority algorithm and UI request

by Guest on 2019/06/10 11:05:41 AM    
Three small things that have been bugging me for feature requests.

First is the way Tixati manages piece completion. I am noticing that even when there are MANY incomplete pieces stored, it seems to be the algorithm preference for connections to either request or accept brand new pieces rather than completing the ones that are already partially downloaded. This is extremely annoying when conections are poor or peers regularly drop out, as you end up with many wasteful incompletes instead of focussing on completing pieces that only need a few more Kb. The goal of downloading is to COMPLETE! If a peer can supply them, priority should be given to incomplete parts, the most nearly complete first, and only THEN should new parts be opened, otherwise you are potentially wasting bandwidth (and storage SOMEWHERE) if all these parts are never completed. This could be managed smartly, so in particular torrents with very low peer numbers and download speeds are forced to complete incomplete pieces where possible to maximise completion. Think of it like the "superseeding" option that maximises remote upload completion, but on the sub-piece scale for downloading.

This has an impact on wherever all these incomplete parts are being stored, of course. I've noticed "Trying to save piece" messages so I am suspecting that they are NOT stored in situ... Clearly the BEST and most efficient place for them would be to store incoming data directly IN PLACE in the output file. That way you (a) don't have to waste time moving it in later, (b) always have as much good data as possible in the output file rather than storing incompletes separately in case the torrent fails, when a 95% complete piece stored in the file will produce a much smaller video glitch or data loss than a whole missing part, and (c) aren't hammering another disk with all that incoming data - probably the C: drive which for most of us by now will be an SSD, so we definitely DON'T want massive amounts of download data being stored there as it considerably reduces the lifespan. (This is even more critical as the resiliance of new SSDs is going down significantly as MLC is replaced by TLC and QLC). Since there is some sort of mechanism going on to keep track of which segments of an incomplete are downloaded, it shouldn't matter whether they are being stored separately or just saved straight into the output file, and checking wouldn't be affected. Just don't download EVERYTHING into a hidden folder under C:\Users\AppData\Tixati or wherever before saving out, because that's going to kill SSD boot drives over time.

The second thing is another request for more sanity in the UI. If it needs to be toggleable with a user preference, fine, but PLEASE reduce the utterly meaningless precision in the Time Left column. Two levels only is reasonable and meaningful: days and hours, hours and minutes, but never bother with seconds until there are under 10 minutes left. The way download speeds vary (especially over DAYS, never mind hours or minutes) it is completely bogus information that just clutters up an already busy interface. The constantly changing seconds value also distracts the eye from more critical information as well as wasting CPU cycles updating the interface with garbage data. I'd suspect that most people would find a shorter completion string easier to read and understand too without all the extra:useless:numbers.

Where the interface DOES need more precision is for completion values under 1% and over 99%. The difference between actual 0% and say 0.1% is significant for understanding how (or whether) a torrent is downloading, as is between 99.0% and 99.9%. Between 1% and 1.1% not so much. That's an easy UI fix.

Thanks for listening.
by Guest on 2019/06/11 07:23:41 AM    
Just to amplify the point about where incomplete parts are being saved... if everything incoming really IS being saved to a central location and collated into complete parts BEFORE being saved out into the correct place in the output file, then that is the MOST inefficient way possible of handling it. You are literally doubling the file I/O of Tixati, as everything is being written twice, and that central disk is taking the entire download bandwidth unnecessarily (TWICE - both writing AND reading it out again) when it should just be spread evenly over all the disks you are ultimately downloading files to - almost certainly NOT anyone's boot drive. That's not really good for the I/O footprint of Tixati as a background application, let alone for that central disk if an SSD.

You have data structures for each torrent noting which parts are complete, and which subparts of incomplete parts have been downloaded. There is no conceptual or practical difference between having those data structures point to incompletes as tiny part files somewhere or to the relevant portion of the ultimate output file, which only contains junk data until those parts are saved in. All you are doing is creating extra unnecessary part management resaving and deleting completed parts.

Having incoming data saved directly in the file instead of accumulating remotely is better for the downloader, as currently until a part is saved in there is no benefit to having a 95% complete part, as if never completed it will never be merged into the output file. That's potentially throwing data and bandwidth away. You cannot assume that ALL torrents complete, so saving as much as possible of a file in situ is critical. Since it's over-writing junk data anyway from the original file creation, it makes no difference if the part data turns out bad when it gets verified: just set the part to download again fresh from different peers.

I'd add another very simple and useful UI request - some way of noting easily in the list interface (and in the details tab of the dialog for each torrent) whether with the CURRENT peers it is possible to complete downloading. It just needs an icon (say a green tick), or a change of colour in the listing. Knowing whether the only current peer in a very slow torrent with few seeds has 57.7% is less important than knowing whether that 57.7% overlaps with all your missing parts and you have a chance of completing eventually.
by Guest on 2019/06/11 10:03:21 AM    
Here's a perfect example of what I think seems to be wrong with the piece completion algorithm...

This is at the end of a long and painful download with a single seed that was rarely online. Twenty-five pieces left - almost all partially complete. The torrent is set to download aggressive sequential (Seq+). This seed has just disconnected, then reconnected, and instead of completing ANY of the already partially downloaded parts, including the one just above it that it was sending when it disconnected, it has started sending a BRAND NEW part, leaving all the other partials incomplete. Not only that, but the Seq+ setting should have encouraged (or IMHO, at this stage FORCED) it to complete the earliest of these partials first, including the one that's almost complete, not start from near the end.

Completion of existing partials should be a top priority that supercedes any choice of the remote client, or poor algorithmic choice of Tixati in requesting parts. That way completion of WHOLE parts is maximised, the torrent has better health and is less susceptible to seed outtages. As things are at the moment, there's a significant chance of data loss when every partial, even those almost complete, counts for precisely NOTHING when they are stored separately outside the output file. Prioritising partials would mitigate this problem.

This web site is powered by Super Simple Server