Help and Support
Ask a question, report a problem, request a feature...
<<  Back To Forum

Fantasy Feature: Dynamic BW Priority skew

by Bugmagnet on 2014/05/27 02:23:20 AM    
I am proposing a system of dynamically adjusting bandwidth priority according to personal preference (BWP), availability, and supply vs demand ratio as an alternative option to the common "stop sharing at xxx% ratio" or stop by time. Let's stop "stopping"!

Here's what I think we need from Kevin and Krew:

1] Break out seed count from the status column. It has to be isolated and available
2] Break out peer count from the status column. It has to be isolated and available
3] add a seed to peer ration column (simple math since those are already available counts)
4] add configurable options for sharing policies based on:
1) personal preference per the existing bandwidth priority (BWP)settings,
2) Supply (# of seeds)
3) Demand (# of peers) and
4] Peer:Seed ratio

The goal is to make the bandwidth priority dynamic, not simply a static setting and then second
define:
TS= total seeds (seeds+1) (1=me)
TSF= assigned count value
BWP = preset BandWidth priority (as is now, default to 5)
SDF = supply and demand factor (ratio)
ABWP= applied BandWidth Priority (supercedes existing preset BWP)
where:
TS>128 TSF=1
TS>64 TSF=2
TS>32 TSF=3
TS>16 TSF=4
TS>8 TSF=5
TS>4 TSF=6
TS>2 TSF=7
TS>1 TSF=8
TS=1 TSF=9  (user is the only seed)
IOW, this factor gives preference to torrents with lesser seed count
and P:S ratios are:
> 5 SDF=9
> 4 SDF=8
> 3 SDF=7
> 2.5 SDF=6
> 2 SDF=5
>1.5 SDF=4
> 1 SDF=3
>.5 SDF=2
<= .5 SDF=1
This factor adjusts crudely for Supply and demand, again adding additional preference for those  for those torrents with many peers and few seeds. (Obviously the ratio factors are bogus as I can't predict the effect - Whether these are left to the user to determine or are preset in code Will need to be decided by Keven & Krew.)

pseudo-code for a simple formula might be:
ABWP =  [/3
or
ABWP =  [/2 + BWP]/2 (gives a greater import to the user set BW Priority)

first is computing the dynamic supply-demand factor, perhaps as
Total Seeds (Tseeds) = seeds + me, so never zero (avoids division by 0 in the ratio math :)
If Tseeds = 1 (just me) then SDF=9
If TSeeds < 4 and peers/TSeeds> 1 then SDF=7
If TSeeds < 4 and peers/TSeeds> 2 then SDF=8
If TSeeds < 4 and peers/TSeeds> 3 then SDF=9
If TSeeds > 30 and peers/TSeeds> 2 then SDF=2
If TSeeds > 30 and peers/TSeeds> 1 then SDF=1
If TSeeds >  20 and peers/TSeeds> 2 then SDF=3
If TSeeds >  20 and peers/TSeeds> 1 then SDF=2
If TSeeds >  10 and peers/TSeeds> 2 then SDF=4
If TSeeds >  10 and peers/TSeeds> 1 then SDF=3
If TSeeds >  3 and peers/TSeeds> 2 then SDF=6
If TSeeds >  3 and peers/TSeeds> 1 then SDF=5
so, for a torrent with a high BW Priority (BWP=7) user selected and a dynamic assessment of supply and demand (2 TSeeds 3 peers (3:2 ratio, SDF 4) the adjusted ABWP (using the second formula example) could be calculated as follows:
ABWP = [[8+4]/2 + 7]/2 = 6.5
for an orphan, 0 other seeds  (TSF= 9) but the same 3 interested peers (3:1 ratio, SDF=7):
ABWP = [[9+7]/2 + 7]/2 = 7.5
If by manual setting of BWP by torrent or category to something other (9?) than the default (5), then:
ABWP = [[9+7]/2 + 9]/2 = 8.5
If I set an ULTRAHigh BW Priority level on a torrent (BWP=9) but it was found to be well supported by 31 other seeds (TSF=4)  and have under 30 peers in need (30:31 ratio, SDF=2) , the BW priority would be adjusted like:
ABWP = [[4+2]/2 + 9]/2 = 6
The latter 2 examples make the point. User had selected these torrents for the max, Ultra High BW Priority (9) but the actual applied BW Priority was skewed to grant more to the one with less proportional seeding available and less for the well seeded one.
The goal would be to allow the user to define bandwidth policy formula  based on those factors, for example:
(If I am the sole seed, you'll get my utmost attention, conditioned on my personal preference for BWP which deems socially significant information gets a higher rating than the p0rn or warez of the week)
Where, personal preference BWP might be 9 for a documentary on Fukushima radiation in tuna and 1 for Miley Cyrus twerking.
Simply adjusting to supply and demand doesn't cut it since the demand is typically for diversions and distractions over education and critical thinking.  I am not sure how to craft the final algorythm but think it would be along these lines.
This enhamced capability would then provide a very critical improvement over the current Settings/Transfers/General/Auto Start/Stop options. We need an alternative stopping, so we can stop stopping. We need to change the narrative to If "I have it, I share it". but user will be able to choose which torrents get preference over others, with this dynamic bandwidth priority control.

Rationale:

Word has it that the average life of a torrent is 37 days
It would take a cultural shift to change that
we need the tools to create that cultural shift.

I read an old article this week that claimed the entire contents of the US Library of Congress was 10 terabytes. I thought really? The file server I am building has 12 TB of raw space. I could actually have everything the LOC has on my low-power micro-computer??? (turns out the author was uninformed and it would take more space than 10TB but in reality 10 TB is a HUGE amount of media...too much to keep to oneself. It HAS to be shared.

I have no clue to Kevin's next amazing project. Maybe it will facilitate the maintenance of a decentralized library of all the known and recorded knowledge in the world. I'll wait on that...for now its tweak tixati time.

The concept of torrent transfers is great. Unfortunately the beancounters, the control freaks, want to ruin it my imposing external ratio enFORCEment. Note, force is imbedded even in the terminology. That speaks to domination not liberty. I digress </rant>

I have spoken before about the need to breakout the peer and seed count from the "status" column. That is critical to my sharing manipulation. I have created a manual workaround but that is tedious and no sane person would even try. Only those with obsessive compulsive tendencies need apply.

My idea is to create a paradigm shift that would make torrents the goto source for information. An alternative to commercial search and be tracked cyberstalking 'services' such as google, yahoo and youtube.  To get there, we can't tolerate a 37 day half-life for new torrents. We need a "once in, always in" solution. The idea of seeding ratios and stopping at xx% needs to be abandoned. Hopefully we can stop the beancounting. In case you noticed, somehow the magic is you can give a file away but you still have it. Magic. That is the gamechanger for the master class that wants to own and control information in the information age and on the information highway. Sorry, but the knowledge of the world belongs to the people of the world, all of them, not the "minority of the opulent" who are so desperate of losing dominion and control they have resorted to assigning the US Department of Homeland Security, established under the ruse of 'protecting against terrorists', to now protect the "intellectual property" of the "owning" class. The reality is that the knowledge they claim to own has been stolen from those that created and developed it over eons of human existence. It is theft of the commons from the commons. I digress? not really </rant>

Back to the paradigm shift and making it possible, more probable. I have abandoned all references to stop sharing a torrent at some arbitrary preset ratio. I figure if I have source to seed what someone needs, the door is open. always. I don't care if the UPload/download ratio for that torrent is 92:1 or not (actual fact). I have dozens of torrents with over a 50:1 ratio, hundreds with a 5:1 ratio. I don't do 1:1 = stop.
Let's all stop stopping and start building a global goto decentralized network for information sharing.
by Pete on 2014/05/27 09:34:29 AM    
I appreciate your effort but I think any such system won't work, or perhaps it will but not as intended. The weak point is bandwidth priority based on seeds and peers numbers. I posted why these numbers are usually not correct and never meaningful, here's the topic: http://forum.tixati.com/support/470/

I think Tixati needs a simpler system to prevent queues stalling. It could be opening more download/upload slots temporarily if average downloading/seeding speed is below set level for chosen period of time. For example: average bandwidth below 50% of bandwidth limit for half an hour will start next queued transfer temporarily (until Tixati is closed). Of course this won't allocate more bandwidth to tasks where it's most needed, but it will at least prevent not uploading at all or at slow speed, while some tasks are waiting in a queue.
by Bugmagnet on 2014/05/28 12:58:05 AM    
Ok... I been thinking this through and think some additional refinement of process might be useful.

It seems that this idea of a dynamic automatic BW adjustment has been thought of and about my several people.
In addition to http://forum.tixati.com/support/470/?p=1#2096  , there were others also:

http://forum.tixati.com/support/46/

http://forum.tixati.com/support/6/

http://forum.tixati.com/support/673/

I started to rethink it when I noticed the aggressive difference when a selected torrent was set to "Ultra High"

It seems much of the concern is over idle UPload BandWidth, with the idea of simply opening an extra slot or 2 until the UL BW limit is reached.

I have opened up over 700 slots so that is not an issue with me. I never have any idle UL BW. The BW graph is always flatlined at the limit I set.  The concern for me is, I like to give some preference to select files. They get higher priority. If there is less demand for these, then the BW is allocated to other files.

In addition to factoring in my personal priority preference as to torrent topic (using the existing BW Priority settings), I have another preference for allotting more BW to torrents with few if any other seeds, as I tried to explain in the original post here.

Pete has indicated that factoring in # or peers and the Seed:Peer ratio are ineffective. I don't see any other means to allocate BW by need. The fact that the reported numbers of seeds and peers might not be precisely accurate or indicate the U/D capacity of those connections doesn't mean they can't be useful factors in allocating BW. Might not be 100 perfect but better than nothing.

I am questioning the relevance of any tracker reported counts anyway. Over the last few hours I started scanning my active connections and found that less then 10% came from trackers. Most of the connections came from PEX and "Incoming" (though I am not clear what that means exactly).

I came up with a temporary, impractical workaround to achieve some semblance of a solution to this BW sharing dilemma.
I sacrificed the use of the "Category" field as a topical grouping and instead use it for manually dividing torrents according to number of seeds and peers. This process is far to tedious without having sortable columns for seeds and peers broken out from the Transfers/Status column, but I did it to test the concept. Don't try that at home. It'll drive you bonkers.

I created 16 categories labeled with a 2-digit number. The first digit  was 0 for 0 seeds, 1 for 1-3 seeds, 5 for 4-8 seeds and 9 for >8 seeds. the second numbers were the same for the number of peers. Think of doing that manually for 700 active torrents and you'll know how crazy I am. Then realize that the number will have changed the next day and the categories will need to be readjusted again. As I said, having to do this manually it totally impractical.  So the torrents in category 15 have 1-3 seeds and 4-8 peers (in theory). Once so grouped, I can select all in that category and apply my BW Priority of choice. The problem is keeping them in the right category corral. That varies too much to be maintained manually.

But it works....to a degree. The more I worked with this, it became evident that the existing BW priority algorithm might need to be reviewed and refined. But that is another matter.

Bottom line, using idle BW to trigger more slots is not useful when BW is constantly maxed out.
The problem is allocating who gets what.

The existing BW priority does address a users sharing preferences, but it does not factor in directing resources to where they are needed the most. For me that calls for focusing on those torrents where there are few if any other seeds and of those, focusing on which have the most peers, iow a low Seed:Peer ratio. Sure, another seed might have a huge BW pipe and skew the results, but if so, then within a short time there will be more seeds and less peers on that torrent.

In addition to creating a dynamic BW allocation algorithm factoring in the number of seeds, the Seed:Peer ratio and the BW Priority, perhaps the UL:DL ratio could be used also to select a different set of values for this, an option that may be useful or deemed desirable, where after a preset ratio has been reached, the BW priority would be reduced somewhat.  This of course provides a very viable alternative to simply stop sharing a torrent at some point in time or by ratio.




This web site is powered by Super Simple Server