2

I'm using pv for sending files via ssh.

I can change "active pv" the limit at under 100M without any problem. When i set active pv process to 100M or 1G or higher I cant change rate anymore...

BUT! if i change 5-10 times 1M to 2M, 2M to 1M pv can set sometimes to new rate.

I couldn't find any solution for the problem. Any idea?

Examples:

pv -R "15778"  -f -F "%p***%t***%e***%r***%b" -L 1M  
pv -R "15778"  -f -F "%p***%t***%e***%r***%b" -L 1G  
pv -R "15778"  -f -F "%p***%t***%e***%r***%b" -L 1M (not working anymore)  
  • @roaima its hard to show live process. But i will edit with examples – Morphinz Jan 24 '18 at 13:13
  • Looks like a bug – roaima Jan 24 '18 at 13:25
  • ... or some sort of buffering perhaps — I see the limit being applied even with a single request, however it does take a very long time! – Stephen Kitt Jan 24 '18 at 13:36
  • I wondered about buffering too. Maybe running an unchangeable pv -L 10M </dev/zero as a source to the pv under test can exclude a possible buffering issue – roaima Jan 24 '18 at 14:00
3

This is caused by accounting in pv, which effectively means its rate-limiting is read-limited rather than write-limited. Looking at the source code shows that rate-limiting is driven by a “target”, which is the amount remaining to send. If rate-limiting is on, once per rate limit evaluation cycle, the target is increased by however much we’re supposed to send according to the rate limit; the target is then decreased by however much is actually written. This means that if you set the rate limit to a value larger than the actual write capacity, the target will keep going up; reducing the rate limit won’t then have any effect until pv has caught up with its target (including what it’s allowed to write according to the new rate limit).

To see this in action, start a basic pv:

pv /dev/zero /dev/null

Then control that:

pv -R 32605 -L 1M; sleep 10; pv -R 32605 -L 1G; sleep 1; pv -R 32605 -L 1M

You’ll see the impact of the target calculations by varying the duration of the second sleep...

Because of the write limitation, this only causes an issue when you set the rate limit to a value greater than the write capacity.

In a little more detail, here’s how the accounting works with a flow initially limited to 1M, then to 1G for 5s, then back to 1M, on a connection capable of transmitting 400M:

Time    Rate     Target Sent    Remaining
1       1M       1M     1M      0
2       1G       1G     400M    600M
3       1G       1.6G   400M    1.2G
4       1G       2.2G   400M    1.8G
5       1G       2.8G   400M    2.4G
6       1G       3.4G   400M    3G
7       1M       3001M  400M    2601M
8       1M       2602M  400M    2202M
9       1M       2203M  400M    1803M
10      1M       1804M  400M    1404M
11      1M       1405M  400M    1005M
12      1M       1006M  400M    606M
13      1M       607M   400M    207M
14      1M       208M   208M    0
15      1M       1M     1M      0

It takes 7s for the rate limit to be applied again. The longer the time spent with a high rate limit, the longer it takes for the reduced rate limit to be enforced...

The fix for this is quite straightforward, if you can recompile pv: in loop.c, change line 154 to target = (from target +=), resulting in

                   || (cur_time.tv_sec == next_ratecheck.tv_sec
                       && cur_time.tv_usec >=
                       next_ratecheck.tv_usec)) {
                       target =
                           ((long double) (state->rate_limit)) /
                           (long double) (1000000 /
                                          RATE_GRANULARITY);

Once that’s done, rate limit reductions are applied immediately (well, within one rate-limit cycle).

  • I know I'm not supposed to say +1 but I'm going to do so. At least for a little while. This is a great answer to a question that piqued my curiosity. – roaima Jan 24 '18 at 15:13
0

It seems to be a buffering issue. Here is my test-bed:

pv --pidfile /tmp/pv.pid --rate-limit 1K </dev/zero |
    ssh remote 'cat>/dev/null'

and here is my control:

pv --rate-limit 100M --remote $(cat /tmp/pv.pid)
sleep 1
pv --rate-limit 1K --remote $(cat /tmp/pv.pid)

With a one second interval it takes around 13 seconds for the running pv to reduce from its attempt at 100MB/s (1Gb/s) down to its final target of 1KB/s. Increasing the sleep interval by 1 second increases the time to achieve the final target by almost 10 seconds:

Sleep   Delay
 1       13
 2       22
 3       28
 4       37

Four samples isn't really enough for a trend line so I'm going to avoid suggesting it's a linear correlation.

  • Can you try 500M or 1000M ? I have the problem only when i set higher than 100M i believe. – Morphinz Jan 24 '18 at 14:26
  • You’ll only get the problem with rate-limits higher than your line capacity (see my answer for the explanation). Presumably your line capacity is ~100M. – Stephen Kitt Jan 24 '18 at 14:36
  • @StephenKitt I'm running GbE so I have a theoretical maximum upper limit of 100MB/s, yes. – roaima Jan 24 '18 at 15:08
  • Sorry roaima, I was addressing @Morphinz’s comment ;-). – Stephen Kitt Jan 24 '18 at 15:11
  • @StephenKitt no my line capacity is 10G and i have 40G fiber too. I only set 500 or 1G. – Morphinz Jan 24 '18 at 15:29
0

I am correcting myself; pv can change speed.. I don't know why but it need just some time according to your speed limit... If you set it to 1G, you have to wait 45 sec to reduce speed.
5G - 5 min.
10G - 7 min.

For example:

Commands:

pv --pidfile /tmp/pv.pid --rate-limit 10G </dev/zero | ssh 10.1.1.5 'cat>/dev/null'
pv --rate-limit 1M --remote $(cat /tmp/pv.pid)

-ON 10Gb/s network card:

3.99GiB 0:02:26 [ 157MiB/s] (Right here i just changed to 1M)
26.1GiB 0:02:30 [ 160MiB/s]
77.6GiB 0:09:38 [1.01MiB/s]

After 7 minutes the speed changed finally...

-ON 1Gb/s network card:

I started again with 10G limit.

770MiB 0:00:07 [ 112MiB/s]
44.5GiB 0:06:49 [ 111MiB/s]
46.4GiB 0:07:31 [1.00MiB/s]

Results are the same. If you change speed 10G to 1M you need to wait 7min at least. But if you change speed 1M to 10G you don't need to wait any second. I don't think its just about with buffer because 7min (45Gb) must too big for buffer. but this is just my opinion.

  • Right, my use of the term “buffering” was incorrect and misleading, “accounting” is more appropriate. I’ve added an example to my answer which hopefully will clarify things. – Stephen Kitt Jan 25 '18 at 8:46
  • @StephenKitt Do you have any other idea about the problem? Upgrade works great. Downgrade needs time but why? – Morphinz Jan 25 '18 at 13:46
  • Did you see the example in my updated answer? When downgrading, it takes a while for the target to catch up, and pv doesn’t limit its output. – Stephen Kitt Jan 25 '18 at 13:49
  • @StephenKitt Yes I saw. But 7 min is too long. Can i speed up downgrading? – Morphinz Jan 26 '18 at 7:28
  • You can if you’re able to recompile pv, see my updated answer. I sent the patch to the pv author, perhaps it will be included in a future release. – Stephen Kitt Jan 26 '18 at 7:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.