5 added 1032 characters in body
source | link

On a GNU system and if you have pv, you could do:

cmd='
   that command | to execute &&
     as shell code'

yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh

The -P20 is to execute at most 20 $cmd at the same time.

-L10 limits the rate to 10 bytes per second, so 5 lines per second.

If your $cmds become two slow and causes the 20 limit to be reached, then xargs will stop reading until one $cmd instance at least returns. pv will still carry on writing to the pipe at the same rate, until the pipe gets full (which on Linux with a default pipe size of 64KiB will take almost 2 hours).

At that point, pv will stop writing. But even then, when xargs resumes reading, pv will try and catch up and send all the lines it should have sent earlier as quickly as possible so as to maintain a 5 lines per second average overall.

What that means is that as long as it's possible with 20 processes to meet that 5 run per second on average requirement, it will do it. However when the limit is reached, the rate at which new processes are started will not be driven by pv's timer but by the rate at which earlier cmd instances return. For instance, if 20 are currently running and have been for 10 seconds, and 10 of them decide to finish all at the same time, then 10 new ones will be started at once.

Example:

$ cmd='date +%T.%N; exec sleep 2'
$ yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh
09:49:23.347013486
09:49:23.527446830
09:49:23.707591664
09:49:23.888182485
09:49:24.068257018
09:49:24.338570865
09:49:24.518963491
09:49:24.699206647
09:49:24.879722328
09:49:25.149988152
09:49:25.330095169

On average, it will be 5 times per second even if the delay between two runs will not always be exactly 0.2 seconds.

With ksh93 (or with zsh if your sleep command supports fractional seconds):

typeset -F SECONDS=0
n=0; while true; do
  your-command &
  sleep "$((++n * 0.2 - SECONDS))"
done

That puts no bound on the number of concurrent your-commands though.

On a GNU system and if you have pv, you could do:

cmd='
   that command | to execute &&
     as shell code'

yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh

The -P20 is to execute at most 20 $cmd at the same time.

-L10 limits the rate to 10 bytes per second, so 5 lines per second.

Example:

$ cmd='date +%T.%N; exec sleep 2'
$ yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh
09:49:23.347013486
09:49:23.527446830
09:49:23.707591664
09:49:23.888182485
09:49:24.068257018
09:49:24.338570865
09:49:24.518963491
09:49:24.699206647
09:49:24.879722328
09:49:25.149988152
09:49:25.330095169

On average, it will be 5 times per second even if the delay between two runs will not always be exactly 0.2 seconds.

With ksh93 (or with zsh if your sleep command supports fractional seconds):

typeset -F SECONDS=0
n=0; while true; do
  your-command &
  sleep "$((++n * 0.2 - SECONDS))"
done

That puts no bound on the number of concurrent your-commands though.

On a GNU system and if you have pv, you could do:

cmd='
   that command | to execute &&
     as shell code'

yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh

The -P20 is to execute at most 20 $cmd at the same time.

-L10 limits the rate to 10 bytes per second, so 5 lines per second.

If your $cmds become two slow and causes the 20 limit to be reached, then xargs will stop reading until one $cmd instance at least returns. pv will still carry on writing to the pipe at the same rate, until the pipe gets full (which on Linux with a default pipe size of 64KiB will take almost 2 hours).

At that point, pv will stop writing. But even then, when xargs resumes reading, pv will try and catch up and send all the lines it should have sent earlier as quickly as possible so as to maintain a 5 lines per second average overall.

What that means is that as long as it's possible with 20 processes to meet that 5 run per second on average requirement, it will do it. However when the limit is reached, the rate at which new processes are started will not be driven by pv's timer but by the rate at which earlier cmd instances return. For instance, if 20 are currently running and have been for 10 seconds, and 10 of them decide to finish all at the same time, then 10 new ones will be started at once.

Example:

$ cmd='date +%T.%N; exec sleep 2'
$ yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh
09:49:23.347013486
09:49:23.527446830
09:49:23.707591664
09:49:23.888182485
09:49:24.068257018
09:49:24.338570865
09:49:24.518963491
09:49:24.699206647
09:49:24.879722328
09:49:25.149988152
09:49:25.330095169

On average, it will be 5 times per second even if the delay between two runs will not always be exactly 0.2 seconds.

With ksh93 (or with zsh if your sleep command supports fractional seconds):

typeset -F SECONDS=0
n=0; while true; do
  your-command &
  sleep "$((++n * 0.2 - SECONDS))"
done

That puts no bound on the number of concurrent your-commands though.

4 deleted 3 characters in body
source | link

On a GNU system and if you have pv, you could do:

cmd='
   that command | to execute &&
     as shell code'

yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh

The -P20 is to execute at most 20 $cmd at the same time.

-L10 limits the rate to 10 bytes per second, so 5 lines per second.

Example:

$ cmd='date +%T.%N; exec sleep 0.3'2'
$ yes | pv -qL10 | xargs -n1 -P200P20 sh -c "$cmd" sh
09:49:23.347013486
09:49:23.527446830
09:49:23.707591664
09:49:23.888182485
09:49:24.068257018
09:49:24.338570865
09:49:24.518963491
09:49:24.699206647
09:49:24.879722328
09:49:25.149988152
09:49:25.330095169

On average, it will be 5 times per second even if the delay between two runs will not always be exactly 0.2 seconds.

With ksh93 (or with zsh if your sleep command supports fractional seconds):

typeset -F SECONDS=0
n=0; while true; do
  your-command &
  sleep "$((++n * 0.2 - SECONDS))"
done

That puts no bound on the number of concurrent your-commands though.

On a GNU system and if you have pv, you could do:

cmd='
   that command | to execute &&
     as shell code'

yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh

The -P20 is to execute at most 20 $cmd at the same time.

-L10 limits the rate to 10 bytes per second, so 5 lines per second.

Example:

$ cmd='date +%T.%N; exec sleep 0.3'
$ yes | pv -qL10 | xargs -n1 -P200 sh -c "$cmd"
09:49:23.347013486
09:49:23.527446830
09:49:23.707591664
09:49:23.888182485
09:49:24.068257018
09:49:24.338570865
09:49:24.518963491
09:49:24.699206647
09:49:24.879722328
09:49:25.149988152
09:49:25.330095169

On average, it will be 5 times per second even if the delay between two runs will not always be exactly 0.2 seconds.

With ksh93 (or with zsh if your sleep command supports fractional seconds):

typeset -F SECONDS=0
n=0; while true; do
  your-command &
  sleep "$((++n * 0.2 - SECONDS))"
done

That puts no bound on the number of concurrent your-commands though.

On a GNU system and if you have pv, you could do:

cmd='
   that command | to execute &&
     as shell code'

yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh

The -P20 is to execute at most 20 $cmd at the same time.

-L10 limits the rate to 10 bytes per second, so 5 lines per second.

Example:

$ cmd='date +%T.%N; exec sleep 2'
$ yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh
09:49:23.347013486
09:49:23.527446830
09:49:23.707591664
09:49:23.888182485
09:49:24.068257018
09:49:24.338570865
09:49:24.518963491
09:49:24.699206647
09:49:24.879722328
09:49:25.149988152
09:49:25.330095169

On average, it will be 5 times per second even if the delay between two runs will not always be exactly 0.2 seconds.

With ksh93 (or with zsh if your sleep command supports fractional seconds):

typeset -F SECONDS=0
n=0; while true; do
  your-command &
  sleep "$((++n * 0.2 - SECONDS))"
done

That puts no bound on the number of concurrent your-commands though.

3 added 492 characters in body
source | link

On a GNU system and if you have pv, you could do:

cmd='
   that command | to execute &&
     as shell code'

yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh

The -P20 is to execute at most 20 $cmd at the same time.

-L10 limits the rate to 10 bytes per second, so 5 lines per second.

Example:

$ cmd='date +%T.%N; exec sleep 0.3'
$ yes | pv -qL10 | xargs -n1 -P200 sh -c "$cmd"
09:49:23.347013486
09:49:23.527446830
09:49:23.707591664
09:49:23.888182485
09:49:24.068257018
09:49:24.338570865
09:49:24.518963491
09:49:24.699206647
09:49:24.879722328
09:49:25.149988152
09:49:25.330095169

On average, it will be 5 times per second even if the delay between two runs will not always be exactly 0.2 seconds.

With ksh93 (or with zsh if your sleep command supports fractional seconds):

typeset -F SECONDS=0
n=0; while true; do
  your-command &
  sleep "$((++n * 0.2 - SECONDS))"
done

That puts no bound on the number of concurrent your-commands though.

On a GNU system and if you have pv, you could do:

cmd='
   that command | to execute &&
     as shell code'

yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh

The -P20 is to execute at most 20 $cmd at the same time.

-L10 limits the rate to 10 bytes per second, so 5 lines per second.

Example:

$ cmd='date +%T.%N; exec sleep 0.3'
$ yes | pv -qL10 | xargs -n1 -P200 sh -c "$cmd"
09:49:23.347013486
09:49:23.527446830
09:49:23.707591664
09:49:23.888182485
09:49:24.068257018
09:49:24.338570865
09:49:24.518963491
09:49:24.699206647
09:49:24.879722328
09:49:25.149988152
09:49:25.330095169

On average, it will be 5 times per second even if the delay between two runs will not always be exactly 0.2 seconds.

On a GNU system and if you have pv, you could do:

cmd='
   that command | to execute &&
     as shell code'

yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh

The -P20 is to execute at most 20 $cmd at the same time.

-L10 limits the rate to 10 bytes per second, so 5 lines per second.

Example:

$ cmd='date +%T.%N; exec sleep 0.3'
$ yes | pv -qL10 | xargs -n1 -P200 sh -c "$cmd"
09:49:23.347013486
09:49:23.527446830
09:49:23.707591664
09:49:23.888182485
09:49:24.068257018
09:49:24.338570865
09:49:24.518963491
09:49:24.699206647
09:49:24.879722328
09:49:25.149988152
09:49:25.330095169

On average, it will be 5 times per second even if the delay between two runs will not always be exactly 0.2 seconds.

With ksh93 (or with zsh if your sleep command supports fractional seconds):

typeset -F SECONDS=0
n=0; while true; do
  your-command &
  sleep "$((++n * 0.2 - SECONDS))"
done

That puts no bound on the number of concurrent your-commands though.

2 added 492 characters in body
source | link
1
source | link