Tag Info

New answers tagged

1

sh <<-STRESS & $( printf 'myprog &\n%.0b' \ `seq 1 ${MAX_CONCURRENT_PROCS}` ) STRESS echo "$!" I agree with the comment @msw makes above. This will write you a script to be launched by a backgrounded sh process and print out the child sh process's pid so you can monitor it and its children as it works.


0

tmux has this capability. (along with many other useful capabilities in the same vein)


0

Do you need to know the exact code or just whether it failed or not? If you use GNU Parallel instead of xargs you can do: echo $LIST | parallel -d, -P 0 rsync --rsync-path='sudo rsync' ${RSYNC_OPTS} {} ${DEST_DIR} When it completes $? will contain how many jobs failed. To get the exact exit code use '--joblog file' to generate a tab separated file ...


1

I don't see why it would not be possible -- the system can certainly juggle 200 parallel tasks. However, it almost certainly is not desirable, unless there is some specific reason you need that exact number of tasks running in parallel. This seems unlikely; the only reason I could see would be because you need them all existing at the same time because ...


1

It is possible to run parallel in background. Recently, I was working on similar scripts which I needed to execute in parallel and I got the suggestions in this answer. If you go to the link specified as part of my answer, I see the below answer. GNU Parallel can work as a counting semaphore. This is slower and less efficient than its normal mode. An alias ...


1

From Understanding the Linux Kernel: In Linux, process priority is dynamic. The scheduler keeps track of what processes are doing and adjusts their priorities periodically; in this way, processes that have been denied the use of the CPU for a long time interval are boosted by dynamically increasing their priority. Correspondingly, processes running for a ...


0

I actually demonstrate exactly how this sort of thing might be done in another answer here. That answer was to a question about ensuring 2 logs were maintained by a background process, so I demonstrated it with 10. Demo Script cat <<-\DEMO >|${s=/tmp/script} printf 'tty is %s\nparent pid is %s\npid is pid=%s\n' \ "$(tty)" "$PPID" "$$" exec ...


1

In your example the combine command will just be run as soon as the subshell exits (and provided the last background process was started without an error). The subshell will exit immediately after the jobs are started since there is no wait command. If you want to execute a command based on the return value of two or more simultaneous background processes, ...


0

You can use the wait command: (echo starting & sleep 10 & wait) && echo done You can see the "starting" line happens right away, and the "done" waits for 10 seconds.


0

One way to do this could look something like : AnalysisProg <<PREPROCESS /dev/stdin $( { process1=$( pipe | line | 1 >&2 & echo $! ) process2=$( pipe | line | 2 >&2 & echo $! ) while ps -p $process1 $process2 >/dev/null; do sleep 1 done } 2>&1 ) #END PREPROCESS In this way you background both ...


7

cxw's answer is no doubt the preferable solution, if you only have 2 files. If the 2 files are just examples and you in reality have 10000 files, then the '&' solution will not work, as that will overload your server. For that you need a tool like GNU Parallel: ls Data* | parallel 'cat {} | this | that |theother | grep |sed | awk |whatever > ...


0

Try use this. rm -f Data1Res.csv rm -f Data2Res.csv Data1 | this | that |theother | grep |sed | awk |whatever > Data1Res.csv & Data2 | this | that |theother | grep |sed | awk |whatever > Data2Res.csv & while true do ps aux | grep -v grep | grep -i -E 'Data1Res.csv|Data2Res.csv' &> /dev/null if [ $? -ne 0 ] then AnalysisProg -i ...


11

Use wait. For example: Data1 ... > Data1Res.csv & Data2 ... > Data2Res.csv & wait AnalysisProg will: run the Data1 and Data2 pipes as background jobs wait for them both to finish run AnalysisProg. See, e.g., this question.



Top 50 recent answers are included