1

I am running GNU parallel and after some time I am getting:

parallel: Warning: no more file handles Raising ulimit -n or etc/security/limits.conf may help.

What argument would be appropriate to add to the parallel command in order to overcome this?

I changed limits.conf to unlimited but then I could not use sudo or login as root or ssh to my box same issue like here

here is the piece of code that I am using. I have 2 files, one with passwords second with hosts.

passPasswords_and_hosts() {
        `sudo sshpass -p "$1" ssh -o ConnectTimeout=2 root@"$2" -o StrictHostKeyChecking=no "$command_linux"`
}
export -f testone
export -p command_linux
parallel --tag -k passPasswords_and_hosts :::: "$passwords" "$linux_hosts"
5
  • 1
    I think we need to see the command you are running and a small example that can be reproduced. – jesse_b Dec 17 '20 at 15:10
  • Can you post output of ulimit -a? – Ole Tange Dec 18 '20 at 8:22
  • @OleTange core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 256634 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited – dwt.bar Dec 21 '20 at 16:55
  • @OleTange and the rest: max user processes (-u) 4096 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited – dwt.bar Dec 21 '20 at 16:58
  • 'open files (-n) 1024' You need to increase this - just like the warning says. Each parallel job takes 4 file handles. – Ole Tange Dec 21 '20 at 17:36
3

To avoid having tmp files laying around after abnormal termination (think kill -9 or system crash), GNU Parallel opens tmp files and removes them immediately. But it keeps the files open.

To meet the requirement of --keep-order it has to keep all the files open that have not been printed yet. So if you have 1000000 commands and command 2 is stuck forever, then GNU Parallel will happily run command 3 and beyond, but if command 2 does not come unstuck, then GNU Parallel will eventually run out of file handles (it uses 4 file handles per job).

In your case your passPasswords_and_hosts probably becomes stuck at some point. In your output it would be the job following the last output (i.e. the job not yet printed).

So try running that job by hand and see if there is some obvious problem.

You can also remove -k. Then your stuck job will still use 4 file handles, but all the following jobs that completes, will not, as they will be printed when they are done.

Finally you can use --timeout. I normally use --timeout 1000%. This means that if a job takes longer than 10x the median run time of all successful jobs, then it is killed. It works for a remarkable range of situations.

2
  • Thank you, that is very helpful, I tested the IP where my job is getting stuck. So nmap mistakenly recognized OS as Linux while it was windows. It was corrected once I used "nmap -F -O " instead of "nmap -O -p 22". So when I sshpass to it I get: "searching for password prompt using match "assword" SSHPASS read: Username" I also tried to just ssh to it and I cannot log in to it as root (because it's not a Linux machine) I am looking for a way to move on to next ip and kill this process if something like that happens – dwt.bar Dec 21 '20 at 16:50
  • so I just tested --timeout and it works like a charm. from GNU Parallel documentation: If jobs taking more than a certain amount of time are known to fail, they can be stopped with --timeout. The accuracy of --timeout is 2 seconds: parallel --timeout 4.1 sleep {}\; echo {} ::: 2 4 6 8 Output: 2 4 GNU parallel can compute the median runtime for jobs and kill those that take more than 200% of the median runtime: parallel --timeout 200% sleep {}\; echo {} ::: 2.1 2.2 3 7 2.3 Output: 2.1 2.2 3 – dwt.bar Dec 21 '20 at 17:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.