0

I am using gnu parallel to fire some concurrent process. Below is the command am using.

parallel 'node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES' ::: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 ::: 1 2 3 4 5 6 | grep "Query Executed"|wc -l

As per expectation it should fire 120 parallel process concurrently. But when I did execute the ps command I see only 90 process running. Below is the output.

[prod PTEST1] parunkarthick@dev118:~> ps -ef|grep node
2951      66965 196290 15 07:50 pts/25   00:00:00 perl /usr/bin/parallel node  /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES ::: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 ::: 1 2 3 4 5 6
2951      67082  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 1 1
2951      67087  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 1 2
2951      67088  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 1 3
2951      67093  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 1 4
2951      67098  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 1 5
2951      67103  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 1 6
2951      67108  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 2 1
2951      67113  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 2 2
2951      67118  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 2 3
2951      67123  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 2 4
2951      67128  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 2 5
2951      67133  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 2 6
2951      67138  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 3 1
2951      67143  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 3 2
2951      67148  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 3 3
2951      67153  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 3 4
2951      67158  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 3 5
2951      67163  66965 25 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 3 6
2951      67168  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 4 1
2951      67173  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 4 2
2951      67178  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 4 3
2951      67183  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 4 4
2951      67188  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 4 5
2951      67193  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 4 6
2951      67198  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 5 1
2951      67203  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 5 2
2951      67208  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 5 3
2951      67213  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 5 4
2951      67218  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 5 5
2951      67223  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 5 6
2951      67228  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 6 1
2951      67233  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 6 2
2951      67238  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 6 3
2951      67243  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 6 4
2951      67248  66965 25 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 6 5
2951      67253  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 6 6
2951      67258  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 7 1
2951      67263  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 7 2
2951      67268  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 7 3
2951      67273  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 7 4
2951      67278  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 7 5
2951      67283  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 7 6
2951      67288  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 8 1
2951      67293  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 8 2
2951      67298  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 8 3
2951      67303  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 8 4
2951      67308  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 8 5
2951      67313  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 8 6
2951      67318  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 9 1
2951      67323  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 9 2
2951      67328  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 9 3
2951      67333  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 9 4
2951      67338  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 9 5
2951      67343  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 9 6
2951      67348  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 10 1
2951      67353  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 10 2
2951      67358  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 10 3
2951      67363  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 10 4
2951      67368  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 10 5
2951      67373  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 10 6
2951      67378  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 11 1
2951      67383  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 11 2
2951      67388  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 11 3
2951      67393  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 11 4
2951      67398  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 11 5
2951      67403  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 11 6
2951      67404  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 12 1
2951      67409  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 12 2
2951      67418  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 12 3
2951      67423  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 12 4
2951      67428  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 12 5
2951      67433  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 12 6
2951      67438  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 13 1
2951      67443  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 13 2
2951      67448  66965 24 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 13 3
2951      67453  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 13 4
2951      67458  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 13 5
2951      67463  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 13 6
2951      67468  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 14 1
2951      67473  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 14 2
2951      67478  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 14 3
2951      67483  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 14 4
2951      67488  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 14 5
2951      67493  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 14 6
2951      67498  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 15 1
2951      67503  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 15 2
2951      67508  66965 23 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 15 3
2951      67513  66965 22 07:50 pts/25   00:00:00 node /home/parunkarthick/NodeProject/nodescript_pd.js F_CHARGES 15 4
2951      67523  66905  0 07:50 pts/0    00:00:00 grep node
[prod PTEST1] parunkarthick@dev118:~> ps -ef|grep node|wc -l
90

Any reason why only 90 processes got fired concurrently?

And when I checked the ulimt -u it has the limit as 2000 so really not able to understand why it only spawned 90 parallel process alone.

[prod PTEST1] parunkarthick@dev118:~> ulimit -u
2000

Any help would be really appreciated!!!

  • Did you try replacing the node command with something simple like echo to first find out if this is parallel not forking all the tasks or if the problem is further down? – Patrick Mevzek Jan 2 '18 at 15:24
0

Does this run 250 sleeps?

seq 10000 10250 | parallel -j0 sleep

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.