Running commands in parallel, with or without GNU parallel. For virtualization on Macs, see /parallels. For parallel ports, see /parallel-port.

learn more… | top users | synonyms

1
vote
4answers
104 views

How can i run a piece of code in background?

Can I run a piece of code in background instead of using another script. [sesiv@itseelm-lx4151 ~]$ cat ./testback2 #!/bin/bash start_time=$(date +%s) for i in {1..5} do ./testscript & done wait ...
1
vote
1answer
27 views

Is it safe to resize a partition while writing to it?

I'm working on a script for automatically setting up Amazon Linux servers. I create them with 100gb virtual disks, but the main partition is always 8gb. No problem, I call sudo resize2fs /dev/sda1 at ...
1
vote
1answer
23 views

Can GNU Parallel execute more parallel processes?

Can I for example execute: parallel -j 200 < list0 Where "list" has: nice -n -20 parallel -j 100 < list2 nice -n -20 parallel -j 100 < list1 Would this be feasible/possible?
1
vote
1answer
20 views

Why does parallel slow down after a while?

I'm executing 60 scripts with GNU parallel(they all have wget commands in there)but I have noticed that after a few hours execution will slow down a bit. What could be causing this? I'm executing ...
1
vote
1answer
39 views

Gnu Parallel: more than one per CPU? [closed]

If I only have two cores, does parallel only run two processes at a time? I guess I'm still confused about processes, threads, and cores....
1
vote
1answer
37 views

Memory-efficient utility to return N first sorted values

I would like to implement a very popular MapReduce example using only existing programs operating in a UNIX way. The problem is to find N most frequent values in an enormous amount of data. The ...
1
vote
1answer
36 views

How to get exit code of a parallel process launched by xargs?

I'd made a script for launching parallel rsync process: #! /bin/bash LIST=$1 DEST_DIR=$2 RSYNC_OPTS=$3 #echo "rsyncing From=$SRC_DIR To=$DEST_DIR RSYNC_OPTS=$RSYNC_OPTS" echo $LIST|xargs -n1 -d, ...
1
vote
1answer
60 views

Strategies for building distributed computing system with hybrid Mac and Linux systems

I want to build a distributed computing system to run Matlab, C and other programming languages for scientific computing. Now I've several old Mac machines with Lion Mac OS installed acted as web ...
1
vote
0answers
237 views

MPI mpdboot error

I have a problem with mpi in a 3-box cluster with machines named head, node1 and node2 respectively. I install a newer version on node2 and the problems arises. When I include only head and node2 in ...
0
votes
1answer
24 views

Starting GNU Parallel in background?

Is it possible to start GNU Parallel in the background, or somehow, send it to the background? I want to do this so that I can use two commands that run parallel at the same time. And why dont I ...
0
votes
3answers
44 views

running an executable in parallel by dividing the input directory tree

Commands like find and grep would benefit greatly if they were to run in parallel. Is there a utility or a program that takes a command as an argument and makes it run in parallel by running multiple ...
0
votes
1answer
553 views

Rsync from multiple folders to one

Is it safe to have multiple folders Rsync simultaneously into one? Each of the source folders contains a subset of the destination folder, and everybody just syncs into a master destination folder. I ...
0
votes
1answer
69 views

ERROR: cube viewer is not available!

I have installed and configured the following: cube-4.2.1 opari2-1.1.1 papi-5.2.0 scalasca-2.0 scorep-1.2.1 I am trying to run scalasca -examine, but I get the following error: ERROR: cube viewer ...
0
votes
0answers
35 views

log4cxx/logger.h or libcql/cql.hpp fatal error while doing make -j4

I am running make -j4 and I am getting below exception... userName@phx5qa01c-4e23:~/build$ make -j4 [ 1%] [ 2%] Building CXX object nda/common/loadtest/CMakeFiles/loadtester.dir/LoadTester.cc.o ...
-1
votes
1answer
48 views

GNU Parallel slows down

I'm learning GNU parallel usage and did the following: $ for i in {1.txt,2.txt}; do time wc -l $i; done 100 1.txt real 0m0.010s user 0m0.000s sys 0m0.010s 10000012 2.txt real 0m0.069s ...