AMBIGUOUS TAG — DO NOT USE. For the Mac virtualization software, use [parallels]. For the `parallel` utilities, use [gnu-parallel] or [moreutils-parallel]. For parallel computing in general, use [parallelism]. For parallel ports once common on PCs, use [parallel-port].

learn more… | top users | synonyms

2
votes
2answers
1k views

Can GNU Parallel execute more parallel processes?

Can I for example execute: parallel -j 200 < list0 Where "list" has: nice -n -20 parallel -j 100 < list2 nice -n -20 parallel -j 100 < list1 Would this be feasible/possible?
2
votes
1answer
1k views

How to run Abinit via MPI?

I'm trying to run Abinit in parallel, since in sequential mode it takes too much time. I have followed all steps in the abinit tutorials to run it in parallel, and they say to use OpenMPI, but I ...
2
votes
3answers
181 views

How to split the array in set of five files and download them in parallel?

I am trying to copy files from testMachineB and testMachineC into testMachineA as I am running my shell script on testMachineA. If the file is not there in testMachineB, then it should be there in ...
2
votes
1answer
1k views

Can I force Linux to boot its initializations parallel? [closed]

I was wondering if I could manage to initialize drivers, services etc. (all these jobs what Linux does during startup) in parallel instead of sequentially. I want to aggressively lower the boot time. ...
2
votes
1answer
213 views

replacement inside parallel command string

I want to download multiple files and save them to the md5 sum of its name: cat list | parallel -j4 "md5=$(wget -O - {} | tee tmpfile_{#} | md5sum | cut -d ' ' -f 1); mv tmpfile_{#} $md5" but the ...
2
votes
1answer
138 views

Find | parallel executing script with path from find + other arguments

Due to my lack of Bash knowledge I've tried for a few hours now to get something like this to work: find Directories -mindepth 4 -type d -print0 | parallel -0 -j0 ./MyScript -d {Found Directory} {1} :...
2
votes
1answer
594 views

How to get exit code of a parallel process launched by xargs?

I'd made a script for launching parallel rsync process: #! /bin/bash LIST=$1 DEST_DIR=$2 RSYNC_OPTS=$3 #echo "rsyncing From=$SRC_DIR To=$DEST_DIR RSYNC_OPTS=$RSYNC_OPTS" echo $LIST|xargs -n1 -d, echo|...
2
votes
2answers
1k views

scp does not display output when used with gnu parallel

I have a short script that uses scp to copy files to a number of remote hosts (yes, I know about rdist and rsync; they both fail to work for a few of the hosts - that's not the point here; I'm only ...
2
votes
2answers
683 views

Parallel vs Distributed vs Traditional File system

I am trying to understand the differences between these three file system at a very basic level. Distributed FS: HDFS Parallel FS : Lustre Traditional FS : ext4/ext3/ NTFS/FAT etc. I want to ...
1
vote
1answer
6k views

Is it possible to run two commands at the same time in a shell script?

I know that on the command line I can use & to run a command in the background. But I'm wondering if I can do it in a script. I have a script like this: date_stamp=$(date +"%Y-%m-%d" --date='...
1
vote
2answers
186 views

How can I count the time that needs systemd to boot a default target and then graph it?

When I wanted to know how long systemd actually needed to boot the default target, how would I do that ? An then, is it possible to create a graph to show which unit takes how much time to initialize ...
1
vote
2answers
28 views

GNU Parallel and sshpass with server list in a loop

With this loop we sequential update on all servers (server list = consul members | grep awk {'print $2'} | cut -d ":" -f1) the package consul. for i in $(consul members | grep awk {'print $2'} | cut -...
1
vote
3answers
134 views

tarring in parallel

Ab Oceanographer friend at work needs to back up many months worth of data. She is overwhelemed so I volunteered to do it. There are hundreds of directories to be backed up and we want to tar/bzip ...
1
vote
1answer
26 views

submitting jobs to get 3 nodes running parallel executions

I have a submit script looks like below, it tries to run a large number of instances of csce.py in backgrounds with 3 nodes.... in a laptop, this usually could successfully automatically distribute ...
1
vote
1answer
237 views

How to copy in two folders simultaneously using GNU parallel by spawning multiple threads?

I am trying to copy files from machineB and machineC into machineA as I am running my below shell script on machineA. If the files is not there in machineB then it should be there in machineC for ...
1
vote
3answers
493 views

running an executable in parallel by dividing the input directory tree

Commands like find and grep would benefit greatly if they were to run in parallel. Is there a utility or a program that takes a command as an argument and makes it run in parallel by running multiple ...
1
vote
1answer
474 views

Is there a simple way of limiting the number of child processes that execute in parallel?

I have a bash script that kicks off multiple rsync processes in parallel. I'd like to limit the number of simultaneous child processes but not serialize the whole script. Here's a simplified version ...
1
vote
4answers
188 views

How can i run a piece of code in background?

Can I run a piece of code in background instead of using another script. [sesiv@itseelm-lx4151 ~]$ cat ./testback2 #!/bin/bash start_time=$(date +%s) for i in {1..5} do ./testscript & done wait ...
1
vote
2answers
88 views

Run multiple scripts and continue until every script is done

my question is, can I run multiple scripts at once. I have access to a cluster with 32 cpus but at the same time I have 900 scripts. Now what I would like to do is, put every script in a queue, and ...
1
vote
1answer
61 views

Is it safe to resize a partition while writing to it?

I'm working on a script for automatically setting up Amazon Linux servers. I create them with 100gb virtual disks, but the main partition is always 8gb. No problem, I call sudo resize2fs /dev/sda1 at ...
1
vote
1answer
1k views

Rsync from multiple folders to one

Is it safe to have multiple folders Rsync simultaneously into one? Each of the source folders contains a subset of the destination folder, and everybody just syncs into a master destination folder. I ...
1
vote
2answers
62 views

Need to format 5 disks in parallel

I tried this xargs -p 5 -n1 mke2fs -t ext4 /dev/sd[d-p]1 and got this [/dev/mapper]# seq 1 15 | xargs -p 5 -n 1 mke2fs -t ext4 /dev/sd[b-p]1 5 -n 1 mke2fs -t ext4 /dev/sdb1 /dev/sdc1 /dev/sdd1 /...
1
vote
1answer
432 views

Few questions about system calls and kernel modules (kernel services in parallel)

I have few questions about system calls and kernel modules. Let's assume we have two applications (A and B) and each of them is running on a different core. (A-CPU 0, B-CPU 1) When two applications ...
1
vote
1answer
581 views

How to launch multiple instances of a program in parallel using parallel?

I would like to execute a command: curl example.com multiple times in parallel using GNU parallel. I tried: parallel -j 3 curl example.com but it doesn't work.
1
vote
1answer
83 views

Wine and parallel processing

I'm quite new to linux, right now, running LMDE on my machine. I was wondering if wine can support the parallel processing needs of some software. Anyone knows any performance benchmark of wine ? ...
1
vote
1answer
210 views

Starting GNU Parallel in background?

Is it possible to start GNU Parallel in the background, or somehow, send it to the background? I want to do this so that I can use two commands that run parallel at the same time. And why dont I ...
1
vote
1answer
197 views

Why does parallel slow down after a while?

I'm executing 60 scripts with GNU parallel(they all have wget commands in there)but I have noticed that after a few hours execution will slow down a bit. What could be causing this? I'm executing ...
1
vote
1answer
579 views

Gnu Parallel: more than one per CPU? [closed]

If I only have two cores, does parallel only run two processes at a time? I guess I'm still confused about processes, threads, and cores....
1
vote
1answer
77 views

Memory-efficient utility to return N first sorted values

I would like to implement a very popular MapReduce example using only existing programs operating in a UNIX way. The problem is to find N most frequent values in an enormous amount of data. The ...
1
vote
0answers
82 views

GNU parallel hangs when using --load 100% argument

On a machine with RH5 (Scientific Linux SL release 5.1), I have no issues with using the '--load 100%' argument. On a machine with RH6 (Scientific Linux release 6.2), the parallel hangs after printing ...
1
vote
0answers
36 views

It is possible to run these in GNU parallel [closed]

I would like to run these: screen watch -n 0 php /usr/share/nginx/html/2siti.php & screen watch -n 0 php /usr/share/nginx/html/2siti.php & screen watch -n 0 php /usr/share/nginx/html/2siti....
1
vote
1answer
334 views

Strategies for building distributed computing system with hybrid Mac and Linux systems

I want to build a distributed computing system to run Matlab, C and other programming languages for scientific computing. Now I've several old Mac machines with Lion Mac OS installed acted as web ...
1
vote
0answers
512 views

MPI mpdboot error

I have a problem with mpi in a 3-box cluster with machines named head, node1 and node2 respectively. I install a newer version on node2 and the problems arises. When I include only head and node2 in ...
0
votes
2answers
307 views

Why does GNU Parallel slow down?

I'm learning GNU parallel and tried the following: $ for i in {1.txt,2.txt}; do time wc -l $i; done 100 1.txt real 0m0.010s user 0m0.000s sys 0m0.010s 10000012 2.txt real 0m0.069s user ...
0
votes
1answer
77 views

Parallelize openssl as input to dd

I am using @tremby's great idea to fill a disk with random data. This involves piping openssl, which is encrypting a lot of zeros, to dd (bs=4M). I'm maxing out the single core on which this is ...
0
votes
1answer
477 views

Running shell script on cluster in parralel

I thought dsh(Distributed Shel) will be a good option but when i ran my shell script across my nodes, I didnt get expected output, dsh -aM -c bash /home/cloudera/bash_script.sh server2@x.x.x.110: ...
0
votes
1answer
195 views

ERROR: cube viewer is not available!

I have installed and configured the following: cube-4.2.1 opari2-1.1.1 papi-5.2.0 scalasca-2.0 scorep-1.2.1 I am trying to run scalasca -examine, but I get the following error: ERROR: cube viewer ...
0
votes
0answers
167 views

log4cxx/logger.h or libcql/cql.hpp fatal error while doing make -j4

I am running make -j4 and I am getting below exception... userName@phx5qa01c-4e23:~/build$ make -j4 [ 1%] [ 2%] Building CXX object nda/common/loadtest/CMakeFiles/loadtester.dir/LoadTester.cc.o ...