0

I do some scientific calculation on a PC(in fact several PCs) and I want to know how many jobs should I submit one time. lscpu shows:

CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    2
Core(s) per socket:    4
Socket(s):             1

The ambiguity is 'Thread'. I searched the net and learned something about it. But I still felt confused (it is said that how many jobs should I submit is depend). I do not care about the details of machines. For example, now I have an executable file. If I run it directly, it spends about 10 min. Assume now I have 800 of them need to be run. Should I run 4 of them a time or 8 to reduce the total time cost?

  • Is your PC dedicated to this one task, or must some resources be retained for other programs? – RonJohn Jan 15 '18 at 3:58
  • I use this PC to run jobs only. By the way, I use bsub (openlava). – hengyue li Jan 15 '18 at 4:48
1

If this PC has an Intel CPU, then the Thread(s) per core most certainly indicates hyper-threading.

https://en.wikipedia.org/wiki/Hyper-threading

For each processor core that is physically present, the operating system addresses two virtual (logical) cores and shares the workload between them when possible. The main function of hyper-threading is to increase the number of independent instructions in the pipeline; it takes advantage of superscalar architecture, in which multiple instructions operate on separate data in parallel.

.

Should I run 4 of them a time or 8 to reduce the total time cost?

It depends. Some tasks run faster under hyper-threading, and some don't. You'll have to test that yourself.

Assume now I have 800 of them need to be run.

I'd use GNU Parallel to handle this problem.

https://www.gnu.org/software/parallel/

GNU parallel is a shell tool for executing jobs in parallel using one or more computers.

If you've got a list of of files in . which need to be processed, this will work:

find . | parallel -j4 yourprogram

If your earlier tests show that it runs faster with hyper-threading, then change the "4" to an "8".

EDIT: forgot to mention that sometimes programs run faster when you disable HT in the BIOS.

  • I made the program myself. Is there any guideline I can follow to write the code so that I can know if I use 4 or 8? – hengyue li Jan 15 '18 at 4:50
  • I don't know. Here's one link, though: software.intel.com/en-us/articles/… – RonJohn Jan 15 '18 at 5:40
  • You may have to use a number less than 4 if the memory requirements are large (you don't want to have thrashing) or there's a lot of file i/o required relative to the disk throughput that you have. Or you could use a number higher than 8 if each job alternates between periods of pure CPU usage and i/o wait. Best thing is to test. – Mark Plotnick Jan 15 '18 at 7:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.