Questions tagged [multithreading]
The multithreading tag has no usage guidance.
3
votes
1answer
48 views
Is mkdir -p safe to parallelize?
I'm writing Makefile rules like this:
out/%.x: ...
mkdir -p $(@D)/
...
I intend to parallelize this Makefile (-j), which will trigger many mkdir -p's to simultaneously.
Is this safe? Or do ...
2
votes
0answers
50 views
Atomic write and tell for files in O_APPEND mode on linux
I am dealing with a situation where I want to save state persistently for an application that has many processes and many threads within those processes. Normally this would require a heavy use of ...
0
votes
0answers
20 views
How does linux arbitrate access to devices
I would like to know the different methods linux uses to control access to a peripheral device. e.g. for example you have a spi temperature sensor. And multiple threads are trying to get a reading. ...
3
votes
1answer
231 views
'pthread_create: Resource temporarily unavailable' when running multiple docker instances
TLDR
When spinning up multiple docker containers in which I run npm ci, I start getting pthread_create: Resource temporarily unavailable errors (less than 5 docker containers can run fine). I deduce ...
0
votes
0answers
51 views
How can I multithread a for loop to run 1000 times at once?
I have a for loop that is curling the contents of a file, the file contains 2 million web addresses.
I woud like to create a for loop that instead of curling the address 1 at a time, curls 1000 at ...
1
vote
0answers
31 views
Can the overhead of rwlock is higher than the overhead of mutex? [closed]
I implemented a sorted linked list as two versions. First one is using single mutex for the entire list and the sencond one is single rwlock for the entire list. I run half of the operations as Member ...
0
votes
1answer
82 views
Is it possible to manage total threads for pipes of multi-threaded commands?
I am running several tools that have multi-threading capabilities. For example:
tool1 --threads=4 | tool2 --threads=4 | tool3 --threads=4
In an ideal case, I assume this should utilize 12 (4*3) ...
1
vote
1answer
33 views
Distribution of threads among CPUs?
lscpu gives:
Thread(s) per core: 2
Core(s) per socket: 32
When running an intensive 32-threads process, why does htop show almost 100% CPU activity on #1-32, but very little activity on #33-64? ...
1
vote
1answer
193 views
Do threads interrupt each other in Linux? [closed]
I'm learning about critical section of multithreading. I have a general statement:
In a single CPU system, disable interrupt is a solution of race condition.
But I also learn from another site ...
1
vote
0answers
140 views
Resize window in multi-thread ncurses program
The overall question is simple:
How to gracefully resize a window in a multi-thread ncurses program?
Details here. I have read a few relevant questions (1, 2, 3). From my understanding, there are ...
0
votes
0answers
13 views
Thread scheduling behavior between steps of GDB compared to non-stepped execution
The GDB manual states that when using all-stop mode for debugging a multithreaded application, it is not possible to advance every thread in lock-step by exactly one statement. This makes sense since ...
1
vote
0answers
38 views
Does `perf trace -p <PID>` trace all threads?
strace -f -p <PID> traces system calls in all threads of a multi-threaded process. (This also follows forks, there is an option -b execve that tries to address that).
Does perf trace -p <...
8
votes
2answers
3k views
Why on modern Linux, the default stack size is so huge - 8MB (even 10 on some distributions)
For example, on OSX, it's even less than 512k.
Is there any recommended size, having in mind, that the app does not use recursion and does not allocate a lot of stack variables?
I know the question ...
0
votes
0answers
29 views
How do you determine the priority of Linux system processes?
I've currently inherited some legacy code in which the user program spawns two threads running at priority level 49.
Here's a snippet for one of the threads:
SchedParam.sched_priority = 49;
if(...
6
votes
1answer
3k views
Understanding output of lscpu
You can see the output from lscpu command -
jack@042:~$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 56
On-...
1
vote
1answer
232 views
Daemon vs worker and when to use what?
(Note that, below, I am frequently using term "daemon or worker" just to indicate that I dont know which word to use as I dont know the exact difference between them in the context explained below.)
...
2
votes
1answer
68 views
Run process as if on a single-core machine to find a bug
I have a bug in my Linux app that is reproducable only on single-core CPUs.
To debug it, I want to start the process from the command line so that it is limited to 1 CPU even on my multi-processor ...
1
vote
2answers
122 views
Stop rsync scheduled task race condition (large directory, small time interval)
Platform information:
OpenBSD 6.2 amd64
$ rsync --version
rsync version 3.1.2 protocol version 31
I'm trying to sync a large directory (4TB) using the following daily.local file (for Linux admins, ...
1
vote
0answers
31 views
Reconciling %CPU with load average in top
We have a long-running process, which utilizes OpenMP to saturate all available CPUs with computations done in multiple threads (one for each core).
I'm looking at top(1) on a server, where the ...
0
votes
1answer
396 views
How to optimise Aria2 when downloading thousands of small files?
My starting point is this question. Now aria2 is downloading the files but it is not as fast as I expected it (I am running this on a Macbook Pro with an i7, gigabit connection and AC-wifi. I am most ...
1
vote
0answers
237 views
Mutual exclusion between all Linux threads and processes [closed]
I'm trying to write a piece of code that should not been executed simultaneously neither by threads of the same process, nor by different processes. In other words, I need global synchronization ...
1
vote
1answer
307 views
Why is my program slower, despite using more threads? [closed]
I am new to threading , and I wanted to test my newly acquired skills, with a simple task, create an image using multiple threads, the interesting part is that , on a single thread , the program runs ...
2
votes
2answers
203 views
Can I run Linux on specific CPU cores?
I'm making a brainwave detection device.
Will be using an Octa-Core ARM processor for the software processing.
For the management of CPU cores, can I code Linux to strictly run on a specific set of ...
1
vote
3answers
2k views
How to bash multithread?
#!/bin/bash
while IFS="," read ip port; do
ruby test.rb "http://$ip:$port/"&
ruby test.rb "https://$ip:$port/";
done <test1.txt
How would i do this multithreading?
if i do more lines ...
0
votes
1answer
106 views
Multi-threading with for loop [duplicate]
This is part of a bash script code I wish to use multi-threading
for DATABASE in $DATABASES ; do
echo Converting $DATABASE
# Check if the table is MyISAM (we don't want to convert InnoDB ...
2
votes
0answers
287 views
Shielding CPU from scheduler - How to move bound kernel threads from cores?
Suppose I have 4 physical cores (indexed 0-3) and I'm dedicating cores 1-3 to a shield for user applications and trying to steer all system threads onto core 0. I'm following the answers here and ...
0
votes
1answer
29 views
Threads and Their Execution [closed]
Sometimes threads fell that they run long time, and they give their turns to another threads by means of thread scheduling procedure.
1) I wonder what happens if a thread does not want to stop ...
1
vote
1answer
71 views
Run C++ program across computer on network
I have a c++ program that is multithreaded. I believe the throughput would increase if I could run it in the other computers connected by a switch.
All of then are using the same OS(Ubuntu).
Is there ...
4
votes
0answers
305 views
How to mesure the latency of thread switching
I'm learning how to test the performance of a unix-like system.
I've found a tool to measure the context switching time: http://www.bitmover.com/lmbench/lat_ctx.8.html
Now I want to know if it is ...
1
vote
0answers
306 views
why is vim not multi threaded? [closed]
I'm working on a fairly large project and often have multiple buffers (and windows) open at the same time. Now whenever I would do a :windo w! or :bufdo w! to save all open files, vim uses 100% cpu ...
1
vote
2answers
407 views
How to comprehend Cpus_allowed and thread id number?
Environment: OS --debian + python3.
All the output info below ommit unimportant.
Get my computer's cpu info with cat /proc/cpuinfo :
cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu ...
1
vote
1answer
474 views
How to split a file to multiple files with multiple threads?
So I have a 100GB text files And I want to split it into 10000 files.
I used to do such tasks with something like:
split -l <number of lines> -d --additional-suffix=.txt bigfile ...
-5
votes
1answer
45 views
I have to convert a simple program to multiple threads. So that my execution time may reduce [closed]
My task is to make a generic code (using shell prog. Language) which prompts the file name from user then execute the program and break the program in multiple threads (in number of threads a/c to the ...
1
vote
1answer
53 views
When running multicore jobs, will I lose performance if system load averages above the number of cores?
I do a lot of statistical analysis with R, and heavily utilize large multicore instances on AWS. Mostly for hyperparameter searches, cross-validation, and bootstrapping.
Say I have an instance ...
4
votes
1answer
315 views
How to delay bash script until there's enough idle cores to run it?
I'm a student who wants to benchmark a NGS pipeline, monitoring performance according to how many cores it has allocated to it and the size of the input file. For this reason, I wrote a bash script to ...
2
votes
1answer
1k views
Check CPU/thread usage for a node in the Slurm job manager
I am working on a cluster machine that uses the Slurm job manager. I just started a multithreaded code and I would like to check the core and thread usage for a given node ID. For example,
scoreusage ...
1
vote
3answers
6k views
run command on multiple threads
I am running a command (pngquant to be precise: https://github.com/pornel/pngquant) in a terminal window. I noticed, that if I open 4 terminal windows, and run pngquant command in each of them, I get ...
4
votes
2answers
5k views
Bash script multithreading in curl commands
here is a shell script which takes domain and its parameters to find status code . this runs way faster due to threading but misses lot of requests.
while IFS= read -r url <&3; do
while ...
1
vote
0answers
42 views
Multithreading repartition in *nix system
How the reparation of threads per CPU is handle on *nix system?
For example, one process uses 37 threads, machine has 4 cores. Usually all threads are placed on 2 cores. Sometimes at 3 cores. And ...
3
votes
1answer
3k views
What is a limit for number of threads?
I was wondering how many processes can I create on my machine (x64 with 8Gb of RAM and running Ubuntu). So I made simple master process which was continiously creating child processes, and that child ...
4
votes
0answers
175 views
kill -STOP shuts down [closed]
On a 12-core Linux machine, as a user (without root privilege) I start a job on a single process. When I hit Ctrl-Z the process is suspended and kill -CONT resumes it. This works fine.
But when the ...
0
votes
2answers
131 views
Parallel tasks and killing process
I have 3 tasks in a script which have to run as follow.
#1 and 2 need to run in parallel
task1 &
task2
#when task 1 is complete, task 3 is launched
task3
My problem is that task 2 being a ...
1
vote
0answers
62 views
Can i use kernel_thread in kernel when its version is over 3.10?
In my module code, i want to create my thread which doesn't fork from kthreadd, because kthreadd will clone_fs from init, then the fs of init was chroot, so it's not good for me, i just want to create ...
0
votes
1answer
85 views
poor mans backup with find and parallel compression
I'm planning to backup the /home folder of my Kubuntu box to an external drive.
In order to speed up the process I consider zipping large files (individually) so that writing them is faster. I also ...
0
votes
1answer
646 views
Multithreaded applications on a single threaded CPU?
The motivation behind this question arises from exploring the Intel Galileo gen2 board which has a single threaded processor.
I'm looking for a conceptual explanation on what does that mean for all ...
2
votes
0answers
587 views
Why is Firefox stuck, and how might I get it unstuck?
Firefox is hung. Usually I kill it with -1, -9 or -15, but this time, dang it, I want to find out why. I have an hour or so to waste on this, and seek education above actually solving the problem. ...
-1
votes
1answer
69 views
Are there UNIX variants or other OS which make user level threads visible to OS kernel? [closed]
Are there UNIX variants or other OS which make user level threads visible to OS kernel ?
0
votes
1answer
26 views
Is it possible that the kernel doesn't let a already ready thread run even when there is a idle processor?
Suppose each thread's affinity is all the processors. Whenever a thread becomes ready, will the kernel attempt to find an idle processor to run it?
0
votes
1answer
624 views
Processor affinity not respected by iperf threads
I'm running iperf in server mode on a Ubuntu box (14.04 LTS). The hardware is quad core with hyper-threading, so I have cores 0-7 available (0 is paired with 4, 1 with 5 and so on).
I've set the ...
6
votes
1answer
5k views
How do I specify which core a pthread is spawned on
I am running a cluster where each node has an Intel Xeon E5430. /proc/cpuinfo reports 8 cores. I am using C/C++ compiled with gcc ver 5.3.1 on an Ubuntu 16.04LTS.
Distributing my work to each node ...
