Questions tagged [multithreading]

The tag has no usage guidance.

3
votes
1answer
48 views

Is mkdir -p safe to parallelize?

I'm writing Makefile rules like this: out/%.x: ... mkdir -p $(@D)/ ... I intend to parallelize this Makefile (-j), which will trigger many mkdir -p's to simultaneously. Is this safe? Or do ...
2
votes
0answers
50 views

Atomic write and tell for files in O_APPEND mode on linux

I am dealing with a situation where I want to save state persistently for an application that has many processes and many threads within those processes. Normally this would require a heavy use of ...
0
votes
0answers
20 views

How does linux arbitrate access to devices

I would like to know the different methods linux uses to control access to a peripheral device. e.g. for example you have a spi temperature sensor. And multiple threads are trying to get a reading. ...
3
votes
1answer
231 views

'pthread_create: Resource temporarily unavailable' when running multiple docker instances

TLDR When spinning up multiple docker containers in which I run npm ci, I start getting pthread_create: Resource temporarily unavailable errors (less than 5 docker containers can run fine). I deduce ...
0
votes
0answers
51 views

How can I multithread a for loop to run 1000 times at once?

I have a for loop that is curling the contents of a file, the file contains 2 million web addresses. I woud like to create a for loop that instead of curling the address 1 at a time, curls 1000 at ...
1
vote
0answers
31 views

Can the overhead of rwlock is higher than the overhead of mutex? [closed]

I implemented a sorted linked list as two versions. First one is using single mutex for the entire list and the sencond one is single rwlock for the entire list. I run half of the operations as Member ...
0
votes
1answer
82 views

Is it possible to manage total threads for pipes of multi-threaded commands?

I am running several tools that have multi-threading capabilities. For example: tool1 --threads=4 | tool2 --threads=4 | tool3 --threads=4 In an ideal case, I assume this should utilize 12 (4*3) ...
1
vote
1answer
33 views

Distribution of threads among CPUs?

lscpu gives: Thread(s) per core: 2 Core(s) per socket: 32 When running an intensive 32-threads process, why does htop show almost 100% CPU activity on #1-32, but very little activity on #33-64? ...
1
vote
1answer
193 views

Do threads interrupt each other in Linux? [closed]

I'm learning about critical section of multithreading. I have a general statement: In a single CPU system, disable interrupt is a solution of race condition. But I also learn from another site ...
1
vote
0answers
140 views

Resize window in multi-thread ncurses program

The overall question is simple: How to gracefully resize a window in a multi-thread ncurses program? Details here. I have read a few relevant questions (1, 2, 3). From my understanding, there are ...
0
votes
0answers
13 views

Thread scheduling behavior between steps of GDB compared to non-stepped execution

The GDB manual states that when using all-stop mode for debugging a multithreaded application, it is not possible to advance every thread in lock-step by exactly one statement. This makes sense since ...
1
vote
0answers
38 views

Does `perf trace -p <PID>` trace all threads?

strace -f -p <PID> traces system calls in all threads of a multi-threaded process. (This also follows forks, there is an option -b execve that tries to address that). Does perf trace -p <...
8
votes
2answers
3k views

Why on modern Linux, the default stack size is so huge - 8MB (even 10 on some distributions)

For example, on OSX, it's even less than 512k. Is there any recommended size, having in mind, that the app does not use recursion and does not allocate a lot of stack variables? I know the question ...
0
votes
0answers
29 views

How do you determine the priority of Linux system processes?

I've currently inherited some legacy code in which the user program spawns two threads running at priority level 49. Here's a snippet for one of the threads: SchedParam.sched_priority = 49; if(...
6
votes
1answer
3k views

Understanding output of lscpu

You can see the output from lscpu command - jack@042:~$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 56 On-...
1
vote
1answer
232 views

Daemon vs worker and when to use what?

(Note that, below, I am frequently using term "daemon or worker" just to indicate that I dont know which word to use as I dont know the exact difference between them in the context explained below.) ...
2
votes
1answer
68 views

Run process as if on a single-core machine to find a bug

I have a bug in my Linux app that is reproducable only on single-core CPUs. To debug it, I want to start the process from the command line so that it is limited to 1 CPU even on my multi-processor ...
1
vote
2answers
122 views

Stop rsync scheduled task race condition (large directory, small time interval)

Platform information: OpenBSD 6.2 amd64 $ rsync --version rsync version 3.1.2 protocol version 31 I'm trying to sync a large directory (4TB) using the following daily.local file (for Linux admins, ...
1
vote
0answers
31 views

Reconciling %CPU with load average in top

We have a long-running process, which utilizes OpenMP to saturate all available CPUs with computations done in multiple threads (one for each core). I'm looking at top(1) on a server, where the ...
0
votes
1answer
396 views

How to optimise Aria2 when downloading thousands of small files?

My starting point is this question. Now aria2 is downloading the files but it is not as fast as I expected it (I am running this on a Macbook Pro with an i7, gigabit connection and AC-wifi. I am most ...
1
vote
0answers
237 views

Mutual exclusion between all Linux threads and processes [closed]

I'm trying to write a piece of code that should not been executed simultaneously neither by threads of the same process, nor by different processes. In other words, I need global synchronization ...
1
vote
1answer
307 views

Why is my program slower, despite using more threads? [closed]

I am new to threading , and I wanted to test my newly acquired skills, with a simple task, create an image using multiple threads, the interesting part is that , on a single thread , the program runs ...
2
votes
2answers
203 views

Can I run Linux on specific CPU cores?

I'm making a brainwave detection device. Will be using an Octa-Core ARM processor for the software processing. For the management of CPU cores, can I code Linux to strictly run on a specific set of ...
1
vote
3answers
2k views

How to bash multithread?

#!/bin/bash while IFS="," read ip port; do ruby test.rb "http://$ip:$port/"& ruby test.rb "https://$ip:$port/"; done <test1.txt How would i do this multithreading? if i do more lines ...
0
votes
1answer
106 views

Multi-threading with for loop [duplicate]

This is part of a bash script code I wish to use multi-threading for DATABASE in $DATABASES ; do echo Converting $DATABASE # Check if the table is MyISAM (we don't want to convert InnoDB ...
2
votes
0answers
287 views

Shielding CPU from scheduler - How to move bound kernel threads from cores?

Suppose I have 4 physical cores (indexed 0-3) and I'm dedicating cores 1-3 to a shield for user applications and trying to steer all system threads onto core 0. I'm following the answers here and ...
0
votes
1answer
29 views

Threads and Their Execution [closed]

Sometimes threads fell that they run long time, and they give their turns to another threads by means of thread scheduling procedure. 1) I wonder what happens if a thread does not want to stop ...
1
vote
1answer
71 views

Run C++ program across computer on network

I have a c++ program that is multithreaded. I believe the throughput would increase if I could run it in the other computers connected by a switch. All of then are using the same OS(Ubuntu). Is there ...
4
votes
0answers
305 views

How to mesure the latency of thread switching

I'm learning how to test the performance of a unix-like system. I've found a tool to measure the context switching time: http://www.bitmover.com/lmbench/lat_ctx.8.html Now I want to know if it is ...
1
vote
0answers
306 views

why is vim not multi threaded? [closed]

I'm working on a fairly large project and often have multiple buffers (and windows) open at the same time. Now whenever I would do a :windo w! or :bufdo w! to save all open files, vim uses 100% cpu ...
1
vote
2answers
407 views

How to comprehend Cpus_allowed and thread id number?

Environment: OS --debian + python3. All the output info below ommit unimportant. Get my computer's cpu info with cat /proc/cpuinfo : cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu ...
1
vote
1answer
474 views

How to split a file to multiple files with multiple threads?

So I have a 100GB text files And I want to split it into 10000 files. I used to do such tasks with something like: split -l <number of lines> -d --additional-suffix=.txt bigfile ...
-5
votes
1answer
45 views

I have to convert a simple program to multiple threads. So that my execution time may reduce [closed]

My task is to make a generic code (using shell prog. Language) which prompts the file name from user then execute the program and break the program in multiple threads (in number of threads a/c to the ...
1
vote
1answer
53 views

When running multicore jobs, will I lose performance if system load averages above the number of cores?

I do a lot of statistical analysis with R, and heavily utilize large multicore instances on AWS. Mostly for hyperparameter searches, cross-validation, and bootstrapping. Say I have an instance ...
4
votes
1answer
315 views

How to delay bash script until there's enough idle cores to run it?

I'm a student who wants to benchmark a NGS pipeline, monitoring performance according to how many cores it has allocated to it and the size of the input file. For this reason, I wrote a bash script to ...
2
votes
1answer
1k views

Check CPU/thread usage for a node in the Slurm job manager

I am working on a cluster machine that uses the Slurm job manager. I just started a multithreaded code and I would like to check the core and thread usage for a given node ID. For example, scoreusage ...
1
vote
3answers
6k views

run command on multiple threads

I am running a command (pngquant to be precise: https://github.com/pornel/pngquant) in a terminal window. I noticed, that if I open 4 terminal windows, and run pngquant command in each of them, I get ...
4
votes
2answers
5k views

Bash script multithreading in curl commands

here is a shell script which takes domain and its parameters to find status code . this runs way faster due to threading but misses lot of requests. while IFS= read -r url <&3; do while ...
1
vote
0answers
42 views

Multithreading repartition in *nix system

How the reparation of threads per CPU is handle on *nix system? For example, one process uses 37 threads, machine has 4 cores. Usually all threads are placed on 2 cores. Sometimes at 3 cores. And ...
3
votes
1answer
3k views

What is a limit for number of threads?

I was wondering how many processes can I create on my machine (x64 with 8Gb of RAM and running Ubuntu). So I made simple master process which was continiously creating child processes, and that child ...
4
votes
0answers
175 views

kill -STOP shuts down [closed]

On a 12-core Linux machine, as a user (without root privilege) I start a job on a single process. When I hit Ctrl-Z the process is suspended and kill -CONT resumes it. This works fine. But when the ...
0
votes
2answers
131 views

Parallel tasks and killing process

I have 3 tasks in a script which have to run as follow. #1 and 2 need to run in parallel task1 & task2 #when task 1 is complete, task 3 is launched task3 My problem is that task 2 being a ...
1
vote
0answers
62 views

Can i use kernel_thread in kernel when its version is over 3.10?

In my module code, i want to create my thread which doesn't fork from kthreadd, because kthreadd will clone_fs from init, then the fs of init was chroot, so it's not good for me, i just want to create ...
0
votes
1answer
85 views

poor mans backup with find and parallel compression

I'm planning to backup the /home folder of my Kubuntu box to an external drive. In order to speed up the process I consider zipping large files (individually) so that writing them is faster. I also ...
0
votes
1answer
646 views

Multithreaded applications on a single threaded CPU?

The motivation behind this question arises from exploring the Intel Galileo gen2 board which has a single threaded processor. I'm looking for a conceptual explanation on what does that mean for all ...
2
votes
0answers
587 views

Why is Firefox stuck, and how might I get it unstuck?

Firefox is hung. Usually I kill it with -1, -9 or -15, but this time, dang it, I want to find out why. I have an hour or so to waste on this, and seek education above actually solving the problem. ...
-1
votes
1answer
69 views

Are there UNIX variants or other OS which make user level threads visible to OS kernel? [closed]

Are there UNIX variants or other OS which make user level threads visible to OS kernel ?
0
votes
1answer
26 views

Is it possible that the kernel doesn't let a already ready thread run even when there is a idle processor?

Suppose each thread's affinity is all the processors. Whenever a thread becomes ready, will the kernel attempt to find an idle processor to run it?
0
votes
1answer
624 views

Processor affinity not respected by iperf threads

I'm running iperf in server mode on a Ubuntu box (14.04 LTS). The hardware is quad core with hyper-threading, so I have cores 0-7 available (0 is paired with 4, 1 with 5 and so on). I've set the ...
6
votes
1answer
5k views

How do I specify which core a pthread is spawned on

I am running a cluster where each node has an Intel Xeon E5430. /proc/cpuinfo reports 8 cores. I am using C/C++ compiled with gcc ver 5.3.1 on an Ubuntu 16.04LTS. Distributing my work to each node ...