Questions tagged [multithreading]
The multithreading tag has no usage guidance.
113
questions
0
votes
1answer
25 views
Thread safe next number (or round-robin) service in linux
I have an Ubuntu machine and a scenario in which I need to generate the next number (starting from X) between 0 and 20 in a round-robin way.
I need a way to get the next number which will be thread ...
0
votes
0answers
20 views
Linux pthread_cond_wait hangs in NASM
The following NASM code uses pthread_cond_wait to take 4 cores (numbered 0, 8, 16, 24) in core order, starting with core 0. After each core locks the mutex, it enters mutex_lock_loop where it tests ...
1
vote
0answers
32 views
Using libhugetlbfs to transparently back up glibc malloc calls in a multi-threaded application
I'm trying to back up the memory allocations of a multi-threaded application with 1GiB hugepages using libhugetlbfs. However, only the main thread allocations are being assigned hugepages. If I ...
1
vote
2answers
221 views
What types of threads does Java/JVM use from Linux OS's perspective?
A developer friend of mine recently asked the question: On a Linux system when a Java application runs which has threads, how do these threads appear to the underlying Linux OS?
So what are Java ...
0
votes
1answer
384 views
Multithreaded xz, with gzip, pv, and pipes - is this the most efficient I can get?
I'm excited to learn that xz now supports multithreading:
xz --threads=0
But now I want to utilise this as much as possible. For example, to recompress gzips as xz:
gzip -d -k -c myfile.gz | pv | ...
3
votes
1answer
44 views
Why is this code exiting with return code 16?
I am trying my hands on clone() system call to create a Thread. However program is terminating itself as it return from t2_thread() function. Why is this behaviour? What am I missing?
#define ...
1
vote
0answers
108 views
How to correctly create and stop kthreads?
I wrote the following kernel module:
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/delay.h>
#include <linux/kthread.h>
#include <...
0
votes
1answer
32 views
How to limit the amount of spillover to other CPUs from a single process
Please excuse the complete ignorance of my question - I'm probably lacking the right terminology here...
I am trying to understand how a single process (R session) can spill over to all CPUs on our ...
0
votes
0answers
25 views
IIO Blocking Read
I've been working on an Industrial IO device driver for an analog to digital converter. In my application code, I am using a multi-threaded process by triggering the device on a loop to feed data ...
2
votes
0answers
62 views
Linux task state different in /proc/pid/stat compared to Ftrace output of sched events
I'm trying to trace the flow of a piece of code just to understand the points where the tasks go into D/S/R states (uninterruptible/interruptible/running). I do understand the meaning of each state, ...
0
votes
2answers
169 views
Lightweight processes behavior with an new PID
I've been experimenting with lightweight processes. Basically calling the clone function and assigning a new PID to the cloned LWP. This works fine it lets me identify all the children threads of ...
1
vote
0answers
34 views
Behavior of Linux kernel with respect to machine check exceptions (MCE) and threads
I was recently working with custom hardware that was not fully respecting the PCIe bus standards, and certain reads from the device would cause a bus fault and Linux machine check exception. When this ...
3
votes
1answer
105 views
Is mkdir -p safe to parallelize?
I'm writing Makefile rules like this:
out/%.x: ...
mkdir -p $(@D)/
...
I intend to parallelize this Makefile (-j), which will trigger many mkdir -p's to simultaneously.
Is this safe? Or do ...
3
votes
0answers
165 views
Atomic write and tell for files in O_APPEND mode on linux
I am dealing with a situation where I want to save state persistently for an application that has many processes and many threads within those processes. Normally this would require a heavy use of ...
0
votes
0answers
24 views
How does linux arbitrate access to devices
I would like to know the different methods linux uses to control access to a peripheral device. e.g. for example you have a spi temperature sensor. And multiple threads are trying to get a reading. ...
4
votes
1answer
2k views
'pthread_create: Resource temporarily unavailable' when running multiple docker instances
TLDR
When spinning up multiple docker containers in which I run npm ci, I start getting pthread_create: Resource temporarily unavailable errors (less than 5 docker containers can run fine). I deduce ...
0
votes
0answers
65 views
How can I multithread a for loop to run 1000 times at once?
I have a for loop that is curling the contents of a file, the file contains 2 million web addresses.
I woud like to create a for loop that instead of curling the address 1 at a time, curls 1000 at ...
0
votes
1answer
174 views
Is it possible to manage total threads for pipes of multi-threaded commands?
I am running several tools that have multi-threading capabilities. For example:
tool1 --threads=4 | tool2 --threads=4 | tool3 --threads=4
In an ideal case, I assume this should utilize 12 (4*3) ...
1
vote
1answer
74 views
Distribution of threads among CPUs?
lscpu gives:
Thread(s) per core: 2
Core(s) per socket: 32
When running an intensive 32-threads process, why does htop show almost 100% CPU activity on #1-32, but very little activity on #33-64? ...
0
votes
1answer
407 views
Do threads interrupt each other in Linux? [closed]
I'm learning about critical section of multithreading. I have a general statement:
In a single CPU system, disable interrupt is a solution of race condition.
But I also learn from another site ...
1
vote
0answers
267 views
Resize window in multi-thread ncurses program
The overall question is simple:
How to gracefully resize a window in a multi-thread ncurses program?
Details here. I have read a few relevant questions (1, 2, 3). From my understanding, there are ...
2
votes
0answers
261 views
Does `perf trace -p <PID>` trace all threads?
strace -f -p <PID> traces system calls in all threads of a multi-threaded process. (This also follows forks, there is an option -b execve that tries to address that).
Does perf trace -p <...
10
votes
2answers
4k views
Why on modern Linux, the default stack size is so huge - 8MB (even 10 on some distributions)
For example, on OSX, it's even less than 512k.
Is there any recommended size, having in mind, that the app does not use recursion and does not allocate a lot of stack variables?
I know the question ...
8
votes
1answer
9k views
Understanding output of lscpu
You can see the output from lscpu command -
jack@042:~$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 56
On-...
1
vote
1answer
434 views
Daemon vs worker and when to use what?
(Note that, below, I am frequently using term "daemon or worker" just to indicate that I dont know which word to use as I dont know the exact difference between them in the context explained below.)
...
2
votes
1answer
95 views
Run process as if on a single-core machine to find a bug
I have a bug in my Linux app that is reproducable only on single-core CPUs.
To debug it, I want to start the process from the command line so that it is limited to 1 CPU even on my multi-processor ...
1
vote
2answers
289 views
Stop rsync scheduled task race condition (large directory, small time interval)
Platform information:
OpenBSD 6.2 amd64
$ rsync --version
rsync version 3.1.2 protocol version 31
I'm trying to sync a large directory (4TB) using the following daily.local file (for Linux admins, ...
1
vote
0answers
34 views
Reconciling %CPU with load average in top
We have a long-running process, which utilizes OpenMP to saturate all available CPUs with computations done in multiple threads (one for each core).
I'm looking at top(1) on a server, where the ...
1
vote
1answer
587 views
How to optimise Aria2 when downloading thousands of small files?
My starting point is this question. Now aria2 is downloading the files but it is not as fast as I expected it (I am running this on a Macbook Pro with an i7, gigabit connection and AC-wifi. I am most ...
1
vote
1answer
684 views
Why is my program slower, despite using more threads? [closed]
I am new to threading , and I wanted to test my newly acquired skills, with a simple task, create an image using multiple threads, the interesting part is that , on a single thread , the program runs ...
2
votes
2answers
475 views
Can I run Linux on specific CPU cores?
I'm making a brainwave detection device.
Will be using an Octa-Core ARM processor for the software processing.
For the management of CPU cores, can I code Linux to strictly run on a specific set of ...
2
votes
3answers
6k views
How to bash multithread?
#!/bin/bash
while IFS="," read ip port; do
ruby test.rb "http://$ip:$port/"&
ruby test.rb "https://$ip:$port/";
done <test1.txt
How would i do this multithreading?
if i do more lines ...
0
votes
1answer
287 views
Multi-threading with for loop [duplicate]
This is part of a bash script code I wish to use multi-threading
for DATABASE in $DATABASES ; do
echo Converting $DATABASE
# Check if the table is MyISAM (we don't want to convert InnoDB ...
2
votes
0answers
514 views
Shielding CPU from scheduler - How to move bound kernel threads from cores?
Suppose I have 4 physical cores (indexed 0-3) and I'm dedicating cores 1-3 to a shield for user applications and trying to steer all system threads onto core 0. I'm following the answers here and ...
0
votes
1answer
31 views
Threads and Their Execution [closed]
Sometimes threads fell that they run long time, and they give their turns to another threads by means of thread scheduling procedure.
1) I wonder what happens if a thread does not want to stop ...
1
vote
1answer
101 views
Run C++ program across computer on network
I have a c++ program that is multithreaded. I believe the throughput would increase if I could run it in the other computers connected by a switch.
All of then are using the same OS(Ubuntu).
Is there ...
4
votes
0answers
395 views
How to mesure the latency of thread switching
I'm learning how to test the performance of a unix-like system.
I've found a tool to measure the context switching time: http://www.bitmover.com/lmbench/lat_ctx.8.html
Now I want to know if it is ...
1
vote
2answers
622 views
How to comprehend Cpus_allowed and thread id number?
Environment: OS --debian + python3.
All the output info below ommit unimportant.
Get my computer's cpu info with cat /proc/cpuinfo :
cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu ...
2
votes
1answer
746 views
How to split a file to multiple files with multiple threads?
So I have a 100GB text files And I want to split it into 10000 files.
I used to do such tasks with something like:
split -l <number of lines> -d --additional-suffix=.txt bigfile ...
-5
votes
1answer
53 views
I have to convert a simple program to multiple threads. So that my execution time may reduce [closed]
My task is to make a generic code (using shell prog. Language) which prompts the file name from user then execute the program and break the program in multiple threads (in number of threads a/c to the ...
1
vote
1answer
65 views
When running multicore jobs, will I lose performance if system load averages above the number of cores?
I do a lot of statistical analysis with R, and heavily utilize large multicore instances on AWS. Mostly for hyperparameter searches, cross-validation, and bootstrapping.
Say I have an instance ...
5
votes
1answer
410 views
How to delay bash script until there's enough idle cores to run it?
I'm a student who wants to benchmark a NGS pipeline, monitoring performance according to how many cores it has allocated to it and the size of the input file. For this reason, I wrote a bash script to ...
3
votes
2answers
2k views
Check CPU/thread usage for a node in the Slurm job manager
I am working on a cluster machine that uses the Slurm job manager. I just started a multithreaded code and I would like to check the core and thread usage for a given node ID. For example,
scoreusage ...
3
votes
3answers
10k views
run command on multiple threads
I am running a command (pngquant to be precise: https://github.com/pornel/pngquant) in a terminal window. I noticed, that if I open 4 terminal windows, and run pngquant command in each of them, I get ...
5
votes
2answers
7k views
Bash script multithreading in curl commands
here is a shell script which takes domain and its parameters to find status code . this runs way faster due to threading but misses lot of requests.
while IFS= read -r url <&3; do
while ...
2
votes
0answers
54 views
Multithreading repartition in *nix system
How the reparation of threads per CPU is handle on *nix system?
For example, one process uses 37 threads, machine has 4 cores. Usually all threads are placed on 2 cores. Sometimes at 3 cores. And ...
4
votes
1answer
5k views
What is a limit for number of threads?
I was wondering how many processes can I create on my machine (x64 with 8Gb of RAM and running Ubuntu). So I made simple master process which was continiously creating child processes, and that child ...
4
votes
0answers
212 views
kill -STOP shuts down [closed]
On a 12-core Linux machine, as a user (without root privilege) I start a job on a single process. When I hit Ctrl-Z the process is suspended and kill -CONT resumes it. This works fine.
But when the ...
0
votes
2answers
159 views
Parallel tasks and killing process
I have 3 tasks in a script which have to run as follow.
#1 and 2 need to run in parallel
task1 &
task2
#when task 1 is complete, task 3 is launched
task3
My problem is that task 2 being a ...
1
vote
0answers
87 views
Can i use kernel_thread in kernel when its version is over 3.10?
In my module code, i want to create my thread which doesn't fork from kthreadd, because kthreadd will clone_fs from init, then the fs of init was chroot, so it's not good for me, i just want to create ...
