Questions tagged [multithreading]
The multithreading tag has no usage guidance.
130
questions
0
votes
1answer
21 views
Which thread receives SIGURG?
I recently wrote a "study note" about Unix, and I made following proposition about multi-threaded processes:
it will be almost impossible for the kernel to identify the thread that should ...
2
votes
3answers
116 views
Boosting the grep search using GNU parallel
I am using the following grep script to output all the unmatched patterns:
grep -oFf patterns.txt large_strings.txt | grep -vFf - patterns.txt > unmatched_patterns.txt
patterns file contains the ...
2
votes
0answers
34 views
JVM only uses 4 cores
I have a processor-intensive Java program and I want it to use all 8 available cores on the machine.
I've run taskset and it reports the mask to be:
pid 5897's current affinity mask: ff
But in top (...
0
votes
0answers
27 views
Grand-Father, Father, Child Script
OS : Red Hat Enterprise Linux Server V7.4
I need to run the following scripts having grand-father, father, child relationship
Problem Statement :: I wish to run the scripts in the following way :
Run ...
0
votes
1answer
49 views
How to switch from one process to another process and kill the first process
I have the following scenario, I have two programs running one in the background and one in front. The back program is doing some stuff for the front program. once the back program has done the ...
0
votes
0answers
20 views
What is needed and how do I check if my NFS connection uses Receive Side Scaling (RSS)?
RSS allows load balancing network transfers across multiple cpu cores:
a NIC can send different packets to different
queues to distribute processing among CPUs
But which version of NFS is needed to ...
0
votes
1answer
28 views
Bind 9 process of named is using one core cpu only
i Have a DNS server running on Debian os , the named process is taking 394% of cpu.
i noticed it is using one core of the 1 cores provided to the whole VM .
top - 17:29:04 up 105 days, 1:33, 1 user, ...
0
votes
1answer
18 views
Schedule jobs from a queue onto multiple threads
I have a function that has to process all files in a set of directories (anything between 5-300 files). The number of parallel threads to be used is user-specified (usually 4). The idea is to start ...
2
votes
1answer
23 views
All processes running on the same core
I found all processes on my machine to only run on a single core and their core affinity set to 0.
Here is a small python script which reproduces this for me:
import multiprocessing
import numpy as np
...
0
votes
0answers
64 views
AMD 3970x Threadripper - box *seems* to slow down after some heavy work
I'm not sure yet if this is my imagination yet (Nor do I know the most straightforward way to figure it out), but I have an AMD 3970x 32 core 64-thread "ThreadRipper" CPU (MSI Creator TRX4 ...
2
votes
3answers
2k views
How to use multi-threading for creating and extracting tar.xz
I use
tar -cJvf resultfile.tar.xz files_to_compress
to create tar.xz and
tar -xzvf resultfile.tar.xz
to extract the archive in current directory. How to use multi threading in both cases? I don't ...
-1
votes
1answer
56 views
How to find out on which core a thread is running on?
Let's say we have a CPU-intensive application called multi-threaded-application.out that is running on top of Ubuntu with a PID of 10000. It has 4 threads with tid 10001, 10002, 10003, and 10004. I ...
2
votes
2answers
82 views
What are these threads named disk$0, disk$1, etc.?
I am using Ubuntu 20.04 LTS. The kernel version is 5.4.0-42.
Here is an example program:
// mre.c
// Compile with: cc -o mre mre.c -lSDL2
#include <stdio.h>
#include <SDL2/SDL.h>
int main(...
0
votes
1answer
49 views
Does a process filling all logical cores have a negative impact on performance? [closed]
Because of hyper-threading, my CPU has 2 logical processors per core. If I understand the premise of hyper-threading correctly, it allows each core to have a separate cache and instruction pointer for ...
0
votes
2answers
119 views
massive resource consumption during mkfs.ntfs on a zvol, why (and how can I limit this)?
On my machine mkfs.ntfs is slow and results in massive use of resources, preventing me from using the machine for anything else. According to top it (or rather directly related zvol processes) is ...
19
votes
4answers
2k views
Are basic POSIX utilities parallelized?
In a common Linux distribution, do utilities like rm, mv, ls, grep, wc, etc. run in parallel on their arguments?
In other words, if I grep a huge file on a 32-threaded CPU, will it go faster than on ...
0
votes
0answers
154 views
IPSec multithreading via pcrypt worse than single thread
According to this document:
https://www.strongswan.org/docs/Steffen_Klassert_Parallelizing_IPsec.pdf parallelizing IPSec using pcrypt kernel module will make the connection much more efficient (graphs ...
1
vote
1answer
106 views
Threads to process a million files [closed]
How many threads should be used to process a million files? How yould you justify your answer? This is a question from an OS exam from last year and I'm courious how you guys think. I think that 10....
1
vote
1answer
157 views
In Solaris box how do we check if hyper-threading/multi-threading is enabled or not using command?
In Solaris, how we can find the status or check hyper-threading or multi-threading is enabled or not using commands?
0
votes
0answers
182 views
macOSx / what is the kernel variable os_refcnt counting and what might be causing it to overflow?
Trying to figure out why running some software that I am developing causes my Mac to do a panic restart every 16 h or so always with an "os_refcnt" kernel variable overflow when running tcsh scripts.
...
0
votes
0answers
14 views
How to Pre-empt User Level Threads?
I am learning Operating System concepts. So I am working on developing a library to support User Level Thread Creation and Scheduling. So far I can create thread( function pointer) and schedule ...
23
votes
1answer
2k views
How to use “parallel” to speed up “sort” for big files fitting in RAM?
I have a 100 M line file that fits in RAM on a GNU/Linux system.
This is rather slow:
sort bigfile > bigfile.sorted
and does not use all 48 cores on my machine.
How do I sort that file fast?
0
votes
1answer
42 views
Thread safe next number (or round-robin) service in linux
I have an Ubuntu machine and a scenario in which I need to generate the next number (starting from X) between 0 and 20 in a round-robin way.
I need a way to get the next number which will be thread ...
1
vote
0answers
108 views
Using libhugetlbfs to transparently back up glibc malloc calls in a multi-threaded application
I'm trying to back up the memory allocations of a multi-threaded application with 1GiB hugepages using libhugetlbfs. However, only the main thread allocations are being assigned hugepages. If I ...
0
votes
1answer
928 views
What types of threads does Java/JVM use from Linux OS's perspective?
A developer friend of mine recently asked the question: On a Linux system when a Java application runs which has threads, how do these threads appear to the underlying Linux OS?
So what are Java ...
2
votes
1answer
3k views
Multithreaded xz, with gzip, pv, and pipes - is this the most efficient I can get?
I'm excited to learn that xz now supports multithreading:
xz --threads=0
But now I want to utilise this as much as possible. For example, to recompress gzips as xz:
gzip -d -k -c myfile.gz | pv | ...
3
votes
1answer
83 views
Why is this code exiting with return code 16?
I am trying my hands on clone() system call to create a Thread. However program is terminating itself as it return from t2_thread() function. Why is this behaviour? What am I missing?
#define ...
1
vote
1answer
249 views
How to correctly create and stop kthreads?
I wrote the following kernel module:
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/delay.h>
#include <linux/kthread.h>
#include <...
0
votes
1answer
42 views
How to limit the amount of spillover to other CPUs from a single process
Please excuse the complete ignorance of my question - I'm probably lacking the right terminology here...
I am trying to understand how a single process (R session) can spill over to all CPUs on our ...
2
votes
0answers
94 views
Linux task state different in /proc/pid/stat compared to Ftrace output of sched events
I'm trying to trace the flow of a piece of code just to understand the points where the tasks go into D/S/R states (uninterruptible/interruptible/running). I do understand the meaning of each state, ...
0
votes
2answers
252 views
Lightweight processes behavior with an new PID
I've been experimenting with lightweight processes. Basically calling the clone function and assigning a new PID to the cloned LWP. This works fine it lets me identify all the children threads of ...
1
vote
0answers
45 views
Behavior of Linux kernel with respect to machine check exceptions (MCE) and threads
I was recently working with custom hardware that was not fully respecting the PCIe bus standards, and certain reads from the device would cause a bus fault and Linux machine check exception. When this ...
3
votes
1answer
185 views
Is mkdir -p safe to parallelize?
I'm writing Makefile rules like this:
out/%.x: ...
mkdir -p $(@D)/
...
I intend to parallelize this Makefile (-j), which will trigger many mkdir -p's to simultaneously.
Is this safe? Or do ...
4
votes
0answers
333 views
Atomic write and tell for files in O_APPEND mode on linux
I am dealing with a situation where I want to save state persistently for an application that has many processes and many threads within those processes. Normally this would require a heavy use of ...
4
votes
1answer
4k views
'pthread_create: Resource temporarily unavailable' when running multiple docker instances
TLDR
When spinning up multiple docker containers in which I run npm ci, I start getting pthread_create: Resource temporarily unavailable errors (less than 5 docker containers can run fine). I deduce ...
0
votes
1answer
289 views
Is it possible to manage total threads for pipes of multi-threaded commands?
I am running several tools that have multi-threading capabilities. For example:
tool1 --threads=4 | tool2 --threads=4 | tool3 --threads=4
In an ideal case, I assume this should utilize 12 (4*3) ...
1
vote
1answer
127 views
Distribution of threads among CPUs?
lscpu gives:
Thread(s) per core: 2
Core(s) per socket: 32
When running an intensive 32-threads process, why does htop show almost 100% CPU activity on #1-32, but very little activity on #33-64? ...
0
votes
1answer
605 views
Do threads interrupt each other in Linux? [closed]
I'm learning about critical section of multithreading. I have a general statement:
In a single CPU system, disable interrupt is a solution of race condition.
But I also learn from another site ...
1
vote
0answers
395 views
Resize window in multi-thread ncurses program
The overall question is simple:
How to gracefully resize a window in a multi-thread ncurses program?
Details here. I have read a few relevant questions (1, 2, 3). From my understanding, there are ...
3
votes
0answers
417 views
Does `perf trace -p <PID>` trace all threads?
strace -f -p <PID> traces system calls in all threads of a multi-threaded process. (This also follows forks, there is an option -b execve that tries to address that).
Does perf trace -p <...
11
votes
2answers
7k views
Why on modern Linux, the default stack size is so huge - 8MB (even 10 on some distributions)
For example, on OSX, it's even less than 512k.
Is there any recommended size, having in mind, that the app does not use recursion and does not allocate a lot of stack variables?
I know the question ...
12
votes
1answer
16k views
Understanding output of lscpu
You can see the output from lscpu command -
jack@042:~$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 56
On-...
1
vote
1answer
655 views
Daemon vs worker and when to use what?
(Note that, below, I am frequently using term "daemon or worker" just to indicate that I dont know which word to use as I dont know the exact difference between them in the context explained below.)
...
2
votes
1answer
168 views
Run process as if on a single-core machine to find a bug
I have a bug in my Linux app that is reproducable only on single-core CPUs.
To debug it, I want to start the process from the command line so that it is limited to 1 CPU even on my multi-processor ...
2
votes
2answers
486 views
Stop rsync scheduled task race condition (large directory, small time interval)
Platform information:
OpenBSD 6.2 amd64
$ rsync --version
rsync version 3.1.2 protocol version 31
I'm trying to sync a large directory (4TB) using the following daily.local file (for Linux admins, ...
1
vote
0answers
38 views
Reconciling %CPU with load average in top
We have a long-running process, which utilizes OpenMP to saturate all available CPUs with computations done in multiple threads (one for each core).
I'm looking at top(1) on a server, where the ...
1
vote
1answer
781 views
How to optimise Aria2 when downloading thousands of small files?
My starting point is this question. Now aria2 is downloading the files but it is not as fast as I expected it (I am running this on a Macbook Pro with an i7, gigabit connection and AC-wifi. I am most ...
1
vote
1answer
1k views
Why is my program slower, despite using more threads? [closed]
I am new to threading , and I wanted to test my newly acquired skills, with a simple task, create an image using multiple threads, the interesting part is that , on a single thread , the program runs ...
2
votes
2answers
775 views
Can I run Linux on specific CPU cores?
I'm making a brainwave detection device.
Will be using an Octa-Core ARM processor for the software processing.
For the management of CPU cores, can I code Linux to strictly run on a specific set of ...
4
votes
3answers
9k views
How to bash multithread?
#!/bin/bash
while IFS="," read ip port; do
ruby test.rb "http://$ip:$port/"&
ruby test.rb "https://$ip:$port/";
done <test1.txt
How would i do this multithreading?
if i do more lines ...
