Questions tagged [multithreading]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
1answer
21 views

Which thread receives SIGURG?

I recently wrote a "study note" about Unix, and I made following proposition about multi-threaded processes: it will be almost impossible for the kernel to identify the thread that should ...
2
votes
3answers
116 views

Boosting the grep search using GNU parallel

I am using the following grep script to output all the unmatched patterns: grep -oFf patterns.txt large_strings.txt | grep -vFf - patterns.txt > unmatched_patterns.txt patterns file contains the ...
2
votes
0answers
34 views

JVM only uses 4 cores

I have a processor-intensive Java program and I want it to use all 8 available cores on the machine. I've run taskset and it reports the mask to be: pid 5897's current affinity mask: ff But in top (...
0
votes
0answers
27 views

Grand-Father, Father, Child Script

OS : Red Hat Enterprise Linux Server V7.4 I need to run the following scripts having grand-father, father, child relationship Problem Statement :: I wish to run the scripts in the following way : Run ...
0
votes
1answer
49 views

How to switch from one process to another process and kill the first process

I have the following scenario, I have two programs running one in the background and one in front. The back program is doing some stuff for the front program. once the back program has done the ...
0
votes
0answers
20 views

What is needed and how do I check if my NFS connection uses Receive Side Scaling (RSS)?

RSS allows load balancing network transfers across multiple cpu cores: a NIC can send different packets to different queues to distribute processing among CPUs But which version of NFS is needed to ...
0
votes
1answer
28 views

Bind 9 process of named is using one core cpu only

i Have a DNS server running on Debian os , the named process is taking 394% of cpu. i noticed it is using one core of the 1 cores provided to the whole VM . top - 17:29:04 up 105 days, 1:33, 1 user, ...
0
votes
1answer
18 views

Schedule jobs from a queue onto multiple threads

I have a function that has to process all files in a set of directories (anything between 5-300 files). The number of parallel threads to be used is user-specified (usually 4). The idea is to start ...
2
votes
1answer
23 views

All processes running on the same core

I found all processes on my machine to only run on a single core and their core affinity set to 0. Here is a small python script which reproduces this for me: import multiprocessing import numpy as np ...
0
votes
0answers
64 views

AMD 3970x Threadripper - box *seems* to slow down after some heavy work

I'm not sure yet if this is my imagination yet (Nor do I know the most straightforward way to figure it out), but I have an AMD 3970x 32 core 64-thread "ThreadRipper" CPU (MSI Creator TRX4 ...
2
votes
3answers
2k views

How to use multi-threading for creating and extracting tar.xz

I use tar -cJvf resultfile.tar.xz files_to_compress to create tar.xz and tar -xzvf resultfile.tar.xz to extract the archive in current directory. How to use multi threading in both cases? I don't ...
-1
votes
1answer
56 views

How to find out on which core a thread is running on?

Let's say we have a CPU-intensive application called multi-threaded-application.out that is running on top of Ubuntu with a PID of 10000. It has 4 threads with tid 10001, 10002, 10003, and 10004. I ...
2
votes
2answers
82 views

What are these threads named disk$0, disk$1, etc.?

I am using Ubuntu 20.04 LTS. The kernel version is 5.4.0-42. Here is an example program: // mre.c // Compile with: cc -o mre mre.c -lSDL2 #include <stdio.h> #include <SDL2/SDL.h> int main(...
0
votes
1answer
49 views

Does a process filling all logical cores have a negative impact on performance? [closed]

Because of hyper-threading, my CPU has 2 logical processors per core. If I understand the premise of hyper-threading correctly, it allows each core to have a separate cache and instruction pointer for ...
0
votes
2answers
119 views

massive resource consumption during mkfs.ntfs on a zvol, why (and how can I limit this)?

On my machine mkfs.ntfs is slow and results in massive use of resources, preventing me from using the machine for anything else. According to top it (or rather directly related zvol processes) is ...
19
votes
4answers
2k views

Are basic POSIX utilities parallelized?

In a common Linux distribution, do utilities like rm, mv, ls, grep, wc, etc. run in parallel on their arguments? In other words, if I grep a huge file on a 32-threaded CPU, will it go faster than on ...
0
votes
0answers
154 views

IPSec multithreading via pcrypt worse than single thread

According to this document: https://www.strongswan.org/docs/Steffen_Klassert_Parallelizing_IPsec.pdf parallelizing IPSec using pcrypt kernel module will make the connection much more efficient (graphs ...
1
vote
1answer
106 views

Threads to process a million files [closed]

How many threads should be used to process a million files? How yould you justify your answer? This is a question from an OS exam from last year and I'm courious how you guys think. I think that 10....
1
vote
1answer
157 views

In Solaris box how do we check if hyper-threading/multi-threading is enabled or not using command?

In Solaris, how we can find the status or check hyper-threading or multi-threading is enabled or not using commands?
0
votes
0answers
182 views

macOSx / what is the kernel variable os_refcnt counting and what might be causing it to overflow?

Trying to figure out why running some software that I am developing causes my Mac to do a panic restart every 16 h or so always with an "os_refcnt" kernel variable overflow when running tcsh scripts. ...
0
votes
0answers
14 views

How to Pre-empt User Level Threads?

I am learning Operating System concepts. So I am working on developing a library to support User Level Thread Creation and Scheduling. So far I can create thread( function pointer) and schedule ...
23
votes
1answer
2k views

How to use “parallel” to speed up “sort” for big files fitting in RAM?

I have a 100 M line file that fits in RAM on a GNU/Linux system. This is rather slow: sort bigfile > bigfile.sorted and does not use all 48 cores on my machine. How do I sort that file fast?
0
votes
1answer
42 views

Thread safe next number (or round-robin) service in linux

I have an Ubuntu machine and a scenario in which I need to generate the next number (starting from X) between 0 and 20 in a round-robin way. I need a way to get the next number which will be thread ...
1
vote
0answers
108 views

Using libhugetlbfs to transparently back up glibc malloc calls in a multi-threaded application

I'm trying to back up the memory allocations of a multi-threaded application with 1GiB hugepages using libhugetlbfs. However, only the main thread allocations are being assigned hugepages. If I ...
0
votes
1answer
928 views

What types of threads does Java/JVM use from Linux OS's perspective?

A developer friend of mine recently asked the question: On a Linux system when a Java application runs which has threads, how do these threads appear to the underlying Linux OS? So what are Java ...
2
votes
1answer
3k views

Multithreaded xz, with gzip, pv, and pipes - is this the most efficient I can get?

I'm excited to learn that xz now supports multithreading: xz --threads=0 But now I want to utilise this as much as possible. For example, to recompress gzips as xz: gzip -d -k -c myfile.gz | pv | ...
3
votes
1answer
83 views

Why is this code exiting with return code 16?

I am trying my hands on clone() system call to create a Thread. However program is terminating itself as it return from t2_thread() function. Why is this behaviour? What am I missing? #define ...
1
vote
1answer
249 views

How to correctly create and stop kthreads?

I wrote the following kernel module: #include <linux/module.h> #include <linux/kernel.h> #include <linux/delay.h> #include <linux/kthread.h> #include <...
0
votes
1answer
42 views

How to limit the amount of spillover to other CPUs from a single process

Please excuse the complete ignorance of my question - I'm probably lacking the right terminology here... I am trying to understand how a single process (R session) can spill over to all CPUs on our ...
2
votes
0answers
94 views

Linux task state different in /proc/pid/stat compared to Ftrace output of sched events

I'm trying to trace the flow of a piece of code just to understand the points where the tasks go into D/S/R states (uninterruptible/interruptible/running). I do understand the meaning of each state, ...
0
votes
2answers
252 views

Lightweight processes behavior with an new PID

I've been experimenting with lightweight processes. Basically calling the clone function and assigning a new PID to the cloned LWP. This works fine it lets me identify all the children threads of ...
1
vote
0answers
45 views

Behavior of Linux kernel with respect to machine check exceptions (MCE) and threads

I was recently working with custom hardware that was not fully respecting the PCIe bus standards, and certain reads from the device would cause a bus fault and Linux machine check exception. When this ...
3
votes
1answer
185 views

Is mkdir -p safe to parallelize?

I'm writing Makefile rules like this: out/%.x: ... mkdir -p $(@D)/ ... I intend to parallelize this Makefile (-j), which will trigger many mkdir -p's to simultaneously. Is this safe? Or do ...
4
votes
0answers
333 views

Atomic write and tell for files in O_APPEND mode on linux

I am dealing with a situation where I want to save state persistently for an application that has many processes and many threads within those processes. Normally this would require a heavy use of ...
4
votes
1answer
4k views

'pthread_create: Resource temporarily unavailable' when running multiple docker instances

TLDR When spinning up multiple docker containers in which I run npm ci, I start getting pthread_create: Resource temporarily unavailable errors (less than 5 docker containers can run fine). I deduce ...
0
votes
1answer
289 views

Is it possible to manage total threads for pipes of multi-threaded commands?

I am running several tools that have multi-threading capabilities. For example: tool1 --threads=4 | tool2 --threads=4 | tool3 --threads=4 In an ideal case, I assume this should utilize 12 (4*3) ...
1
vote
1answer
127 views

Distribution of threads among CPUs?

lscpu gives: Thread(s) per core: 2 Core(s) per socket: 32 When running an intensive 32-threads process, why does htop show almost 100% CPU activity on #1-32, but very little activity on #33-64? ...
0
votes
1answer
605 views

Do threads interrupt each other in Linux? [closed]

I'm learning about critical section of multithreading. I have a general statement: In a single CPU system, disable interrupt is a solution of race condition. But I also learn from another site ...
1
vote
0answers
395 views

Resize window in multi-thread ncurses program

The overall question is simple: How to gracefully resize a window in a multi-thread ncurses program? Details here. I have read a few relevant questions (1, 2, 3). From my understanding, there are ...
3
votes
0answers
417 views

Does `perf trace -p <PID>` trace all threads?

strace -f -p <PID> traces system calls in all threads of a multi-threaded process. (This also follows forks, there is an option -b execve that tries to address that). Does perf trace -p <...
11
votes
2answers
7k views

Why on modern Linux, the default stack size is so huge - 8MB (even 10 on some distributions)

For example, on OSX, it's even less than 512k. Is there any recommended size, having in mind, that the app does not use recursion and does not allocate a lot of stack variables? I know the question ...
12
votes
1answer
16k views

Understanding output of lscpu

You can see the output from lscpu command - jack@042:~$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 56 On-...
1
vote
1answer
655 views

Daemon vs worker and when to use what?

(Note that, below, I am frequently using term "daemon or worker" just to indicate that I dont know which word to use as I dont know the exact difference between them in the context explained below.) ...
2
votes
1answer
168 views

Run process as if on a single-core machine to find a bug

I have a bug in my Linux app that is reproducable only on single-core CPUs. To debug it, I want to start the process from the command line so that it is limited to 1 CPU even on my multi-processor ...
2
votes
2answers
486 views

Stop rsync scheduled task race condition (large directory, small time interval)

Platform information: OpenBSD 6.2 amd64 $ rsync --version rsync version 3.1.2 protocol version 31 I'm trying to sync a large directory (4TB) using the following daily.local file (for Linux admins, ...
1
vote
0answers
38 views

Reconciling %CPU with load average in top

We have a long-running process, which utilizes OpenMP to saturate all available CPUs with computations done in multiple threads (one for each core). I'm looking at top(1) on a server, where the ...
1
vote
1answer
781 views

How to optimise Aria2 when downloading thousands of small files?

My starting point is this question. Now aria2 is downloading the files but it is not as fast as I expected it (I am running this on a Macbook Pro with an i7, gigabit connection and AC-wifi. I am most ...
1
vote
1answer
1k views

Why is my program slower, despite using more threads? [closed]

I am new to threading , and I wanted to test my newly acquired skills, with a simple task, create an image using multiple threads, the interesting part is that , on a single thread , the program runs ...
2
votes
2answers
775 views

Can I run Linux on specific CPU cores?

I'm making a brainwave detection device. Will be using an Octa-Core ARM processor for the software processing. For the management of CPU cores, can I code Linux to strictly run on a specific set of ...
4
votes
3answers
9k views

How to bash multithread?

#!/bin/bash while IFS="," read ip port; do ruby test.rb "http://$ip:$port/"& ruby test.rb "https://$ip:$port/"; done <test1.txt How would i do this multithreading? if i do more lines ...