46

Actually, your virtual stack size is 8388608 bytes (8 MB). Of course, it's natural to conclude that this can't be right, because that's a ridiculously large amount of memory for every thread to consume for its stack when 99% of the time a couple of KB is probably all they need. The good news is that your thread only uses the amount of physical memory that ...


45

"Why does this happen?" is kind of easy to answer. Imagine you have a corridor that you can fit four people down, side by side. You want to move all the rubbish at one end, to the other end. The most efficient number of people is 4. If you have 1-3 people then you're missing out on using some corridor space. If you have 5 or more people, then at least ...


33

This is a complicated question you're asking. Without knowing more about the nature of your threads it's difficult to say. Some things to consider when diagnosing system performance: Is the process/thread CPU bound (needs lots of CPU resources) Memory bound (needs lots of RAM resources) I/O bound (Network and/or hard drive resources) All of these three ...


30

Given that google killed chrome://memory in March 2016, I am now using smem: # detailed output, in kB apparently smem -t -P chrom # just the total PSS, with automatic unit: smem -t -k -c pss -P chrom | tail -n 1 to be more accurate replace chrom by full path e.g. /opt/google/chrome or /usr/lib64/chromium-browser this works the same for multiprocess firefox ...


26

As Celada mentioned, there would be no point to using multiple threads of execution since a copy operation doesn't really use the cpu. As ryekayo mentioned, you can run multiple instances of cp so that you end up with multiple concurrent IO streams, but even this is typically counter-productive. If you are copying files from one location to another on the ...


25

On a GNU system and if you have pv, you could do: cmd=' that command | to execute && as shell code' yes | pv -qL10 | xargs -n1 -P20 sh -c "$cmd" sh The -P20 is to execute at most 20 $cmd at the same time. -L10 limits the rate to 10 bytes per second, so 5 lines per second. If your $cmds become two slow and causes the 20 limit to be ...


21

int pthread_attr_setstacksize(pthread_attr_t *attr, size_t stacksize); The stacksize attribute shall define the minimum stack size (in bytes) allocated for the created threads stack. In your example, the stack size is set to 8388608 bytes which corresponds to 8MB, as returned by the command ulimit -s So that matches. From the pthread_create() ...


20

A common recommendation is n+1 threads, n being the number of CPU cores available. That way n threads can work the CPU while 1 thread is waiting for disk I/O. Having fewer threads would not fully utilize the CPU resource (at some point there will always be I/O to wait for), having more threads would cause threads fighting over the CPU resource. Threads come ...


16

Well, I believe you could use gnu parallel to accomplish your task. seq 70 | parallel -j70 cp filename You could see a detailed explanation on using gnu parallel from my other answer here. I just tested the above command in my system and I could see that 70 copies of files are being made.


15

As others have said, and as is mentioned in the link you provide in your question, having an 8MiB stack doesn’t hurt anything (apart from consuming address space — on a 64-bit system that won’t matter). Linux has used 8MiB stacks for a very long time; the change was introduced in version 1.3.7 of the kernel, in July 1995. Back then it was presented as ...


13

I wouldn't call it multithreading as such but you could simply launch 70 jobs in the background: for i in {1..70}; do wget http://www.betaservice.domain.host.com/web/hasChanged?ver=0 2>/dev/null & done That will result in 70 wget processes running at once. You can also do something more sophisticated like this little script: #!/usr/bin/env bash ...


12

Your issue is the max user processes limit. From the getrlimit(2) man page: RLIMIT_NPROC The maximum number of processes (or, more precisely on Linux, threads) that can be created for the real user ID of the calling process. Upon encountering this limit, fork(2) fails with the error EAGAIN. Same for pthread_create(3): EAGAIN Insufficient resources ...


12

No, vim is not multithreaded. Multiple cores won't help you here. First we have to agree on what a huge file is. I suppose you mean a file larger than the RAM size. Vim was not designed for large files. Furthermore, when not sufficient line ends are present, vim might not be able to open the file at all. Decide if you want to just read the file content or ...


12

(pacaur uses makepkg, see https://wiki.archlinux.org/index.php/Makepkg ) In /etc/makepkg.conf add MAKEFLAGS="-j$(expr $(nproc) \+ 1)" to run #cores + 1 compiling jobs concurrently. When using bash you can also add export MAKEFLAGS="-j$(expr $(nproc) \+ 1)" to your ~/.bashrc to make this default for all make commands, not only those for AUR packages.


10

They are actually showing the same information in different ways. This is what the -f and -L options to ps do (from man ps, emphasis mine): -f               Do full-format listing. This option can be combined with many other UNIX-style options to add additional columns. It also ...


10

GNU parallel is made for just this sort of thing. You can run your script many times at once, with different data from your input piped in for each one: cat input.txt | parallel --pipe your-script.sh By default it will spawn processes according to the number of processors on your system, but you can customise that with -j N. A particularly neat trick is ...


10

“CPU(s): 56” represents the number of logical cores, which equals “Thread(s) per core” × “Core(s) per socket” × “Socket(s)”. One socket is one physical CPU package (which occupies one socket on the motherboard); each socket hosts a number of physical cores, and each core can run one or more threads. In your case, you have two sockets, each containing a 14-...


9

You can do as @UlrichDangel suggested in the comments and replace the executable gzip with pigz. If you want something a little less invasive you can also create functions for gzip and gunzip and add them to your $HOME/.bashrc file. gzip() { pigz "$@" } export -f gzip gunzip() { unpigz "$@" } export -f gunzip Now when you run zgrep or zcat it will use ...


9

Try ab, you get a nice statistic too: ab -n 10000 -c 70 http://www.betaservice.domain.host.com/web/hasChanged?ver=0 This call will do 10000 requests with concurrency of 70 parallel queries.


8

As other have pointed out (slm answer, EightBitTony answer ) this is a complicated question and more so since you do not describe what you thred do and how they do it. But definitively throwing in more threads can make the things worse. In the field of parallel computing there is Amdahl's law that can be applicable (or cannot, not but you do not describe ...


8

This is 100% normal with respect to threading on any and all operating systems. The documentation for your thread library, any examples and tutorials you may find, etc. are likely to make a point of this as it is often confusing to people when they are learning the ropes of threading. Threads are by default (and by definition) not synchronized. This means ...


8

Probably not. All cron has to do is (to express it simplified) watch until it is time to run one job or the other, and if so, fork a process which runs that job and periodically check if the job is finished in order to clean it up. MT could be used for this waiting, but I think that would be overkill. With the wait()/waitpid() family functions, it is ...


8

The entry in POSIX on "Signal Generation and Delivery" in "Rationale: System Interfaces General Information" says Signals generated for a process are delivered to only one thread. Thus, if more than one thread is eligible to receive a signal, one has to be chosen. The choice of threads is left entirely up to the implementation both to allow the widest ...


7

I found three solutions: With GNU tar, using the awesome -I option: tar -I pigz -xvf /path/to/archive.tar.gz -C /where/to/unpack/it/ With a lot of Linux piping (for those who prefer a more geeky look): unpigz < /path/to/archive.tar.gz | tar -xvC /where/to/unpack/it/ More portable (to other tar implementations): unpigz < /path/to/archive.tar.gz | (...


7

You are experiencing the problem of appending to a file in parallel. The easy answer is: Don't. Here is how you can do it using GNU Parallel: doit() { url="$1" uri="$2" urlstatus=$(curl -o /dev/null --insecure --silent --head --write-out '%{http_code}' "${url}""${uri}" --max-time 5 ) && echo "$url $urlstatus $uri" } export -f doit ...


6

There are a number of implementations of cron for Linux. Whether any of them are multi-threaded or not is not really a question that I think is relevant unless you're trying to write your own implementation of the cron daemon.


6

In the context of a Unix or linux process, the phrase "the stack" can mean two things. First, "the stack" can mean the last-in, first-out records of the calling sequence of the flow of control. When a process executes, main() gets called first. main() might call printf(). Code generated by the compiler writes the address of the format string, and any other ...


6

There is no effective way to parallelize an append I/O operation; each line must be written in turn.


6

The culprit here should be the "CONTEXT SWITCHING". It is the process of saving the state of the current thread to start executing another thread. If a number of threads are given the same priority they need to be switched around till they finish execution. In your case, when there are 50 threads a lot of context switching takes place when compared to just ...


6

Let's say this.py contains the following: #!/usr/bin/python from datetime import datetime now = datetime.now() print now The following in Bash, would execute 10 processes of this.py simultaneously in 20,000 rounds. The next round starts after the 10 processes have completed. This will allow you to execute this.py 200,000 times, while only using 10 ...


Only top voted, non community-wiki answers of a minimum length are eligible