The tag has no wiki summary.

learn more… | top users | synonyms

72
votes
7answers
17k views

Why use swap when there is more than enough RAM

Using swap space instead of RAM can drastically slow down a pc. So why, when I have more than enough RAM available, does my Linux system (Arch) use the swap? Checkout my conky output below: ...
37
votes
5answers
18k views

Can I configure my Linux system for more aggressive file system caching?

I am neither concerned about RAM usage (as I've got enough) nor about losing data in case of an accidental shut-down (as my power is backed, the system is reliable and the data are not critical). But ...
34
votes
1answer
1k views

Diagram of Linux kernel vs. performance tools?

Is there a diagram that shows how the various performance tools such as ip, netstat, perf, top, ps, etc. interact with the various subsystems within the Linux kernel?
25
votes
4answers
7k views

Is there a way to determine the optimal value for the bs parameter to dd?

On occasion I've seen comments online along the lines of "make sure you set 'bs=' because the default value will take too long," and my own extremely-unscientific experiences of, "well that seemed to ...
25
votes
12answers
2k views

How can I tweak my Linux desktop to be more responsive?

I have the experience that Linux works fine until the physical memory is exhausted. As soon as swap space is used the performance is severely degraded and the GUI becomes unresponsive. This problem ...
24
votes
1answer
1k views

Why does “ls *” take so much longer than “ls”?

I have a couple of files in a directory: $ ls | wc -l 9376 Can anybody explain why there is such a huge time difference in using ls * and ls? $ time ls > /dev/null real 0m0.118s user ...
23
votes
9answers
864 views

for vs find in Bash

When looping through files there are two ways: 1. use a for-loop for f in *; do echo "$f" done 2. use find find * -prune | while read f; do echo "$f" done Assuming these two loops will ...
23
votes
5answers
2k views

Why is iterating over a file twice faster than reading it into memory and computing twice?

I'm comparing the following tail -n 1000000 stdout.log | grep '"success": true' | wc -l tail -n 1000000 stdout.log | grep '"success": false' | wc -l with the following log=$(tail -n 1000000 ...
22
votes
3answers
1k views

How to limit resource usage to save CPU+RAM for a certain process?

I have a dev server on which sshd sometimes stops working because the machine runs out of RAM. Yes we are running low on memory and upgrading is not a feasible choice right now. What I want to do is ...
21
votes
2answers
718 views

Should I care about unnecessary cats?

A lot of command-line utilities can take their input either from a pipe or as a filename argument. For long shell scripts, I find starting the chain off with a cat makes it more readable, especially ...
20
votes
11answers
6k views

the lightest way to have a GUI in Linux?

I have been using Gnome and KDE for a while, and I am now running Compiz standalone... Regarding system resource, I think the most heavy ones are feature-rich desktop environments (Gnome, KDE), then ...
20
votes
6answers
5k views

Force directory to always be in cache

I've been testing out different methods to improve the time it takes to compile my entire c++ project. Currently it takes ~5 minutes. I experimented with distcc, ccache, and others. Recently, I ...
19
votes
3answers
2k views

grep: memory exhausted

I was doing a very simple search: grep -R Milledgeville ~/Documents And after some time this error appeared: grep: memory exhausted How can I avoid this? I have 10GB of RAM on my system and few ...
19
votes
4answers
11k views

How to speed up X over SSH on a slow network connection?

Are there any specific recommendations on speeding up X applications over ssh on a slow network connection? In this specific case, I am accessing a server located in west coast from a laptop in east ...
15
votes
3answers
11k views

What is the most high-performance Linux filesystem for storing a lot of small files (HDD, not SSD)?

I have a directory tree that contains many small files, and a small number of larger files. The average size of a file is about 1 kilobyte. There are 210158 files and directories in the tree (this ...
14
votes
2answers
2k views

Measure pipe throughput in the shell

There is a shell command that allows to measure how fast the data goes through it, so you can measure the speed of output of commands in a pipe. So instead of: $ somecommand | anothercommand you ...
14
votes
5answers
6k views

Is there a parallel file archiver (like tar)?

Is there something out there for parallel archiving of files? Tar is great, but I don't use tape archives, and it's more important to me that the archiving happens quickly (with compression like ...
13
votes
2answers
4k views

Why does high disk I/O reduce system responsiveness/performance?

I never quite understood why high disk I/O slowed the system so much. It's strange to me because I would expect the slow-down to affect only those processes dependent on the hard/optical drive data, ...
12
votes
1answer
929 views

Why is my TCP throughput much greater than UDP throughput?

I haven't done anything unusual to my hardware or kernel configurations (all default settings, fresh OS install, Linux kernel 3.11 TCP/IP stack) and I'm averaging about 3.83 million messages per ...
11
votes
1answer
1k views

Is disabling barriers for ext4 safe on a laptop with battery?

The manual page says about the barrier option for ext4: Write barriers enforce proper on-disk ordering of journal commits, making volatile disk write caches safe to use, at some performance ...
11
votes
1answer
521 views

How do I go about tracking down IO usage in Linux and other *nix OS'?

Occasionally, something on my system starts hammering the disk. I would like to log data that would let me know about I/O usage per process, then I could go back to it later and see what keeps doing ...
10
votes
4answers
6k views

Use of O_DIRECT on Linux

If this question is too programmer oriented, let me know. I wonder if there are people familiar with the O_DIRECT flag for the open() system call on Linux 2.6? Linus disparages its use, however high ...
9
votes
2answers
617 views

Why is wc so slow?

Why is the wc utility so slow? When I run it on a large file, it takes about 20 times longer than md5sum: MyDesktop:/tmp$ dd if=/dev/zero bs=1024k count=1024 of=/tmp/bigfile 1024+0 records in 1024+0 ...
9
votes
1answer
2k views

How do you empty the buffers and cache on a Linux system?

Prior to doing some benchmarking work how would one free up the memory (RAM) that the Linux Kernel is consuming for its buffers and cache?
9
votes
1answer
1k views

What's the progress regarding improving system performance/responsiveness during high disk I/O?

Whenever there is high disk I/O, the system tends to be much slower and less responsive than usual. What's the progress on Linux kernel regarding this? Is this problem actively being worked on?
9
votes
1answer
5k views

How can I configure cgroups to fairly share resources between users?

There used to be a kernel config option called sched_user or similar under cgroups. This allowed (to my knowledge) all users to fairly share system resources. In 2.6.35 it is not available. Is there a ...
9
votes
2answers
911 views

echo vs <<<, or Useless Use of echo in Bash Award?

By now the Useless Use of cat Award is very well known, and there's also a mention of a Useless Use of echo (not relevant for this question). I'm wondering if there should be a "Useless Use of echo in ...
9
votes
1answer
2k views

Defragmentation of LVM2 logical volumes

Question: Is there a tool supporting de-fragmentation of LVM2 logical volumes ? (Making their extents into consistent sequences) Preferably by defining desired order of extents (something like ...
8
votes
2answers
1k views

Heavy write activity on SSD nukes system performance

I've noticed that when I do heavy write applications, the whole system slows down. To test this further I ran this to do a (relatively) low-CPU, high disk activity: john -incremental > file_on_SSD ...
8
votes
4answers
8k views

Why is kworker consuming so many resources on Linux 3.0.0-12-server?

Last Friday I upgraded my Ubuntu server to 11.10, which now runs with a 3.0.0-12-server kernel. Since then the overall performance has dropped dramatically. Before the upgrade the system load was ...
8
votes
7answers
2k views

Processor usage increases with 4GB RAM installed

I have an Intel Atom D2700 (Synology NAS DS412+) with 4GB RAM running kernel 3.2.30 x86_64. This unit has a single DIMM slot. One thing I, and other's have found, is that when adding a 4GB DIMM ...
7
votes
6answers
3k views

Sync writes very slow. Ubuntu 10.10, 32 bits, ext4

I am running ActiveMQ on my Macbook Pro that runs Ubuntu 10.10, 32 bits with an ext4 partition. Linux iker-laptop 2.6.35-23-generic-pae #40-Ubuntu SMP Wed Nov 17 22:32:51 UTC 2010 i686 GNU/Linux If ...
7
votes
2answers
810 views

Has anyone got any performance numbers comparing IIS and .NET to Cherokee and Mono?

I am setting up a development server and want to set it up to serve ASP.NET pages using Mono. I am planning on using Cherokee and Mono (http://www.cherokee-project.com/doc/cookbook_mono.html) and ...
7
votes
2answers
48 views

How can other processes affect measurements made with `time`?

If I time a process using the time command, I get output for 'real', 'user', and 'sys'. My understanding from this discussion is that 'real' is wall time, whereas 'user' and 'sys' are process time. ...
7
votes
1answer
1k views

How to make dpkg faster?

Package (un)installation on a Debian system is horribly slow, partly because it works with a whole bunch of (small) files. Short of getting a faster storage, how do I speed it up? I'm thinking maybe ...
7
votes
1answer
2k views

Throttling web crawlers

My website is being DoS'ed by Google webspiders. Google is welcome to index my site, but sometimes it is querying a tagcloud on my site faster than my webserver can produce the results, making my ...
7
votes
1answer
4k views

Using Linux cgroups to balance CPU performance

I have two dual-core Linux systems installed using Linux cgroups with relatively recent kernels; one is running Debian Squeeze, the other Ubuntu 11.04 Natty Narwhal. I've gotten CPU load balancing ...
7
votes
1answer
2k views

Why is ping so slow?

My setup: Server in the cellar + GBit switch. Long cable to the roof. GBit switch. I have three computers in the roof. One has only 100MBit ethernet, one laptop with GBit and a new computer. Ping ...
6
votes
7answers
463 views

Make disk/disk copy slower

Is there a method of slowing down the copy process on Linux? I have a big file, say 10GB, and I'd like to copy it to another directory, but I don't want to copy it with full speed. Let's say I'd like ...
6
votes
2answers
4k views

Diffing two big text files

I have two big files (6GB each). They are unsorted, with linefeeds (\n) as separators. How can I diff them? It should take under 24h.
6
votes
4answers
207 views

Quantify unix responsiveness

I am one of the n users of a shared unix machine. For reasons unknown, the machine is not "responsive" enough. For example, it is slow on interactive commands, it takes few noticeable moments for any ...
6
votes
2answers
482 views

Does the position of partition on disk affect speed?

If I put /var as first partition, then /home and /, will the partition for /var have better performance than if I put other partitions close to head of the disk? Will the disk sector position matter? ...
6
votes
2answers
704 views

I/O-overhead of dm-encrypted device?

What would be the Read-/Write-overhead when using dm-crypt (LUKS) as full disk-encryption (including root partition) on a Linux-Desktop (Ubuntu)? I plan to stack it like so: LUKS > LVM > ext4 The CPU ...
6
votes
1answer
83 views

Do processes in cgroups have to be set every boot? If so, how to persist them?

I was reading this answer to a question on setting limits for processes by using cgroups. The sample given sets limits for a process (in the example, sshd). In the answer, the PID of the process is ...
6
votes
3answers
592 views

zsync vs. jigdo

What's the difference between zsync and jigdo? Their core functionality seems the same, so I'm curious on things like performance, file size, and ease-of-use. Would also be interesting to know why one ...
6
votes
3answers
574 views

How to average time commands?

I am timeing some of my commands. Now to average the measures, I'd like to run and time my commands a specified number of times, and get the results with a calculated mean and standard deviation. The ...
6
votes
2answers
637 views

Poor random read SSD performance on Linux

I recently got an Intel 320-series SSD, and I am having difficulty achieving the advertised 38K IOPS for random 4K reads. Both with fio and my own hacked-together program, I am seeing around 6K IOPS. ...
6
votes
1answer
5k views

How to configure a filesystem+LVM for effective usage of SSD?

ext4 is one of the filesystems recommended for using on SSDs (with a number of built-in techniques and advices how to tune it for better performance and wear-levelling of SSDs). I want to install a ...
5
votes
3answers
594 views

How to speed up my build

I am doing a build on a Linux machine with ubuntu 10.04 on it. I want to know how can I really speed up my build. I have 4 CPUs and lots of RAM. I already reniced the process group to -20. Is there ...
5
votes
2answers
2k views

dd if=/dev/random is 'randomly' bottlenecked with BIG time lags, but I have no idea why

I get wildly different real times when I run the following command. dd if=/dev/random bs=1k count=1 It doesn' happen for if=/dev/null, nor does it happen for if=/dev/urandom I've run it 500 ...