The tag has no wiki summary.

learn more… | top users | synonyms

29
votes
4answers
12k views

Is there a way to determine the optimal value for the bs parameter to dd?

On occasion I've seen comments online along the lines of "make sure you set 'bs=' because the default value will take too long," and my own extremely-unscientific experiences of, "well that seemed to ...
38
votes
2answers
45k views

How do you empty the buffers and cache on a Linux system?

Prior to doing some benchmarking work how would one free up the memory (RAM) that the Linux Kernel is consuming for its buffers and cache?
80
votes
7answers
23k views

Why use swap when there is more than enough free space in RAM?

Using swap space instead of RAM can drastically slow down a PC. So why, when I have more than enough RAM available, does my Linux system (Arch) use the swap? Checkout my conky output below: Also, ...
17
votes
2answers
3k views

Measure pipe throughput in the shell

There is a shell command that allows to measure how fast the data goes through it, so you can measure the speed of output of commands in a pipe. So instead of: $ somecommand | anothercommand you ...
22
votes
4answers
2k views

How to limit resource usage to save CPU+RAM for a certain process?

I have a dev server on which sshd sometimes stops working because the machine runs out of RAM. Yes we are running low on memory and upgrading is not a feasible choice right now. What I want to do is ...
4
votes
2answers
572 views

Why load is high despite the fact that neither CPU or disk is overused

I'm getting the following output from top: Cpu(s): 43.8%us, 32.5%sy, 4.8%ni, 2.0%id, 15.6%wa, 0.2%hi, 1.2%si, 0.0%st Mem: 16331504k total, 15759412k used, 572092k free, 4575980k buffers ...
21
votes
11answers
8k views

the lightest way to have a GUI in Linux? [closed]

I have been using Gnome and KDE for a while, and I am now running Compiz standalone... Regarding system resource, I think the most heavy ones are feature-rich desktop environments (Gnome, KDE), then ...
47
votes
5answers
25k views

Can I configure my Linux system for more aggressive file system caching?

I am neither concerned about RAM usage (as I've got enough) nor about losing data in case of an accidental shut-down (as my power is backed, the system is reliable and the data are not critical). But ...
21
votes
4answers
14k views

How to speed up X over SSH on a slow network connection?

Are there any specific recommendations on speeding up X applications over ssh on a slow network connection? In this specific case, I am accessing a server located in west coast from a laptop in east ...
26
votes
3answers
1k views

Should I care about unnecessary cats?

A lot of command-line utilities can take their input either from a pipe or as a filename argument. For long shell scripts, I find starting the chain off with a cat makes it more readable, especially ...
9
votes
1answer
1k views

What's the progress regarding improving system performance/responsiveness during high disk I/O?

Whenever there is high disk I/O, the system tends to be much slower and less responsive than usual. What's the progress on Linux kernel regarding this? Is this problem actively being worked on?
5
votes
3answers
1k views

Fast way to build a test file with every second listed in YYYY-mm-dd HH:MM:SS format

I want to create a large test file with lines containg dates listed by the second, but my method is taking inordinately long... (or at least, that's how it feels :) ... 43 minutes to create only ...
4
votes
4answers
295 views

Grep of many keywords over many files: speeding it up

I am currently facing a "performance problem" while using grep. I am trying to locate the occurrences of many (10,000+) keywords in many (think Linux kernel repository size) files. The objective is to ...
44
votes
1answer
2k views

Diagram of Linux kernel vs. performance tools?

Is there a diagram that shows how the various performance tools such as ip, netstat, perf, top, ps, etc. interact with the various subsystems within the Linux kernel?
18
votes
4answers
2k views

Is there a fastest way to shutdown the system?

Or is shutdown -h now the fastest it can get? I look for some syscall or similar that will allow to skip lots of the stuff done prior to shutdown (in particular to care about the running proccesses). ...
25
votes
12answers
3k views

How can I tweak my Linux desktop to be more responsive?

I have the experience that Linux works fine until the physical memory is exhausted. As soon as swap space is used the performance is severely degraded and the GUI becomes unresponsive. This problem ...
24
votes
5answers
3k views

Why is iterating over a file twice faster than reading it into memory and computing twice?

I'm comparing the following tail -n 1000000 stdout.log | grep -c '"success": true' tail -n 1000000 stdout.log | grep -c '"success": false' with the following log=$(tail -n 1000000 stdout.log) echo ...
7
votes
2answers
572 views

Why is `tac file | grep foo' (piping) faster than `grep foo < <(tac file)' (process substitution)?

This question was motivated by "Reverse grepping", about grepping a huge file from bottom up. @chaos said: tac file | grep whatever Or a bit more effective: grep whatever < <(tac file) ...
3
votes
1answer
2k views

LVM2 partition alignment

How to do the following with Linux tools : LVM partition alignment and alignment of partition on LVM ? What in case where I want to setup LVM partition (for "pv") and then partitions ? To be more ...
7
votes
2answers
97 views

How can other processes affect measurements made with `time`?

If I time a process using the time command, I get output for 'real', 'user', and 'sys'. My understanding from this discussion is that 'real' is wall time, whereas 'user' and 'sys' are process time. ...
3
votes
2answers
2k views

Optimal SSD configuration

I am thinking about buying an SSD for my Debian system. However, because I don't know what is the optimal arrangement (i.e. what goes on the SSD), I am not sure what is the optimal size. Obviously I ...
2
votes
1answer
1k views

Interpreting tiobench results

I'm running tiobench on my raid1 and it gives me the following results. Why is there ##### shown instead of throughput? Why does seq.reading with 8 threads use 2836% CPU? My results: $tiobench No ...
18
votes
2answers
1k views

Is dash or some other shell “faster” than bash?

I always thought that the only benefit of using dash instead of bash was that dash was smaller, and therefore many instances of dash would start faster at boot time. But I have done some research, ...
26
votes
4answers
2k views

python vs bc in evaluating 6^6^6

I am evaluating the expression 6^6^6 using python and bc separately. The content of the python file is print 6**6**6. When I execute time python test.py, I get the output as real 0m0.067s ...
18
votes
3answers
17k views

What is the most high-performance Linux filesystem for storing a lot of small files (HDD, not SSD)?

I have a directory tree that contains many small files, and a small number of larger files. The average size of a file is about 1 kilobyte. There are 210158 files and directories in the tree (this ...
11
votes
2answers
1k views

Heavy write activity on SSD nukes system performance

I've noticed that when I do heavy write applications, the whole system slows down. To test this further I ran this to do a (relatively) low-CPU, high disk activity: john -incremental > file_on_SSD ...
9
votes
1answer
2k views

How to make dpkg faster?

Package (un)installation on a Debian system is horribly slow, partly because it works with a whole bunch of (small) files. Short of getting faster storage, how do I speed it up? I'm thinking maybe ...
7
votes
2answers
7k views

Diffing two big text files

I have two big files (6GB each). They are unsorted, with linefeeds (\n) as separators. How can I diff them? It should take under 24h.
11
votes
1answer
5k views

Using Linux cgroups to balance CPU performance

I have two dual-core Linux systems installed using Linux cgroups with relatively recent kernels; one is running Debian Squeeze, the other Ubuntu 11.04 Natty Narwhal. I've gotten CPU load balancing ...
7
votes
5answers
8k views

Why is this network connection so slow?

I am having some problems with network performance speed on a Linux server running Ubuntu 9.10. Transfer speeds on all types of traffic are around 1.5MB/s on a 1000mbit/s wired ethernet connection. ...
4
votes
1answer
3k views

Should I disable hyperthreading when concerned about performance of single-threaded applications?

I use a i5-2410M processor, which is setup to do hyperthreading by default on my laptop. Considering that this is a 2-core processor, this means it can do 4 threads at a time. This also means that ...
3
votes
1answer
2k views

Very high CPU load, but nothing significant in top

I'm running Ubuntu Linux 12.04.1, with VirtualMin 4.08.gpl GPL and 2 CPU cores. Pretty much all the time for the last few weeks, it's been running at well above load average of 5, usually up closer ...
7
votes
2answers
275 views

Reduce bad block retry / wait times in Ubuntu

How can I reduce the IO wait time and retry times so that the OS doesn't continually try to write to a failing drive? I have an system that I use to make copies of demo content that gets loaned out ...
5
votes
5answers
433 views

performance monitoring

Is there some performace monitoring tool which would run in background gathering info about all system activity? Somethimes my system (Arch linux, 32 bit) slows down terribly and the top utility ...
3
votes
1answer
515 views

Efficient way to print lines from a massive file using awk, sed, or something else? [duplicate]

If I had a plain-text file containing 8 million lines and I want to print out lines 4,000,010 to 4,000,000 to the screen, which would be more efficient: awk or sed? There is no pattern to the text, ...
11
votes
1answer
531 views

How do I go about tracking down IO usage in Linux and other *nix OS'?

Occasionally, something on my system starts hammering the disk. I would like to log data that would let me know about I/O usage per process, then I could go back to it later and see what keeps doing ...
7
votes
3answers
3k views

What are the exact differences between awk and cut with grep? [closed]

We know that we can get the second column of the line we want from a file using these two techniques: awk '/WORD/ { print $2 }' filename or grep WORD filename| cut -f 2 -d ' ' My questions are: ...
5
votes
2answers
937 views

Inotifywait for large number of files in a directory

What I want to do is, to monitor a directory (not recursive, just one) for new files created and append those files to one single big file as they are being written. The number of files that are ...
4
votes
2answers
1k views

Why is rm slow?

I have been trying to use rsnapshot for making backups, but I'm finding it unusable. While it is able to diff a directory (50gb) and duplicate it (hardlinking every file) in a few minutes, and I can ...
4
votes
2answers
2k views

100.0%sy with poor disk performance

I have RHEL 5.4 kernel 2.6.18-164.el5 and I am experiencing very bad disk performance that happens randomly during working with Oracle. Nothing appears in the logs. When it happens, I see one CPU ...
3
votes
3answers
317 views

Best way to benchmark different encryption solutions on my system

I want to compare different encryption solutions for encrypting my system, possibly different solutions for different parts of the system such as /usr or /home. In particular I look at dm-crypt ...
2
votes
1answer
424 views

xorg performance in openoffice

I've just been monitoring my cpu usage in openoffice calc when cells have been copied vs when they haven't and seen a dramatic increase in cpu usage for the Xorg process. The additional rendering ...
1
vote
0answers
149 views

Why does my CMS run extremely slow in my workstation (Ubuntu)

For over a year I have been struggling with the response speed on my Drupal v7.2x CMS install. Let me explain. I'm running Drupal 7.2x on an Ubuntu 12.10 (upgrade to 13.xx pending) for the purpose of ...
1
vote
1answer
422 views

Speeding a find rm command with test through parallelization

I want to recursively delete all files in directories and subdirectories with number of lines less than 10, and am currently using the following command find . -type f -name "*.txt" | while read; do ...
1
vote
1answer
280 views

performance on Linux system when daisy-chaining multiple USB hard-drives on USB hubs?

What would the I/O access be like as I plug many USB drives into a Linux system with only 2 USB 2.0 ports and 1 USB 3.0 port? I expect I will have to have multiple levels of fan-out by plugging hubs ...
1
vote
1answer
145 views

What Linux tool will tell me if all the mounted disks share roughly the same bandwidth?

What Linux tool will tell me if all the mounted disks share roughly the same bandwidth? Can I do something like iostat for each of my /dev/sd* devices?