Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [performance]

The speed at which a program runs

1
vote
1answer
20 views

Finding performance bottlenecks on Red Hat server hosting KVMs with QEMU?

As a side task I administer a RHEL 7 server used as an internal KVM host. Now it runs eight VMs which, but for one, are idle most of their time, or at least not doing anything at all resource ...
6
votes
1answer
222 views

Huge performance difference of the command find with and without using %M option to show permissions

On my CentOS 7.6, I have created a folder (called many_files) with 3,000,000 files, by running: for i in {1..3000000}; do echo $i>$i; done; I am using the command find to write the information ...
28
votes
11answers
6k views

Why doesn't shell automatically fix “useless use of cat”? [closed]

Many people use oneliners and scripts containing code along the lines cat "$MYFILE" | command1 | command2 > "$OUTPUT" The first cat is often called "useless use of cat" because technically it ...
2
votes
0answers
38 views

Small delay in sudo

I want my zsh prompt to display whether the session has sudo rights (from a previously run sudo command). After looking at the answers to this question, I arranged to use the exit status of sudo -n ...
-1
votes
1answer
24 views

LFTP - Remove files from remote server while uploading via lftp mirror --reverse

I'm using a shell script which executes lftp mirror --reverse to upload files and directories to a remote server. Just before that it removes everything using glob -a rm -r -f *. The problem is, it is ...
0
votes
2answers
76 views

How to search for large directory containing thousands of files?

Under the folder /grid/sdh/hadoop/yarn/local/usercache/hdfs/appcache we have more than 100 recursive folders. One of the folders contain thousands of files. Is it possible to identify this folder? ...
0
votes
1answer
87 views

Linux directories read/write behaviour?

I need to know how the directories of a Linux system are used, in relation to their approximate read/write behaviour in time. So far I have not been able to find any references. Can anybody provide ...
1
vote
0answers
38 views

Disk utilization on mdadm raid 5 is 100% even though none of the member disks is at 100%

I am currently confused about the disk utilisation of one of my machines. The setup: I have a machine containing 4 2TB HDDs. For those HDDs I used mdadm to configure a RAID 5 (called md2) On MD2, I ...
1
vote
1answer
37 views

Better more efficient bash script (grep)

I have a script that I built .. And it works fantastically -- BUT it's going to take an estimated 4 days to run! I was wondering if there's a more efficient way of doing this. Here's what the ...
5
votes
2answers
162 views

Why is `sed` no op much faster than `awk` in this case

I am trying to understand some performance issues related to sed and awk, and I did the following experiment, $ seq 100000 > test $ yes 'NR==100001{print}' | head -n 5000 > test.awk $ yes '...
2
votes
1answer
45 views

What is the fastest way to create multiple thousands of users on a system?

I need to create multiple thousands of users on a linux system. Currently I use the newusers command with a file as the source for all the users. The file contains around 10000 users. Running $ ...
0
votes
0answers
18 views

Performance reduction activating trace-functionality in kernel

Is there any performance reduction to be expected from just activating trace (ftrace) functionality in the Linux kernel, but not using it at run time?
5
votes
1answer
144 views

How does the Linux Kernel handle newer chips that patched only against Spectre Variant 1 but not Variant 2?

How will chips patched for Spectre Variant 1 and Meltdown, like Whiskey Lake and Amber Lake, handle Spectre Variant 2? I'm looking to spec out a new laptop. Currently evaluating the Lenovo x390. It's ...
0
votes
2answers
40 views

Bash Shell takes variable amount of time to load

I use bash on Ubuntu 18.04. I have added a lot of functions and a lot of scripts are sourced in my bashrc. I am able to time the total time it takes to load my terminal, sometimes it takes 0.9 seconds ...
0
votes
2answers
48 views

Debugging boot performance issues in grub / before kernel logging starts

My PC takes a long time booting. I have reason to believe it is caused by delays after grub, but before kernel logging starts (booting takes 30 seconds, but dmesg messages have time stamps between 0....
2
votes
1answer
89 views

filtering a large file with a larg filter

I want to extract all lines of $file1 that start with a string stored in $file2. $file1 is 4 GB large with about 20 million lines, $file2 has 2 million lines, is about 140 MB large and contains two ...
0
votes
0answers
17 views

How to diagnose the difference in OS performance between installation media?

I have two different USB sticks, a 128 GB Samsung and a 64 GB SanDisk. I used Gnome Disks to benchmark both of them and the Samsung one was supposedly better, it had faster read and write speed but it ...
1
vote
2answers
63 views

Long time spent between grub2 menu, before kernel boot

I installed a minimal ubuntu (xubuntu-core). It works well, but boot is a bit too slow. Right after I make my choice in grub, the screen turns black and after around 15 seconds I see the kernel boot ...
5
votes
1answer
271 views

efficiently grep an interval of a sorted file

My file has millions of lines, resides in memory /dev/shm/tmp.file, is accessed by multiple threads, looks like this 831092,25a1bd66f2eec71aa2f0a8bb3d,/path/to/a/file 4324,8d83c29e4d8c71bd66f1bd66fs,/...
1
vote
1answer
57 views

How to analyze top command results: CPU & RAM consumption

I am working on ARM-based processor (OS version: Linux 3.4.35) and I need to analyze the processor's performance while some processes are running, by typing top command, I can see some statistics but ...
0
votes
0answers
22 views

Sizing of Samba Linux Server

Do you know any performance of sizing recommendations about Samba 4.7.1 ? For few informations : 2vCPUs / 1 Core 8GB of RAM 140 simultaneous Windows users / 250 at most / 20 NFS volumes mounted via ...
24
votes
8answers
2k views

Is piping, shifting, or parameter expansion more efficient?

I'm trying to find the most efficient way to iterate through certain values that are a consistent number of values away from each other in a space separated list of words(I don't want to use an array)....
0
votes
0answers
8 views

The nginx provide data download rate is not meet the iperf3 test

I use iperf3 test the IP, there gets 9.3G/s [root@localhost ~]# iperf3 -c 4.4.226.59 Connecting to host 43.224.226.59, port 5201 [ 4] local 4.4.226.69 port 57212 connected to 4.4.226.59 port 5201 [ ...
0
votes
2answers
77 views

Fast hashing in awk?

I find it cool that someone has written MD5 in AWK: https://kaworu.ch/blog/2017/05/23/implementing-md5-in-AWK/ Unfortunately that one is quite slow. I am looking for a fast hashing tool for AWK that ...
4
votes
0answers
44 views

Case insensitve tab completion zsh without increasing startup time significantly

So I wanted to get away from oh-my-zsh because it made my shell startup lag noticeably (>0.1s). Indeed, after removing oh-my-zsh the startup time went to 0.02s but now I started adding features ...
3
votes
2answers
100 views

Linux GUI becomes very unresponsive when doing heavy disk I/O - what to tune?

I have a device /dev/mydisk that is based on a stack of functionality: a LUKS-encrypted, software RAID-1. From time to time, I do a backup of /dev/mydisk contents to an external USB disk, which is ...
0
votes
0answers
42 views

Samba shared folder over openvpn very slow - Linux OS

Recently I've installed Ubuntu 18.04 and I've noticed that sharing a samba folder from a Linux server over a openvpn tunnel is very very slow. Tried to avoid using DNS, but still have no results. ...
0
votes
2answers
33 views

How to remove trailing CR quickly?

This is my source file, it dumps a database into a JSON and worked great with Debian wheezy: #!/bin/sh echo "Status: 200" echo echo { printf ".mode csv\nselect * from cfg;" | sqlite3 /home/ftproot/...
1
vote
2answers
37 views

Speeding up repeated python calls (or, alternatively, porting a complex regex to sed)

I am an academic medical physicist. I do experiments that generate a fair amount of data, and they are expensive to run. My university has a backup system that consists of a robot tape library in a ...
2
votes
1answer
38 views

Improving performance of Script Using find and exec [closed]

I have a script that iterates through a given directory and automatically compresses those that do not contain at least one file used in less than 30 days. Now, I am wondering if I could improve ...
0
votes
1answer
25 views

OS keeps getting slower the more time I leave it open

I recently decided against shutting down my Ubuntu machine in office because it took a lot of time to start it and then open all the applications that I would normally require. After leaving the PC ...
0
votes
1answer
21 views

mv command for many nonexistent files, does it wear off the destination disk at all?

I am trying to continuously move many files (~ 10 000 each minute) from /ramdisk to normal /ssd , but not all files are there. When the mv /ramdisk/file55 /ssd/file55 fails because source is not ...
3
votes
1answer
71 views

Why would NFS have lag after a minute of inactivity?

I am running a version 4 NFS server on a Salix server with kernel 3.10.17. I am mounting the NFS share with the command sudo mount -t nfs 10.0.0.150:/share /mountpoint on my client machine, which is ...
2
votes
0answers
73 views

Linux access windows share without loading to RAM

I have a laptop running an Arch-based Linux system and frequently need to access a windows share at my workplace. This is really annoying since it takes ages to load. Particularly, one folder is ...
22
votes
2answers
2k views

Why does the `updatedb` program run so fast?

Usually when I have programs that are doing a full disk scan and going over all files in the system they take a very long time to run. Why does updatedb run so fast in comparison?
1
vote
0answers
29 views

Performance issues on internal SATA device causing high load

I have a problem with the performance of my laptop. In fact I needed to install quite some things recently and the whole machine froze from time to time completely. First some general information: I ...
2
votes
1answer
116 views

USB 3.0 external hard disk only operating at 480 Mbit/s

When I attach my USB 3.0-Stick dmesg says: [ 3308.966755] usb 2-1.1: new SuperSpeed USB device number 5 using xhci_hcd [ 3308.987443] usb 2-1.1: New USB device found, idVendor=0781, idProduct=5583 [ ...
0
votes
0answers
59 views

Debian 9.5 takes longer to mount emmc than 8.7 did

I have been trying to chase down some boot performance issues for a little while now. Previously we were running debian 8.7 with 4.4.54 kernel. our start up time is systemd-analyze Startup finished ...
0
votes
0answers
60 views

How to calculate the throughput usage of AWS EBS volume

We have a hadoop cluster running on AWS EC2 instances. Configured 16T st1 volume in datanodes for HDFS usage. We need to understand, what is the maximum throughput hitting on this st1 volumes. Since ...
1
vote
2answers
40 views

compare memory performance of two machines (low buffers vs high cached)

I'm trying to compare the performance of two machines that I have. I have in both of the machines a database that was installed with the same settings. Both of the machines have the same amount of cpu(...
1
vote
1answer
41 views

Zsh Performance: Should I create scripts or just add functions to a file I source in .zshrc?

I use a lot of little zsh functions. I have been adding them to a single functions.zsh file which I source in .zshrc. If they are created as scripts, or autoloading is used, will that improve speed? ...
0
votes
2answers
137 views

What exactly does `ps -p PID` do that `ps -q PID` does not?

According to man ps: -p pidlist Select by PID. This selects the processes whose process ID numbers appear in pidlist. Identical to p and --pid. -q pidlist ...
0
votes
2answers
171 views

Is there a way to predict or count the number of page faults?

I am working on hardware performance counter (HPCs) and I seek to use these counters to measure some events accurately. I have found that the non-determinism in hardware performance counter (...
2
votes
1answer
271 views

Why do terminal emulators still have screen flicker?

Why are there still visual artifacts when terminal emulators draw text-based applications? This on recent computers that render 3D games and GUI windows including anti-aliased vector fonts with no ...
2
votes
1answer
250 views

Send sar series to grafana [closed]

sar of sysstat package produces a lot of useful information about system usage regarding various resources; My question (since after a brief query I was unable to find a useful suggestion / solution) ...
0
votes
1answer
26 views

Can I distinguish between swap thrashing and non-swap high IO latency somehow?

My system becomes much, much less responsive, every time I clone a VM image. I'm using virt-manager, and I can see the IO is performed by several qemu-img convert threads. I tried to gather some ...
-1
votes
1answer
299 views

/proc/sys/vm/drop_caches values

I want to know what values are possible for drop_caches, and what the meaning of that values are. For example: echo 3 > /proc/sys/vm/drop_caches What does number 3 mean? Are the numbers, 1, 2 or ...
0
votes
1answer
51 views

Disk I/O Information

I want to get the read and write, service time, queue length and wait time of my disks. the OS is CentOS 6. I use iostat. when I run this command: iostat -x -d /dev/sda the output is: ...
0
votes
0answers
61 views

Performance of active port - netstat

Below command [root@machine1 output]# netstat -s --udp IcmpMsg: InType0: 371137 InType3: 296 InType8: 386843 InType11: 4 InType13: 141 InType15:...
3
votes
2answers
278 views

Disk Performance Monitoring [closed]

I have a physical server with 2 disks, sda and sdb. I want to monitor their I/O and performance. The monitoring element has 3 types: ops, sps, and bps. What are these and which one is better to ...