Questions tagged [optimization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
36 views

Optmization and tuning Linux Server (Mediaserver)

I rebuilt a server, specifically a mediaserver, based on debian 12 stable (kernel 6.1). CPU: i7-6700K(no overclock) RAM: 16GB DDR4 (dual channel) NIC: 2 of 2.5Gbit (bonding balance-tlb) (MTU bond0: ...
vincenzogianfelice's user avatar
0 votes
0 answers
61 views

Big Maildir mailspool - looking for ways to improve performance

I have an extremely large Maildir which contains emails going back many years. I need to rebuild the server and would like to improve the mail storage setup. My main problem is that backups of the ...
philpem's user avatar
0 votes
0 answers
12 views

condensing locale requests on every program load

I ran strace on a number of programs (used syntax strace -fe trace=open --insert program here-- ) and every time I execute a program that automatically forks other processes, I notice lines similar to ...
mike_s's user avatar
  • 11
0 votes
0 answers
72 views

PostgreSQL: indexes and partitions

I have a PostgreSQL database and I noticed a weird behaviour while working with indexes and partitions. The engine version is 10.21. Now, I have a table with this structure: guid varchar(50) PK guid_a ...
Federico Loro's user avatar
1 vote
1 answer
147 views

Speed up grep usage inside bash script

I am currently working on creating a bash script that is supposed to process large log files from one of my programs. When I first started the script took around 15 seconds to complete which wasn't ...
Dzamba's user avatar
  • 11
0 votes
1 answer
185 views

Parse huge amounts of files efficiently

I have a folder that holds hunderds of thousands of files called hp-temps.txt. (There are also tons of subfolders) The content of these files looks like this for example: Sensor Location ...
Lumnezia's user avatar
  • 111
0 votes
1 answer
730 views

CLI tool that compress the given image, whatever file type the image is (png, jpg, gif, webp, svg)?

I know that there are many tools to optimize an image: pngquant optipng jpegoptim gifsicle exiftool ecc but they are all specific for a certain file type. Is there a single command line that, ...
nulll's user avatar
  • 235
1 vote
0 answers
170 views

Pipewire using 2x processes when idling

I recently had my BT headphones stutter and cut out and realized that during CPU intensive processing, pipewire is dropping frames. My overall goal, then, is to streamline processing to make that ...
Duane J's user avatar
  • 123
1 vote
0 answers
331 views

Can't compile linux kernel with -Og/-O0 option for debugging purpoces

Having custom hardware running embedded Linux (OpenWrt) like a charm. CPU - is IMX6ULL (ArmV7) so it is supported by Jlink to debug over JTAG interface. Starting GDB server and step by step debugging ...
user3583807's user avatar
0 votes
0 answers
181 views

Why integer division is faster than bitwise shift in shell?

I'm comparing performance of bash and dash (default sh in Xubuntu 18.04). I expect sh to be faster than bash I expect bitwise shift to be faster than division operator. However, I'm getting ...
Zeta.Investigator's user avatar
1 vote
0 answers
504 views

Is jq internal sort slower than GNU sort?

While filtering through this json file I did a benchmark and found out utilizing jq's internal sort and unique method is actually 25% slower than sort --unique! Command Mean [ms] Min [ms] Max [ms] ...
Zeta.Investigator's user avatar
1 vote
2 answers
526 views

How to Build Latest HandBrake on linux with FDO (PGO) + LTO?

Passing CFLAGS and CXXFLAGS to a HandBrake build for the latest version (v1.3.3 at the time of this writing) will work until you add -flto which will FAIL the whole build. How to build HandBrake with ...
DanglingPointer's user avatar
1 vote
0 answers
54 views

How can I frequently update a file without harming my disk?

I've got an i3 install on a laptop with an SSD. Currently I have it configured to save the WM layout on many different events. The tool that I'm using to do this is built on Python, and I'm just ...
wknr's user avatar
  • 11
-1 votes
2 answers
43 views

Optimize time when creating archive

Currently I am using the following command to create an archive with files older than 7 days: find /var/tunningLog/ -type f -mtime +7 -print0 | tar -czf "/var/tunningLog/$(date '+%Y-%m-%d').tar....
dejanualex's user avatar
0 votes
1 answer
197 views

Optimal command-line search inside a sorted text file

Let's say I have a text file with billions of text lines sorted alphabetically, like Bar=10 Foo=6 Naz=42 How can I search for the line starting with Foo in the most optimal way (the file contains ...
Xenos's user avatar
  • 117
4 votes
1 answer
343 views

Is there an equivalent to Jason Randolph's Super Duper mode vim re-mapping for Linux?

I'm very interested in moving from my Mac to a Linux system. The main barrier is my dependence on Jason Rudolph's Keyboard. Specifically, I need an equivalent to (S)uper (D)uper Mode. Super Duper ...
Balba's user avatar
  • 41
2 votes
1 answer
481 views

How to pipe output of command to two separate commands and store outputs

I have a really long command that runs over a huge file and I am forced to run it twice which doubles the time it takes to run. This is what I am doing at the moment: x=$(command | sort -u) y=$(...
markovv.sim's user avatar
2 votes
1 answer
189 views

KSM / MADVISE : Init & usage in standard desktop environment

Running a 4.19 linux kernel + couple of patches RT oriented + occasionally running a couple of processes SCHED_RR (for "pro"-audio purposes) CONFIG_KSM set in the kernel and ksmd running, I ...
MC68020's user avatar
  • 7,981
1 vote
1 answer
3k views

Build the Linux kernel without gcc optimization

I follow one of many tutorials found on Google results to build and debug the Linux kernel with gcc and kgdb/gdb. And I end up by discovering that is all waste of time. Since I can't compile the ...
Joe Smith's user avatar
0 votes
1 answer
29 views

Timing desktop environment initialization

Is there a way to time desktop environment initialization and perhaps identify delaying candidates?
Sterling Butters's user avatar
-3 votes
1 answer
262 views

Why don't BSD kernels use likely() unlike Linux does?

As pointed out in an answer, there's some use of similar hints to compiler in say, FreeBSD and OpenBSD, but those hints coverage of Linux kernel code is a few magnitudes higher. It might make sense to ...
poige's user avatar
  • 6,241
0 votes
2 answers
175 views

How to resize images on sequential functions?

I am trying to: change the format of my images, then resize their height and width of 40% then optimize their quality to 35% max of the source and the total size of the image to 35% of the origin. ...
DiaJos's user avatar
  • 493
0 votes
0 answers
52 views

Optimize raid drive for disk read

I have a Hardware RAID10 setup at home for my home server. I run a Plex Media server and a Nextcloud cloud server for my family at home. My concern is, that when I am downloading a movie (heavy disk ...
VSZM's user avatar
  • 101
2 votes
1 answer
1k views

Can I compile Linux kernel with -Ofast?

I want to know if compiling the Linux kernel with -Ofast is: Worth it. Supported (what level of optimization do the kernel writers take into account?) Secure. The question arised because ...
Agustín Nieto García's user avatar
-1 votes
1 answer
696 views

What are all the possible optimizations I could apply to a new build of Gentoo? [closed]

With the assumption that I'm using hardware with a very modern CPU (9900K), what are all the possible optimizations I could apply to a new build of Gentoo in order to gain the best possible ...
warsong's user avatar
  • 485
1 vote
0 answers
413 views

how do I emerge gcc 9 on gentoo without creating too many processes?

I am stuck on building gcc 9 with lto USE flag, because it is creating over 30 lto processes when I try to do so. At this point, I have to reboot my PC. I think it may have to do with the new lto ...
jeferson lemos's user avatar
-1 votes
2 answers
2k views

Is it safe to disable clamscan and clamd services on linux server

We have a dedicated CentOS server with 8 CPUs. We often observe clamscan and clamav taking too much CPU. I have heard experts saying linux doesn't require an antivirus. If that is true, is it safe to ...
Kannan's user avatar
  • 167
1 vote
1 answer
508 views

Speed up random sort and delete (rm) for multiple directories

Running MacOS in a terminal which is essentially BSD, hence posting my question here instead of askdifferent. I want to trim my image dataset down in subsequent steps, by deleting files randomly. Some ...
SciGuy's user avatar
  • 207
3 votes
0 answers
2k views

Compile a library aiming to reduce size as much as possible

I have to 'squeeze' all the needed code to run ldaps on an embedded system. So the 3 MiB of lybcrypto and almost 700KiB of libssl are way too much. I'm now experimenting on ldapsearch because reading ...
DDS's user avatar
  • 323
1 vote
0 answers
119 views

Nested Virtualization [closed]

I want to deploy openstack with nested virtualization enabled. Here is the scenario, on physical machine (Ubuntu 16.04) I have 2 VMs running, 1. kvm01 and, 2. cmp001 Kvm01: has 7 VMs up and running ...
Parth Mandaliya's user avatar
20 votes
1 answer
9k views

Why can the kernel not use SSE/AVX registers and instructions?

This post from StackOverflow has this, In some environments there there is a restriction on certain instructions or using certain registers. For example, in the Linux kernel, use of SSE/AVX or FP ...
Evan Carroll's user avatar
  • 30.8k
4 votes
1 answer
1k views

Fastest way to count zero bytes on a partition

My root and home partitions are LUKS-encrypted. I wish to have discard/TRIM enabled on them (I'm aware of security implications). If my understanding is correct, this should result in discarded areas ...
gronostaj's user avatar
  • 708
-1 votes
1 answer
3k views

Why configure/build gcc takes much more time than apt-get install gcc [closed]

Wander to make some modification on libgomp (gcc's OpenMP runtime library), I downloaded the source code of GCC8, and tried to configure/make the whole compiler. The configuration phase terminated ...
user292252's user avatar
7 votes
3 answers
1k views

Do file descriptors optimise writing to files?

Is it equivalent to have commands print to a file directly, as opposed to writing to a file descriptor? Illustration Writing to file directly: for i in {1..1000}; do >>x echo "$i"; done ...
user avatar
2 votes
2 answers
1k views

Set usb flash drive as non rotational drive

I'm trying to optimize the IO schedulers and to use a proper scheduler for rotational and for non rotational drives (different). When I run: cat /sys/block/sd*/queue/rotational I get: 1 <-- ...
user avatar
4 votes
3 answers
116 views

Script optimisation to find duplicates filename in huge CSV

I have several CSV file from 1MB to 6GB generated by inotify script with a list of events formatted as is: timestamp;fullpath;event;size. Those files are formatted like this : timestamp;fullpath;...
Kiwy's user avatar
  • 9,544
4 votes
2 answers
4k views

How to force a Thinkpad to draw charge from internal vs external battery?

My T450s Thinkpad has two built-in batteries--one hot-swappable and one inside the laptop. Lenovo calls the former "external" and the latter "internal". When not plugged in, the external battery ...
Stan's user avatar
  • 41
1 vote
2 answers
619 views

Does zgrep reuse recently uncompressed files?

I am calling zgrep over *.gz files in a for loop in one directory. As the log files are huge (about 1GB each, and about 30 files), the command is taking a long time. I see several uncompressed files ...
mtk's user avatar
  • 27.5k
1 vote
1 answer
364 views

How do you specify a blkparse format specifier that also suppresses summary

When using the live mode, I do: blktrace -d /dev/sdc1 -o -|blkparse -f '%a %c' -i - I just want to limit the displayed data to 'action' and 'cpu id'. Instead I get the summary and all the usual ...
linuxGuy's user avatar
6 votes
1 answer
662 views

How to create a configuration to only connect to WiFi if signal is ≥ 30 %?

Is there any chance to create a configuration that does the following job? Only connect to available WiFi if it's signal is stronger than 30 % At many places, I stay in the border area of barely ...
Dave's user avatar
  • 1,036
2 votes
0 answers
389 views

How do backreferences slow down regular expression search?

Let's say I have a text file with 20 chars per line like this: wertzuiopasdfghjkl< asdfghjkl<yxcvbnm,.- <yxcvbnm,.-123456789 1234567890QWERTZUIOP QWERTZUIOPASDFGHJKL< and so on ... and ...
Philippos's user avatar
  • 13.5k
1 vote
0 answers
200 views

Optimal value for make -j <value> [duplicate]

What is the optimal value of the -j flag for make ? I know that all my source is free of errors. Is it the number of cores? Or the number of logical processors?
user31264's user avatar
  • 129
1 vote
1 answer
669 views

How to increase no. of nfsd threads in WD my cloud (nfs server)? for better performance

There are only 8 default nfsd threads available in WD my cloud, this no. is not sufficient for my requirement. I want to increase default nfsd threads to 20 threads. For this I first connected with ...
Amit Yadav's user avatar
2 votes
0 answers
325 views

Is there a linux interface for arm cp15 register access?

For performance optizimization I should be able to switch bits in ACTLR register of the cp15 unit. I've done stuff like this before in bare metal projects, but from linux user space I can't execute ...
Philippos's user avatar
  • 13.5k
2 votes
0 answers
211 views

selinux user add becoming progressively slower

I've written a service that configures a machine with selinux users who can manage a single port. I'm trying to pre-seed that machine with 1000 users, but selinux is crawling to a halt around user ...
Peter Turner's user avatar
  • 1,654
11 votes
1 answer
662 views

How does ksh93 avoid forks in command substitution

Given cmd='fun(){ echo "$@"; }; fun $(fun $(fun hi))' shells tend to need to make 2 forks to make it happen strace-f(){ strace -f "$@" 2>&1; }; for sh in dash bash zsh ksh; do printf ...
Petr Skocik's user avatar
  • 28.8k
21 votes
5 answers
37k views

Compress a large number of large files fast

I have about 200 GB of log data generated daily, distributed among about 150 different log files. I have a script that moves the files to a temporary location and does a tar-bz2 on the temporary ...
anu's user avatar
  • 352
6 votes
1 answer
2k views

SSD optimization

I know this has been discussed many times, but there are a lot of different opinions across the internet which optimizations are good for SSDs (and whether to use them). Also the technology has ...
NefariousOctopus's user avatar
1 vote
2 answers
2k views

Optimize/Replace 'find' command

I am looking to optimize this find command. We have to move a bunch (sometimes almost a million) files to a directory for further processing. We are looking for a way to increase the speed of this ...
Jared Mackey's user avatar
4 votes
2 answers
289 views

How do I write a script to automatically optimize all the PNGs in my MediaWiki images directory?

I have MediaWiki 1.25.2 installed on my 64-bit Debian 8.1 VirtualBox machine and I'd like to be able to upload the images/ directory from this installation to my DropBox. Being a cheapskate I'm trying ...
Josh Pinto's user avatar
  • 3,493