Questions tagged [optimization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
27 votes
4 answers
73k views

How to compile without optimizations -O0 using CMake

I am using Scientific Linux (SL). I am trying to compile a project that uses a bunch of C++ (.cpp) files. In the directory user/project/Build, I enter make to compile and link all the .cpp files. I ...
user4352158's user avatar
21 votes
5 answers
37k views

Compress a large number of large files fast

I have about 200 GB of log data generated daily, distributed among about 150 different log files. I have a script that moves the files to a temporary location and does a tar-bz2 on the temporary ...
anu's user avatar
  • 352
20 votes
1 answer
9k views

Why can the kernel not use SSE/AVX registers and instructions?

This post from StackOverflow has this, In some environments there there is a restriction on certain instructions or using certain registers. For example, in the Linux kernel, use of SSE/AVX or FP ...
Evan Carroll's user avatar
  • 30.9k
11 votes
1 answer
663 views

How does ksh93 avoid forks in command substitution

Given cmd='fun(){ echo "$@"; }; fun $(fun $(fun hi))' shells tend to need to make 2 forks to make it happen strace-f(){ strace -f "$@" 2>&1; }; for sh in dash bash zsh ksh; do printf ...
Petr Skocik's user avatar
  • 28.8k
7 votes
3 answers
1k views

Do file descriptors optimise writing to files?

Is it equivalent to have commands print to a file directly, as opposed to writing to a file descriptor? Illustration Writing to file directly: for i in {1..1000}; do >>x echo "$i"; done ...
user avatar
6 votes
4 answers
40k views

How to calculate the memory consumed by a "C" program in linux

I have written two C programmes one is using function pointer, and the other without function pointer. Now i want to know the memory consumed by the two programmes , to see how memory can ...
Nagraj's user avatar
  • 61
6 votes
1 answer
662 views

How to create a configuration to only connect to WiFi if signal is ≥ 30 %?

Is there any chance to create a configuration that does the following job? Only connect to available WiFi if it's signal is stronger than 30 % At many places, I stay in the border area of barely ...
Dave's user avatar
  • 1,036
6 votes
1 answer
2k views

SSD optimization

I know this has been discussed many times, but there are a lot of different opinions across the internet which optimizations are good for SSDs (and whether to use them). Also the technology has ...
NefariousOctopus's user avatar
5 votes
3 answers
16k views

Allocating Swap Space with KVM

Consider the following scenario: a host with 2 GiB runs a few guests using KVM.  Each guest does usually not need much memory; they are given 256 MiB each and run services that mostly twiddle their ...
countermode's user avatar
  • 7,543
4 votes
3 answers
116 views

Script optimisation to find duplicates filename in huge CSV

I have several CSV file from 1MB to 6GB generated by inotify script with a list of events formatted as is: timestamp;fullpath;event;size. Those files are formatted like this : timestamp;fullpath;...
Kiwy's user avatar
  • 9,554
4 votes
2 answers
289 views

How do I write a script to automatically optimize all the PNGs in my MediaWiki images directory?

I have MediaWiki 1.25.2 installed on my 64-bit Debian 8.1 VirtualBox machine and I'd like to be able to upload the images/ directory from this installation to my DropBox. Being a cheapskate I'm trying ...
Josh Pinto's user avatar
  • 3,493
4 votes
1 answer
343 views

Is there an equivalent to Jason Randolph's Super Duper mode vim re-mapping for Linux?

I'm very interested in moving from my Mac to a Linux system. The main barrier is my dependence on Jason Rudolph's Keyboard. Specifically, I need an equivalent to (S)uper (D)uper Mode. Super Duper ...
Balba's user avatar
  • 41
4 votes
1 answer
1k views

Fastest way to count zero bytes on a partition

My root and home partitions are LUKS-encrypted. I wish to have discard/TRIM enabled on them (I'm aware of security implications). If my understanding is correct, this should result in discarded areas ...
gronostaj's user avatar
  • 708
4 votes
2 answers
4k views

How to force a Thinkpad to draw charge from internal vs external battery?

My T450s Thinkpad has two built-in batteries--one hot-swappable and one inside the laptop. Lenovo calls the former "external" and the latter "internal". When not plugged in, the external battery ...
Stan's user avatar
  • 41
4 votes
2 answers
950 views

CPU optimized compiled debian packages

I really don't know how the Debian maintainers check for all the bugs in every release of Debian, but isn't it the case that if all codes for every package have been patched up and tested for all CPU ...
Dehbop's user avatar
  • 4,659
3 votes
0 answers
2k views

Compile a library aiming to reduce size as much as possible

I have to 'squeeze' all the needed code to run ldaps on an embedded system. So the 3 MiB of lybcrypto and almost 700KiB of libssl are way too much. I'm now experimenting on ldapsearch because reading ...
DDS's user avatar
  • 323
2 votes
1 answer
482 views

How to pipe output of command to two separate commands and store outputs

I have a really long command that runs over a huge file and I am forced to run it twice which doubles the time it takes to run. This is what I am doing at the moment: x=$(command | sort -u) y=$(...
markovv.sim's user avatar
2 votes
1 answer
189 views

KSM / MADVISE : Init & usage in standard desktop environment

Running a 4.19 linux kernel + couple of patches RT oriented + occasionally running a couple of processes SCHED_RR (for "pro"-audio purposes) CONFIG_KSM set in the kernel and ksmd running, I ...
MC68020's user avatar
  • 7,981
2 votes
1 answer
1k views

Can I compile Linux kernel with -Ofast?

I want to know if compiling the Linux kernel with -Ofast is: Worth it. Supported (what level of optimization do the kernel writers take into account?) Secure. The question arised because ...
Agustín Nieto García's user avatar
2 votes
2 answers
1k views

Set usb flash drive as non rotational drive

I'm trying to optimize the IO schedulers and to use a proper scheduler for rotational and for non rotational drives (different). When I run: cat /sys/block/sd*/queue/rotational I get: 1 <-- ...
user avatar
2 votes
1 answer
87 views

optimisation of bash globs

Looking for a guide to optimising regexp matches in bash. I have a script that loops over a very long list of URLs looking for patterns. Currently it looks a little like the fragment below. Is there ...
user2085362's user avatar
2 votes
0 answers
389 views

How do backreferences slow down regular expression search?

Let's say I have a text file with 20 chars per line like this: wertzuiopasdfghjkl< asdfghjkl<yxcvbnm,.- <yxcvbnm,.-123456789 1234567890QWERTZUIOP QWERTZUIOPASDFGHJKL< and so on ... and ...
Philippos's user avatar
  • 13.5k
2 votes
0 answers
325 views

Is there a linux interface for arm cp15 register access?

For performance optizimization I should be able to switch bits in ACTLR register of the cp15 unit. I've done stuff like this before in bare metal projects, but from linux user space I can't execute ...
Philippos's user avatar
  • 13.5k
2 votes
0 answers
211 views

selinux user add becoming progressively slower

I've written a service that configures a machine with selinux users who can manage a single port. I'm trying to pre-seed that machine with 1000 users, but selinux is crawling to a halt around user ...
Peter Turner's user avatar
  • 1,664
2 votes
0 answers
446 views

Ubuntu Bootchart - How do I actually interpret this chart?

I'd like to start obsessing over optimizing my Ubuntu boot time, but I've searched online for multiple days—and only have a vaguely more coherent idea of what I'm looking at. Here's my bootchart. I ...
AmagicalFishy's user avatar
1 vote
2 answers
619 views

Does zgrep reuse recently uncompressed files?

I am calling zgrep over *.gz files in a for loop in one directory. As the log files are huge (about 1GB each, and about 30 files), the command is taking a long time. I see several uncompressed files ...
mtk's user avatar
  • 27.6k
1 vote
1 answer
147 views

Speed up grep usage inside bash script

I am currently working on creating a bash script that is supposed to process large log files from one of my programs. When I first started the script took around 15 seconds to complete which wasn't ...
Dzamba's user avatar
  • 11
1 vote
2 answers
528 views

How to Build Latest HandBrake on linux with FDO (PGO) + LTO?

Passing CFLAGS and CXXFLAGS to a HandBrake build for the latest version (v1.3.3 at the time of this writing) will work until you add -flto which will FAIL the whole build. How to build HandBrake with ...
DanglingPointer's user avatar
1 vote
1 answer
510 views

Speed up random sort and delete (rm) for multiple directories

Running MacOS in a terminal which is essentially BSD, hence posting my question here instead of askdifferent. I want to trim my image dataset down in subsequent steps, by deleting files randomly. Some ...
SciGuy's user avatar
  • 207
1 vote
1 answer
669 views

How to increase no. of nfsd threads in WD my cloud (nfs server)? for better performance

There are only 8 default nfsd threads available in WD my cloud, this no. is not sufficient for my requirement. I want to increase default nfsd threads to 20 threads. For this I first connected with ...
Amit Yadav's user avatar
1 vote
2 answers
2k views

Optimize/Replace 'find' command

I am looking to optimize this find command. We have to move a bunch (sometimes almost a million) files to a directory for further processing. We are looking for a way to increase the speed of this ...
Jared Mackey's user avatar
1 vote
1 answer
864 views

How to speed up an apache webserver?

We have a PC: Intel(R) Core(TM)2 Duo CPU E7300 @ 2.66GHz 3 GByte RAM HDD: 160 GByte WD1600AAJS-08L7A0 And we have Ubuntu 12.04 on it with apache 2.2.22-1ubuntu1.7. Serving just static HTML files (...
user avatar
1 vote
0 answers
170 views

Pipewire using 2x processes when idling

I recently had my BT headphones stutter and cut out and realized that during CPU intensive processing, pipewire is dropping frames. My overall goal, then, is to streamline processing to make that ...
Duane J's user avatar
  • 123
1 vote
0 answers
332 views

Can't compile linux kernel with -Og/-O0 option for debugging purpoces

Having custom hardware running embedded Linux (OpenWrt) like a charm. CPU - is IMX6ULL (ArmV7) so it is supported by Jlink to debug over JTAG interface. Starting GDB server and step by step debugging ...
user3583807's user avatar
1 vote
0 answers
504 views

Is jq internal sort slower than GNU sort?

While filtering through this json file I did a benchmark and found out utilizing jq's internal sort and unique method is actually 25% slower than sort --unique! Command Mean [ms] Min [ms] Max [ms] ...
Zeta.Investigator's user avatar
1 vote
0 answers
54 views

How can I frequently update a file without harming my disk?

I've got an i3 install on a laptop with an SSD. Currently I have it configured to save the WM layout on many different events. The tool that I'm using to do this is built on Python, and I'm just ...
wknr's user avatar
  • 11
1 vote
0 answers
413 views

how do I emerge gcc 9 on gentoo without creating too many processes?

I am stuck on building gcc 9 with lto USE flag, because it is creating over 30 lto processes when I try to do so. At this point, I have to reboot my PC. I think it may have to do with the new lto ...
jeferson lemos's user avatar
1 vote
0 answers
119 views

Nested Virtualization [closed]

I want to deploy openstack with nested virtualization enabled. Here is the scenario, on physical machine (Ubuntu 16.04) I have 2 VMs running, 1. kvm01 and, 2. cmp001 Kvm01: has 7 VMs up and running ...
Parth Mandaliya's user avatar
1 vote
1 answer
364 views

How do you specify a blkparse format specifier that also suppresses summary

When using the live mode, I do: blktrace -d /dev/sdc1 -o -|blkparse -f '%a %c' -i - I just want to limit the displayed data to 'action' and 'cpu id'. Instead I get the summary and all the usual ...
linuxGuy's user avatar
1 vote
0 answers
200 views

Optimal value for make -j <value> [duplicate]

What is the optimal value of the -j flag for make ? I know that all my source is free of errors. Is it the number of cores? Or the number of logical processors?
user31264's user avatar
  • 129
1 vote
0 answers
152 views

VNC rendering is all fragmented

I have my VNC server so configured, that it requires me to establish an ssh tunnel and connect to it on localhost:1 with my vncviewer. Now the connectivity seems to work fine but the rendering of the ...
stdcerr's user avatar
  • 2,079
1 vote
0 answers
839 views

What are the consequences of disabling nmi_watchdog?

My aim was to improve battery life for my laptop. I have already applied some advised modifications like enabling ALPM, and adding pcie_aspm=force drm.vblankoffdelay=1 i915.semaphores=1 at grub ...
Stephane Rolland's user avatar
1 vote
1 answer
3k views

Build the Linux kernel without gcc optimization

I follow one of many tutorials found on Google results to build and debug the Linux kernel with gcc and kgdb/gdb. And I end up by discovering that is all waste of time. Since I can't compile the ...
Joe Smith's user avatar
0 votes
1 answer
187 views

Parse huge amounts of files efficiently

I have a folder that holds hunderds of thousands of files called hp-temps.txt. (There are also tons of subfolders) The content of these files looks like this for example: Sensor Location ...
Lumnezia's user avatar
  • 111
0 votes
0 answers
36 views

Optmization and tuning Linux Server (Mediaserver)

I rebuilt a server, specifically a mediaserver, based on debian 12 stable (kernel 6.1). CPU: i7-6700K(no overclock) RAM: 16GB DDR4 (dual channel) NIC: 2 of 2.5Gbit (bonding balance-tlb) (MTU bond0: ...
vincenzogianfelice's user avatar
0 votes
0 answers
61 views

Big Maildir mailspool - looking for ways to improve performance

I have an extremely large Maildir which contains emails going back many years. I need to rebuild the server and would like to improve the mail storage setup. My main problem is that backups of the ...
philpem's user avatar
0 votes
0 answers
12 views

condensing locale requests on every program load

I ran strace on a number of programs (used syntax strace -fe trace=open --insert program here-- ) and every time I execute a program that automatically forks other processes, I notice lines similar to ...
mike_s's user avatar
  • 11
0 votes
0 answers
72 views

PostgreSQL: indexes and partitions

I have a PostgreSQL database and I noticed a weird behaviour while working with indexes and partitions. The engine version is 10.21. Now, I have a table with this structure: guid varchar(50) PK guid_a ...
Federico Loro's user avatar
0 votes
1 answer
732 views

CLI tool that compress the given image, whatever file type the image is (png, jpg, gif, webp, svg)?

I know that there are many tools to optimize an image: pngquant optipng jpegoptim gifsicle exiftool ecc but they are all specific for a certain file type. Is there a single command line that, ...
nulll's user avatar
  • 235
0 votes
0 answers
181 views

Why integer division is faster than bitwise shift in shell?

I'm comparing performance of bash and dash (default sh in Xubuntu 18.04). I expect sh to be faster than bash I expect bitwise shift to be faster than division operator. However, I'm getting ...
Zeta.Investigator's user avatar