Tag Info

New answers tagged


When SMP support was first added to Linux, it used a "giant lock" or "BKL" (big kernel lock), this was still present right up until a few years ago. This effectively made the kernel single threaded (except I believe for hardware interrupt servicing), so no more than one syscall could be active, which of course limited performance for many types of workload. ...


Based on http://www.gnu.org/software/parallel/man.html#EXAMPLE:-Grepping-n-lines-for-m-regular-expressions parallel -k parallel --pipepart --block 100M -a file2 -k grep {} :::: file1 Simpler and maybe faster: parallel -j0 -k grep {} file2 :::: file1


grep -F -f needles.txt haystack.log is what you want. -F makes grep use simple pattern matching, if you do not need full regex. This tends to be a HUGE speedup. Depending on what you're searching for, you might not need regex, so might as well get the benefits of faster execution of simpler code. -f is a file of patterns you're looking for. So instead ...


There is other way to make things faster: Use grep -f file1 file2 >output.txt. You could also use gnu parallel: http://www.gnu.org/software/parallel/parallel_tutorial.html

Top 50 recent answers are included