6
votes
2answers
93 views

How come sed is so fast?

I thought about whether this question is suitable for SE or not, I hope you agree it is. Some time ago I asked on SE how to find text in files and leave the file with only the matching lines that ...
0
votes
1answer
219 views

How do I create a RAM disk on Linux Mint?

I am running a package that adds data to the end of 140–300 files each time through the process. I'm thinking of moving the process to a RAM disk, and using rsync to copy it back when done. My goal ...
2
votes
3answers
2k views

Grep in couple thousands files

I have directory with cca 26 000 files and I need to grep in all these files. Problem is, that I need it as fast as possible, so it's not ideal to make script where grep will take name of one file ...
0
votes
1answer
291 views

Simple bash script taking too long to execute

I wrote the following function in my .bashrc for use instead of rm * : function re() { cp . -R /home/glset/ritwik/trash/ rm * } It works .. but it is very slow. Even for small files it takes too ...
5
votes
1answer
2k views

Fast elimination of duplicate lines across multiple files

I have a huge amount of data in which each (data-)line should be unique. There are a lot of files in one folder in which this is already true. It is about 15GB splitted into roughly 170 files with ...
2
votes
1answer
133 views

Read only new parts of a file?

I have a large CSV file, that grows by some entries every day (created by an external software which writes always to this one file, no splitting possible). For performance reasons and further ...
11
votes
4answers
7k views

Use of O_DIRECT on Linux

If this question is too programmer oriented, let me know. I wonder if there are people familiar with the O_DIRECT flag for the open() system call on Linux 2.6? Linus disparages its use, however high ...