Does anyone here know the resources it takes for editors to search and replace in a large file? The reason I am asking, is I have a 32 core server that HTOP is only showing one core at 100% when search/replace on a 3GB file with an editor. I am wondering if my editor search/replace is a single threaded, and if so is there a way to delegate more resources so that these tasks don't take so long? I have to say, it is frustrating to see 31 idle cores and 1 at 100% when a task is taking 25 to 30 minutes.

Oh and if it makes a difference, the RAM is 32GB sitting at 19GB used including cache.

migrated from serverfault.com Feb 13 '13 at 21:32

This question came from our site for system and network administrators.

  • 1
    Humph... why don't you split the file into 32 portions and run 32 instances of the script and then splice the results? (Which is, BTW, the whole point of Map/Reduce) – Deer Hunter Feb 13 '13 at 18:44
  • 2
    If you are doing a search and replace on a 3GB file, then I strongly suggest you look at using the stream editor. – Zoredache Feb 13 '13 at 19:00
  • A few more commands for you to look for: in bash - wait, split -b, cat. – Deer Hunter Feb 13 '13 at 19:15

An editor may or may not be multi-threaded, but, even if it is, it is unlikely to use threads for this purpose for one simple reason: doing so would not provide any advantages for normative use, and it would no doubt create developer headaches and possibly compromise features that are considered important (for normative use).

Given an infinite amount of time and an infinite number of programmers, no doubt all software would be wildly optimized down to the smallest most irrelevant details, extensively tested to ensure these optimizations do not negatively impact anything, etc. Sans that, no one wants to spend their time coding features 99.9% of users will never to appreciate, especially if the 0.1% who do do so because, eg, (an analogy) they really wanted to open soup cans with a hammer.

As a few people have pointed out, loading a 3 GB file into a text editor to do a search and replace is well, something you would do if the only way you knew how to do a search and replace is in a text editor. I'm not trying to insult you with that, BTW, just give you a friendly nudge -- now is the time to broaden some horizons ;)


Most likely, the editors are single threaded. You'd probably be better off splitting your file into 32 pieces and using something like perl or sed to search and replace.


Check out sed, the stream editor. It has a command set somewhat like vi, but doesn't read the file in to process it, it reads, modifies and writes out one line at a time (mostly, look at the manual). So you can at least reduce the time needed to read in the file (the editor has to build complex in-memory data structures) and then write it out.

[I'm (not so) surprised that the current batch of editors are able to process such files, I fondly remember that the original vi crashed badly with files that were a few dozen of KiB in size... sic transit gloria mundii.]


Search and replace over 3GB of text is a hard task for any editor. The best solution in my opinion, is to use Perl. You can use perl to automatically divide the file into smaller portions and run the regex in parallel on each portion. There is lots of way you can code this into perl. I will post one example later.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.