Editors Nano/VIM not using resources

Question

Does anyone here know the resources it takes for editors to search and replace in a large file? The reason I am asking, is I have a 32 core server that HTOP is only showing one core at 100% when search/replace on a 3GB file with an editor. I am wondering if my editor search/replace is a single threaded, and if so is there a way to delegate more resources so that these tasks don't take so long? I have to say, it is frustrating to see 31 idle cores and 1 at 100% when a task is taking 25 to 30 minutes.

Oh and if it makes a difference, the RAM is 32GB sitting at 19GB used including cache.

Answer 1

An editor may or may not be multi-threaded, but, even if it is, it is unlikely to use threads for this purpose for one simple reason: doing so would not provide any advantages for normative use, and it would no doubt create developer headaches and possibly compromise features that are considered important (for normative use).

Given an infinite amount of time and an infinite number of programmers, no doubt all software would be wildly optimized down to the smallest most irrelevant details, extensively tested to ensure these optimizations do not negatively impact anything, etc. Sans that, no one wants to spend their time coding features 99.9% of users will never to appreciate, especially if the 0.1% who do do so because, eg, (an analogy) they really wanted to open soup cans with a hammer.

As a few people have pointed out, loading a 3 GB file into a text editor to do a search and replace is well, something you would do if the only way you knew how to do a search and replace is in a text editor. I'm not trying to insult you with that, BTW, just give you a friendly nudge -- now is the time to broaden some horizons ;)

Answer 2

Most likely, the editors are single threaded. You'd probably be better off splitting your file into 32 pieces and using something like perl or sed to search and replace.

Answer 3

Check out sed, the stream editor. It has a command set somewhat like vi, but doesn't read the file in to process it, it reads, modifies and writes out one line at a time (mostly, look at the manual). So you can at least reduce the time needed to read in the file (the editor has to build complex in-memory data structures) and then write it out.

[I'm (not so) surprised that the current batch of editors are able to process such files, I fondly remember that the original vi crashed badly with files that were a few dozen of KiB in size... sic transit gloria mundii.]

Answer 4

Search and replace over 3GB of text is a hard task for any editor. The best solution in my opinion, is to use Perl. You can use perl to automatically divide the file into smaller portions and run the regex in parallel on each portion. There is lots of way you can code this into perl. I will post one example later.