How can I edit a really big file with vi? e.g.: a 20 GByte log file. Are there any modifications for it to handle this size?

  • 1
    What are you exactly trying to do with such a file? – Adam Byrtek May 26 '11 at 17:47

If you're using Vim on a 32-bit system, I don't think there is a way to make it do this. It has a hard file size limit based on the size of a 32-bit integer.

If you can do your editing on a 64-bit system, Vim's file size limit becomes something in the exabytes range.

Vim also has an arbitrary lower limit, over which it will page chunks of the file into RAM from its swap file. On the 64-bit CentOS 5.x system I'm currently using, the Vim default for this is approximately 3 GB. You can raise the maxmem and maxmemtot limits in order to avoid swapping, if you have enough real RAM to load the entire file. If you do not, you'll end up using the OS's general-purpose swap space instead, which probably won't be any faster. To be clear, you do not need to raise this limit. It just allows Vim to use more real RAM, if you have it.

If you mean some other implementation of vi, please say which. File size limits are one of the areas where vi implementations differ.

  • 1
    I opened an 8GB file in Gvim 32-bit (4 GB RAM)... it took 41 minutes to load :) .. askubuntu.com/questions/28847/… – Peter.O May 26 '11 at 17:02
  • @fred: Interesting. It appears that the Vim docs are out of date. It must now know how to do swapping to get around the 2 GB limit on 32-bit systems. If you check your maxmem setting (:set mm) you will probably find that it is much lower than I found on my 64-bit system. The file open time is due to all the swapping needed. If you can raise maxmem by giving it real RAM, it may get a lot faster. Ultimately, the solution is still to do such things on a 64-bit box with lots of RAM. – Warren Young May 26 '11 at 17:17
  • My maxmem=1660058 .... Most definitely RAM will help.. but have to saving 20 GB would certainly make me think twice about making multiple/regualar saves, unless vim is somehow very smart about how it saves data, and/or the file is on an SSD drive :)... editing 20 GB; interesting... – Peter.O May 26 '11 at 17:52
  • My maxmem is 1678096 on Arch i686, vim 7.3.177. The maxmemtot is the same value. – Reid May 26 '11 at 18:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.