I have a big CSV file.
I would like to view my file using
less or some command like it which doesn't have to read the whole file at once to show me part of it.
Is there a command out there which can show me my file in comma-aligned columns?
I'm not sure if that is enough for you, but you could make use of
You could wrap it up in some script to view different parts of the file at a time (probably without
| <m> shell-command <m> represents any mark letter. Pipes a section of the input file to the given shell command. The section of the file to be piped is between the first line on the current screen and the position marked by the letter. may also be ^ or $ to indi- cate beginning or end of file respectively. If is . or new- line, the current screen is piped. m Followed by any lowercase letter, marks the current position with that letter.
This isn't really a unixy question, but in any case, I'd recommend using something different from
for help. See also
for writing to a file etc.
EDIT: I happened to have a csv file which is 1.1G and is 934991 lines long.
This skips 500,000 lines and reads 5 lines. R takes 1 minute to return this, and tops out at 620M usage (!) It looks like it may be reading the skipped lines into memory, though that makes no sense.
The upshot: I'm not really an R fan, but for handling small to medium data sets, especially if you want to do statistical analyses on it, you could do worse than R. An alternative is Python and some csv processing library.