Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a CSV that is 40 000 lines long. What is the simplest tool to read from line x to line y?

I am sure there are more professional ways to do it but I would just like a bash script that works in the form of

 readFile --from 10 --to 20

so that I can process ten lines, then call it again and bring in ten more lines. I was considering just using an AWK script but can you do it even more simply with something like cat, grep, tail or head?

share|improve this question
up vote 2 down vote accepted

while getopts "f:t:" option; do
  case "$option" in 
    f) from=$OPTARG ;;
    t) to=$OPTARG ;;
if [ -z "$from" ] || [ -z "$to" ]; then
  echo "must give both -f and -t" >&2
  exit 1

awk -v "from=$from" -v "to=$to" 'from <= NR && NR <= to' filename

# or:
sed -n "$from,$to p; $to q" filename

# or:    
while read line; do
  i=$(( i + 1 ))
  if [ $i -ge $from ] && [ $i -le $to ]; then
    printf "%s\n" "$line"
  [ $i -eq $to ] && break
done < filename
share|improve this answer
Thank you that was awesome I hope I can at least get to the point where I am that good at the simple things in life. That last one is completely beyond me at the moment. – Prospero Sep 2 '11 at 12:42
I should note that this takes as arguments -f and -t instead of the requested --from and --to – glenn jackman Sep 2 '11 at 14:59
It seems this script is doing the same thing three times :) – Bernhard Jan 1 '14 at 10:38
and you get to pick one of them. That's what "or" means – glenn jackman Jan 1 '14 at 13:17

Extracting lines 10 to 20 of a file is a simple combination of head and tail:

tail -n +11 | head -n 10 | mycommand

This command skips 10 lines and processes the next 10. If you want to process all the lines in the file in sequence, but by groups of 10, there are better ways. You can read 10 lines with head repeatedly. Note the use of chunk=$(…; echo a) followed by stripping away the a, to work around the suppression of final newlines in command substitutions. This approach has the advantage of working even if the input is a pipe (which you can't rewind), and it's faster for a regular file too. Warning, untested code, typed directly in the browser.

while chunk=$(head -n 10; echo a); chunk=${chunk#a}; [ -n "$chunk" ]; do
  printf %s "$chunk" | mycommand
done <filename

Alternatively, you can make awk do the job. Again, untested.

awk '
    {chunk = chunk $0 RS}
    NR % 10 {print chunk | "mycommand"; close("mycommand"); chunk="" }
    END {if (chunk != "") {print chunk | "mycommand"; close("mycommand"); chunk="" }}
' <filename
share|improve this answer
I love the magic of combining different utilities ty. – Prospero Sep 3 '11 at 0:26

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.