Supposing I have a file named sample.txt which contains data in the following format:

1 John
2 Crema
3 Abraham
1 John

I can read the file line by line using the following way:

while read p
echo $p
done <sample.txt

Is there any way that I can compare a particular line with its following lines so that I can perform multiple task such as duplicate elimination or pattern matching, etc? What additional things will I have to add to accomplish the comparison task?

  • 1
    "duplicate elimination or pattern matching etc." Unix has many many good tools to do all those things and more. It would help if you were more specific about what you wanted to accomplish. – John1024 Sep 14 '15 at 20:32
  • I just want to apply the mechanism of comparing a line with all it's following ones manually @John1024 – Black Sep 14 '15 at 20:34
  • 1
    Unix has many many "mechanisms" for "comparing" lines. If you want truly useful answers, that is still too vague. – John1024 Sep 14 '15 at 21:19
while read p; do
    # compare p with prev somehow
done <sample.txt

This allows you to compare any two directly adjacent lines. If you want to compare lines further apart in the file, it becomes more complicated and may not be best written in shell (though the specific details will vary based on the task).

awk '{if (NR==1) STR=$0; if ($0==STR) print NR}' file

In this case I use line number 1 as basis for comparison. Modify NR==1 to whichever line number you want to compare to.

It will print all line numbers which match the selected line, including the selected line. Applied to your example it will print:


Once you have line numbers, you can eliminate them via e.g. sed:

sed -i '4d' file

Or perform other tasks on those lines (e.g. further awk with NR=line number)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.