Basically, I had a file, which contained around 90 usernames.

I had to delete all of those usernames from the passwd file, which was a total of 300 lines long.

I tried to come up with a way to at least filter the duplicate usernames and print out the ones, which have to remain but had no sucess.

Let's say that the file, where I had the 90 usernames for removal contains:



The passwd file contains these usernames, along with a lot more



The usernames inside the passwd file were randomly spread inside the file, so diff -y wouldn't have done the job.

My goal here was to compare the two file, for example cat the file.txt and use the output to search inside the passwd. The result should be either a removal of the duplicate lines, or a printing of the unique lines.

marked as duplicate by αғsнιη, don_crissti, Rui F Ribeiro, G-Man, cas Jan 18 '18 at 7:58

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 2
    Please provide example input and expected output, and at least show what you have tried and explain how it did not work as expected or intended. – DopeGhoti Jan 17 '18 at 19:27

The comm command can be used to compare and contrast two sorted files:

comm <(sort file.txt) <(cut -d: -f1 /etc/passwd | sort)
  • The first column contains lines from the first file that do not appear in the second file.
  • The second column contains lines from the second file that do not appear in the first file.
  • The third column contains lines that appear in both files.

You can omit one or two columns from the three-column output (comm -13 ... will omit columns 1 and 3, for example).

Not asked in your question, but requested in a follow-up command, is how to take a list of usernames and extract the corresponding entries from /etc/passwd:

( echo root; echo sys ) | sed -r 's!(.*)!^\1:!' | grep -f - /etc/passwd
  • That did the job in separating the users that have to remain in the passwd file! Thank you! However, considering that the passwd file does not contain raw usernames, but user:test:123:123 for example. How can I make sure that these remain in the output of the command. Match the duplicates by the usernames only, but still return the entire lines as output. Can this be achieved in any way? – George.S Jan 19 '18 at 9:38
  • @GeorgeS this is another straightforward pipeline, but I've updated my question with an illustration for you. – roaima Jan 19 '18 at 9:59

I think the easiest way may be to do :

grep -v -x -f file_infotodelete reference_file  > result_file  

Use below awk one liner to achieve.

Below command will delete file.txt content from file passwd. Tested it worked fine

awk 'NR==FNR{a[$1];next}!($1 in a){print $1}' file.txt passwd

Not the answer you're looking for? Browse other questions tagged or ask your own question.