I use the following awk command ( in my bash script ) in order to remove duplicate lines from /etc/fstab file and then we copy the update fstab ( fstab.correct ) file to /etc/fstab

awk '/^#/ || "'!'"a[\$0]++' /etc/fstab > /etc/fstab.correct
cp /etc/fstab.correct /etc/fstab

since I want to prevent the copy - cp /etc/fstab.correct /etc/fstab

it will be more elegant we can use sed , or perl line liner to remove the duplicate lines insted of awk

I will appreciate to get suggestion how to remove duplicate lines from fstab file by sed or perl one line liner

  • Using sed seems impossible without sorting the file, which I presume you cannot consider since you are talking about /etc/fstab. Using perl, this one-liner should work well: perl -i -ne 'print if ! $x{$_}++' /etc/stab. – Giuseppe Clemente Jul 12 '20 at 9:05
  • ok please post your answer , seems its good solution – yael Jul 12 '20 at 9:11
  • if your awk has the inplace option, tha should write to the file. – Jetchisel Jul 12 '20 at 11:00

If sorting the file is not an option, it is impossible, as far as I know, to remove duplicate lines using sed.

With perl though, you can use this:

perl -i -ne 'print if ! $x{$_}++' /etc/stab

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.