How to filter out lines of a command output that occur in a text file?

Question

Let's say we have a text file of forbidden lines forbidden.txt. What is a short way to filter all lines of a command output that exist in the text file?

cat input.txt | exclude-forbidden-lines forbidden.txt | sort

Answer 1

Use grep like this:

$ grep -v -x -F -f forbidden.txt input.txt

That long list of options to grep means

• -v Invert the sense of the match, i.e. look for lines not matching.
• -x When matching a pattern, require that the pattern matches the whole line, i.e. not just anywhere on the line.
• -F When matching a pattern, treat it as a fixed string, i.e. not as a regular expression.
• -f Read patterns from the given file (forbidden.txt).

Then pipe that to sort or whatever you want to do with it.