Remove lines that do not start with a pattern from a given set of patterns

Question

I have a file that contains data like this:

report aaaaaaaa  
-  ..  
-th bbbbbbbbb  
-to ccccccccc

.. --.

Question: I want to remove any line that does not start with following strings :

report  
-th  
-to

that means the desire output will remove all those middle unwanted dots and hashes and will look like this :

report aaaaaaaa  
-th bbbbbbbbb  
-to ccccccccc

sed/awk/grep/etc any solution that will work.

Answer 1

Using sed to modify the file in place:

sed -i '/^\(report\|-t\(h\|o\)\)/!d' your_file

This instructs sed to delete all lines not matching the pattern. The pattern itself is ^ (start of line), followed by either report or -t followed by either h or o.

You should note that this is not actual in-place modification: sed creates a temporary backup copy and overwrites the original file with it.

If you want sed to keep a backup copy of the original file (which might be a good idea if the file contains critical data), give the -i switch an extension to create a backup file:

sed -i'.bak' -e '/^\(report\|-t\(h\|o\)\)/!d' your_file

will modify your_file and create a backup of the original called your_file.bak.

A side note

Please don't misconstrue my intentions or take offence at this, but I've noticed that you have many similar regex/text-processing related questions. I advise you to start learning sed, awk and grep on your own to help speed up your productivity. Again, don't get me wrong, I'm all too happy to help (as are most people around here); it's just that I think that you stand to benefit hugely from picking up these tools for your daily use.

Just to prove how helpful people are around here, consider @slm's suggestion in the comments below and feel free to drop by this chatroom any time for questions.

Answer 2

You can use simple grep for this:

grep -E '^(report|-th|-to)' filename

or, shorter,

grep -E '^(report|-t[ho])' filename

Answer 3

Using awk:

awk '/^report|^-t[ho]/' file

Answer 4

Using sed:

sed -n -e '/^report\|^-th\|^-to/p' filename

Answer 5

Using Perl:

perl -ne 'print if /^report|^-t[ho]/' filename > newfile

or, to edit in place (like sed, perl will also make a temporary backup so this is not true in place editing):

perl -i.bak -ne 'print if /^report|^-t[ho]/' filename

That will make a copy of the original file called filename.bak and overwrite your original file with the edited version.

Answer 6

The questioner has made two points:

• wanting to remove any line not starting with "report" or "-th" or "-to".
• the desired output should remove "all those middle unwanted dots and hashes (sic)"

The solutions, at this time, address the first point and thereby also the second. But suppose the the file is larger and looks like:

report aaaaaaaa  
-  ..  
-th bbbbbbbbb  
-to ccccccccc
anything else
.. --.
-tp ddd
-tq eee
     -  -----

Wouldn't addressing OP's second point do the needful?

sed -r -i.bak '/^[ |.|-]*$/d' input-file 

does the job of removing presumably unwanted lines containing just spaces, dots and dashes and retaining the rest, whatever that is.
I would think that the risk of either approach is that the nature of the file isn't properly defined.

Answer 7

Using Raku (formerly known as Perl_6)

~$ raku -ne '.put if m/^ [ report | \-th | \-to ] /;' file.txt

Sample Input:

report aaaaaaaa  
-  ..  
-th bbbbbbbbb  
-to ccccccccc

.. --.

Sample Output:

report aaaaaaaa  
-th bbbbbbbbb  
-to ccccccccc

This is a similar answer to the Perl answer posted by @terdon. Raku's -ne command line flags autochomps the linewise input, but using put prints with an EOL terminator (remember put in Raku as an acronym for print-using-terminator). The call .put is short for $_.put, which outputs the $_ topic variable (i.e. the input line).

Otherwise, grouping regex atoms in Raku is accomplished by square brackets, and any non-alnum character must be escaped to be understood literally (otherwise Raku presumes it to have special Regex meaning, like . dot).

https://raku.org