How can I remove all comments from a file?

Question

I have a file which includes comments:

foo
bar
stuff
#Do not show this...
morestuff
evenmorestuff#Or this

I want to print the file without including any of the comments:

foo
bar
stuff
morestuff
evenmorestuff

There are a lot of applications where this would be helpful. What is a good way to do it?

Answer 1

One way to remove all comments is to use grep with -o option:

grep -o '^[^#]*' file

where

• -o: prints only matched part of the line
• first ^: beginning of the line
• [^#]*: any character except # repeated zero or more times

Note that empty lines will be removed too, but lines with only spaces will stay.

Answer 2

I believe sed can do a much better job of this than grep. Something like this:

sed '/^[[:blank:]]*#/d;s/#.*//' your_file

Explanation

• sed will by default look at your file line by line and print each line after possibly applying the transformations in the quotes. (sed '' your_file will just print all the lines unchanged).
• Here we're giving sed two commands to perform on each line (they're separated by a semicolon).
• The first command says: /^[[:blank:]]*#/d. In English, that means if the line matches a hash at its beginning (preceded by any number of leading blanks), delete that line (it will not be printed).
• The second command is: s/#.*//. In English that is, substitute a hash mark followed by as many things as you can find (till the end of the line, that is) with nothing (nothing is the empty space between the final two //).
• In summary, this will run through your file deleting lines that consist entirely of comments and any lines left after that will have the comments stricken out of them.

Answer 3

As others have pointed out, sed and other text-based tools won't work well if any parts of a script look like comments but actually aren't. For example, you could find a # inside a string, or the rather common $# and ${#param}.

I wrote a shell formatter called shfmt, which has a feature to minify code. That includes removing comments, among other things:

$ cat foo.sh
echo $# # inline comment
# lone comment
echo '# this is not a comment'
[mvdan@carbon:12] [0] [/home/mvdan]
$ shfmt -mn foo.sh
echo $#
echo '# this is not a comment'

The parser and printer are Go packages, so if you'd like a custom solution, it should be fairly easy to write a 20-line Go program to remove comments in the exact way that you want.

Answer 4

Example input

cat example.sh

#!/bin/bash
# example script

echo "# test";# echo "# test"

# check the first parameter
if [ "$1" = "#" ]; then 
  # test couple of different cases
  echo "#"; # output # character 
  echo '\#'; # output # character '#' for test purpose
  echo \#\#\#; # comment # comment # comment '# comment'
  echo \#
  echo \#;
  echo \#; # comment
fi
# end of the script

Remove comments

sed -e '1{/^#!/ {p}}; /^[\t\ ]*#/d;/\.*#.*/ {/[\x22\x27].*#.*[\x22\x27]/ !{:regular_loop s/\(.*\)*[^\]#.*/\1/;t regular_loop}; /[\x22\x27].*#.*[\x22\x27]/ {:special_loop s/\([\x22\x27].*#.*[^\x22\x27]\)#.*/\1/;t special_loop}; /\\#/ {:second_special_loop s/\(.*\\#.*[^\]\)#.*/\1/;t second_special_loop}}' example.sh

Result

cat example.sh

#!/bin/bash

echo "# test";

if [ "$1" = "#" ]; then 
  echo "#"; 
  echo '\#'; 
  echo \#\#\#; 
  echo \#
  echo \#;
  echo \#;
fi

Read how it works at source: https://blog.sleeplessbeastie.eu/2012/11/07/how-to-remove-comments-from-a-shell-script/

Answer 5

You can achieve the required output using sed command. The below command had done the trick for me.

sed 's/#.*$//g' FileName

Where

• #.*$ - Regexp will filter all the string that starts with # up to the end of the line

Here we need to remove those lines so we replaced with empty so skipping 'replacement' part.

• g - mentioning repeated search of the pattern until end of file is reached.

General syntax of sed: s/regexp/replacement/flags FileName

Answer 6

Use a command like:

grep -E -v "^#|^$" <file-name>

• -v inverts the match, i.e. only outputs lines which do not match
• ^# matches lines starting with #
• ^$ matches empty lines

Answer 7

You can use invert match like this:

grep -v "#" filename

-v
Select lines not matching any of the specified patterns. (As specified by POSIX.)

Answer 8

cat YOUR_FILE | cut -d'#' -f1

It uses # as column separator and keeps just the first column (that is everything before #).

Answer 9

This worked for me

sed -i.old -E  "/^(#.*)$/d" file 

Answer 10

Following the 2nd answer of Joseph R., I add /^$/d to remove blank line.

sed '/^[[:blank:]]*#/d;s/#.*//;/^$/d'

Answer 11

I like Joseph R.'s answer but needed it to strip // comments too, so I modified it slightly & tested on RHEL:

# no comments alias
alias nocom="sed -E '/^[[:blank:]]*(\/\/|#)/d;s/#.*//' | strings"

# example
cat SomeFile | nocom | less

I bet there's a better way to remove blank lines than using strings but it was the quick & dirty solution I used.

Answer 12

None of the other answers seem to do this justice, they either leave in empty lines, or leave in lines where the comment isn't at the first character. I ended up using this:

cat << EOF >> ~/.bashrc
alias nocom='sed -e "/^\s*#/d" -e "/^\s*$/d"'
EOF

This sets up an alias, so that you don't have to memorize it (which is impossible to begin with). Open a new session, and you'll have the new nocom command. Then you can just

nocom /etc/foobar.conf

Cheers.

Answer 13

awk '!/^#/{gsub(/#.*/,"",$0);print}' filename

output

foo
bar
stuff
morestuff
evenmorestuff

Answer 14

The best solution would be to use the command:

sed -i.$(date +%F) '/^#/d;/^$/d' ntp.conf

The -i is the in-place edit but the prefix directly following tells sed to create a backup. In this case with a date extension (ntp.conf.date) We run two commands each with an address space, the first deletes the commented lines and the second, separated from the first by a semi-colon, deletes the blank lines.

I found this solution on : theurbanpenguin.com

Answer 15

My self modifying program (script file) strips the comments to cut the source down from over 500 kB to itself: half the size with just the programming instructions. Reason: to shorten the startup for 0.8 seconds. When the new source file is found, the stripdown takes 0.1 sec.

I have an area after the program with notes, so I mark it with #d- and that is the end of the program.

awk '{
    if ( $1 !~ /^#/ || $1 ~ /^#!/ )
    {
        if ( match($0,/[\t ]#/) ) { print substr($0, 0, RSTART) }
        else if ( NF ) { print }
    }
    else if ( /^#d-/ ) { exit }
}' source.sh > strip.sh

The above works flawlessly in a correctly written script (not for the question) as asdf="5"; echo "$asdf"#test demonstrates we need a space before the #, possibly some other markers apply, too. Limitation is a free character # in something like echo line or similar.

This solution leaves 154 characters # in my program which are used in the programming context, leaving even the ones in the above command (as it is a self modifying program, it parses the above lines, too).

if ( $1 !~ /^#/ || $1 ~ /^#!/ ) 

finds the lines not starting with # or #! (meaning: skip the lines starting with # or #!)...

if ( match($0,/[\t ]#/) )

... and check if they contain an inline comment: a # preceded by the tab or the space (here we could add markers).

{ print substr($0, 0, RSTART) }

when an inline comment is found, the above cuts the line dropping out the comment and writing out the line before the comment

else if ( NF ) { print }

simply prints out lines which are not empty (or: skip over empty lines)

else if ( /^#d-/ ) { exit }

can be omitted completely, it will finish stripping when the #d- is found at the beginning of a line.

The above is readable and complete version. It could be shortened if you want bare bones version to paste into the command line:

awk '/^[^#]/||/^#!/{if(match($0,/[\t ]#/))print substr($0,0,RSTART);else print}' source.sh > strip.sh

Answer 16

I'm posting what works for me and seems to make the most sense, after reading through the others, with explanation. A couple of posts came close, but I could not comment yet (because I'm a newb):

grep -E -v "(^#.*|^$)" filename

• -E = interpret the following pattern as a regular expression, similar to using egrep
• -v = print the inversion of the pattern (lines that do not match the expression will be printed)
• "(^#.*|^$)" = this has a pipe which designates an OR statement. This expression says to print any line that starts with a # (and anything else after it) OR any line with zero characters between the beginning and end of the line.

The -v will print on the screen the inversion of that, which will be any line with characters that does not start with a #.