43

I have a file which includes comments:

foo
bar
stuff
#Do not show this...
morestuff
evenmorestuff#Or this

I want to print the file without including any of the comments:

foo
bar
stuff
morestuff
evenmorestuff

There are a lot of applications where this would be helpful. What is a good way to do it?

13
  • 3
    you cannot remove parts of a line with grep. you can use sed for this
    – miracle173
    Commented Sep 24, 2014 at 20:12
  • 2
    Your text and your example contradict. You write about lines being commented out, but clearly from the last line you mean line parts. And then the first line with a comment is deleted including EOL, and second second might be, but it is not clear as that is the last line. Please rephrase 'lines commented out' to be exact and disambiguate your examples.
    – Anthon
    Commented Sep 25, 2014 at 3:12
  • 6
    try using awk -F\# '$1!="" { print $1 ;} ' .
    – Archemar
    Commented Sep 25, 2014 at 7:02
  • 3
    How would a line like echo '#' # output a # be handled?
    – Kusalananda
    Commented Jun 29, 2017 at 12:38
  • 4
    @Questionmark I might be clever, but I'm not writing-a-shell-grammar-parser clever.
    – Kusalananda
    Commented Jun 29, 2017 at 15:29

16 Answers 16

65

One way to remove all comments is to use grep with -o option:

grep -o '^[^#]*' file

where

  • -o: prints only matched part of the line
  • first ^: beginning of the line
  • [^#]*: any character except # repeated zero or more times

Note that empty lines will be removed too, but lines with only spaces will stay.

15
  • 8
    I would use grep -v '^#' file > newfilewithoutcomments Commented Jun 29, 2017 at 12:30
  • 6
    It should be noted this is NOT a general method for shell scripts, as for example the line somvar='I am a long complicated string ## with special characters' # and I am a comment will not be handled correctly.
    – Wildcard
    Commented Sep 6, 2017 at 21:45
  • 6
    How did this get 40 upvotes and become selected as the best answer??? It doesn't even handle the simple case print "#tag" # Print a hashtag.. Commented Oct 24, 2019 at 2:57
  • 3
    Like most answers, this will even kill the shebang!
    – Popup
    Commented May 11, 2020 at 15:06
  • 1
    in case you want to include the bash shebang like #!/bin/bash, use grep -v -P '^\s*#(?!!).*$' , here -v means exclude what grep match and -P means Perl regex which is more powerful.
    – Narutokk
    Commented Sep 1, 2022 at 15:37
48

I believe sed can do a much better job of this than grep. Something like this:

sed '/^[[:blank:]]*#/d;s/#.*//' your_file

Explanation

  • sed will by default look at your file line by line and print each line after possibly applying the transformations in the quotes. (sed '' your_file will just print all the lines unchanged).
  • Here we're giving sed two commands to perform on each line (they're separated by a semicolon).
  • The first command says: /^[[:blank:]]*#/d. In English, that means if the line matches a hash at its beginning (preceded by any number of leading blanks), delete that line (it will not be printed).
  • The second command is: s/#.*//. In English that is, substitute a hash mark followed by as many things as you can find (till the end of the line, that is) with nothing (nothing is the empty space between the final two //).
  • In summary, this will run through your file deleting lines that consist entirely of comments and any lines left after that will have the comments stricken out of them.
5
  • 8
    It will also delete anything found after a hash inside a string, no ? E.g. mystring="Hello I am a #hash" will become mystring="Hello I am a" Commented Mar 12, 2017 at 16:11
  • @javadba, yes, but at that point you might as well use a full parser. What's going to be using this data that can understand quotes and variable assignments but can't handle comments? (This is why many config files such as crontab only allow full-line comments, with or without leading whitespace, but do not allow trailing comments on a line. The logic is MUCH simpler. Use only the first of the two Sed instructions in this answer for a crontab comment stripper.)
    – Wildcard
    Commented Sep 6, 2017 at 22:01
  • great answer, this looks like a great balance of utility vs. complexity for a wide array of general use-cases, but in the case that you know ahead of time that you only need to delete lines starting directly with # (in column 1), is there any benefit to sed over grep -v "^#" ?
    – RBF06
    Commented Jan 15, 2019 at 16:36
  • 1
    Like most answers, this will even kill the shebang!
    – Popup
    Commented May 11, 2020 at 15:07
  • 4
    A small enhancement... sed '/^[[:blank:]]*#/d;s/[[:blank:]]*#.*//' your_file. Your original command leaves some trailing blanks in a line. My enhancement gets rid of those, too. Commented Nov 14, 2021 at 17:47
7

As others have pointed out, sed and other text-based tools won't work well if any parts of a script look like comments but actually aren't. For example, you could find a # inside a string, or the rather common $# and ${#param}.

I wrote a shell formatter called shfmt, which has a feature to minify code. That includes removing comments, among other things:

$ cat foo.sh
echo $# # inline comment
# lone comment
echo '# this is not a comment'
[mvdan@carbon:12] [0] [/home/mvdan]
$ shfmt -mn foo.sh
echo $#
echo '# this is not a comment'

The parser and printer are Go packages, so if you'd like a custom solution, it should be fairly easy to write a 20-line Go program to remove comments in the exact way that you want.

6

Example input

cat example.sh
#!/bin/bash
# example script

echo "# test";# echo "# test"

# check the first parameter
if [ "$1" = "#" ]; then 
  # test couple of different cases
  echo "#"; # output # character 
  echo '\#'; # output # character '#' for test purpose
  echo \#\#\#; # comment # comment # comment '# comment'
  echo \#
  echo \#;
  echo \#; # comment
fi
# end of the script

Remove comments

sed -e '1{/^#!/ {p}}; /^[\t\ ]*#/d;/\.*#.*/ {/[\x22\x27].*#.*[\x22\x27]/ !{:regular_loop s/\(.*\)*[^\]#.*/\1/;t regular_loop}; /[\x22\x27].*#.*[\x22\x27]/ {:special_loop s/\([\x22\x27].*#.*[^\x22\x27]\)#.*/\1/;t special_loop}; /\\#/ {:second_special_loop s/\(.*\\#.*[^\]\)#.*/\1/;t second_special_loop}}' example.sh

Result

cat example.sh
#!/bin/bash

echo "# test";

if [ "$1" = "#" ]; then 
  echo "#"; 
  echo '\#'; 
  echo \#\#\#; 
  echo \#
  echo \#;
  echo \#;
fi

Read how it works at source: https://blog.sleeplessbeastie.eu/2012/11/07/how-to-remove-comments-from-a-shell-script/

4
  • 1
    That's the way to do it!
    – Popup
    Commented Apr 21, 2020 at 6:52
  • That's by far the best solution so far, but I even this fails on multi-line strings with hashes in them. echo -e "this is \n#NOT# a comment" I have no idea of how to solve that, though.
    – Popup
    Commented Apr 21, 2020 at 11:37
  • sed: 1: "1{/^#!/ {p}}; /^[\t\ ]* ...": extra characters at the end of p command
    – Cœur
    Commented May 2, 2022 at 12:43
  • 1
    And actually, it's full of holes. It would cut echo "#\"#" as a comment.
    – Cœur
    Commented May 2, 2022 at 13:05
3

You can achieve the required output using sed command. The below command had done the trick for me.

sed 's/#.*$//g' FileName

Where

  • #.*$ - Regexp will filter all the string that starts with # up to the end of the line

Here we need to remove those lines so we replaced with empty so skipping 'replacement' part.

  • g - mentioning repeated search of the pattern until end of file is reached.

General syntax of sed: s/regexp/replacement/flags FileName

3
  • 2
    note: 4th line replaced with new line in this case. Commented Sep 26, 2014 at 7:23
  • 3
    Try that with a script containing that sed command...
    – Kusalananda
    Commented Jun 29, 2017 at 12:40
  • 4
    It won't handle print "#tag" # Print a hashtag. Commented Oct 24, 2019 at 3:01
2

Use a command like:

grep -E -v "^#|^$" <file-name>
  • -v inverts the match, i.e. only outputs lines which do not match
  • ^# matches lines starting with #
  • ^$ matches empty lines
0
2

You can use invert match like this:

grep -v "#" filename

-v
Select lines not matching any of the specified patterns. (As specified by POSIX.)

5
  • 2
    @alinh Thanks for reviewing the answer. Please note that the question required not only the beginning of the line but anywhere in the file. This is also showing in his/her expected result in the question above. My answer would be incorrect if I only look for beginning of the line.
    – Raza
    Commented Sep 25, 2014 at 14:19
  • zzz. my bad, didn't see the last line :(
    – alinh
    Commented Sep 25, 2014 at 14:33
  • 2
    This will completely remove the line starting with evenmorestuff in the OP's example.
    – Joseph R.
    Commented Sep 26, 2014 at 3:22
  • @JosephR. good catch. I missed that earlier. In this case grep -o '^[^#]*' file would be the best solution. this is already explained by jimmij. thanks for your review
    – Raza
    Commented Sep 26, 2014 at 17:52
  • 1
    It won't handle print "#tag" # Print a hashtag. Commented Oct 24, 2019 at 3:01
1
cat YOUR_FILE | cut -d'#' -f1

It uses # as column separator and keeps just the first column (that is everything before #).

2
  • 1
    If YOUR_FILE is a script containing those commands, the script would leave cat YOUR_FILE | cut -' in the file on that line.
    – Kusalananda
    Commented Jun 29, 2017 at 12:41
  • It doesn't even handle the simple case print "#tag" # Print a hashtag.. Commented Jan 3, 2020 at 18:51
1

This worked for me

sed -i.old -E  "/^(#.*)$/d" file 
1
  • 2
    It won't handle print "#tag" # Print a hashtag. Commented Oct 24, 2019 at 3:01
1

Following the 2nd answer of Joseph R., I add /^$/d to remove blank line.

sed '/^[[:blank:]]*#/d;s/#.*//;/^$/d'
1
  • This would screw your script if you're using something like: EXTENSION="${FILENAME##*.}" (it'll delete ##*.}) Commented Nov 8, 2020 at 8:41
1

I like Joseph R.'s answer but needed it to strip // comments too, so I modified it slightly & tested on RHEL:

# no comments alias
alias nocom="sed -E '/^[[:blank:]]*(\/\/|#)/d;s/#.*//' | strings"

# example
cat SomeFile | nocom | less

I bet there's a better way to remove blank lines than using strings but it was the quick & dirty solution I used.

1
  • 1
    It won't handle print "#tag" # Print a hashtag. Commented Oct 24, 2019 at 3:02
0

None of the other answers seem to do this justice, they either leave in empty lines, or leave in lines where the comment isn't at the first character. I ended up using this:

cat << EOF >> ~/.bashrc
alias nocom='sed -e "/^\s*#/d" -e "/^\s*$/d"'
EOF

This sets up an alias, so that you don't have to memorize it (which is impossible to begin with). Open a new session, and you'll have the new nocom command. Then you can just

nocom /etc/foobar.conf

Cheers.

1
  • 2
    It won't handle print "#tag" # Print a hashtag. Commented Oct 24, 2019 at 3:04
0
awk '!/^#/{gsub(/#.*/,"",$0);print}' filename

output

foo
bar
stuff
morestuff
evenmorestuff
0

The best solution would be to use the command:

sed -i.$(date +%F) '/^#/d;/^$/d' ntp.conf

The -i is the in-place edit but the prefix directly following tells sed to create a backup. In this case with a date extension (ntp.conf.date) We run two commands each with an address space, the first deletes the commented lines and the second, separated from the first by a semi-colon, deletes the blank lines.

I found this solution on : theurbanpenguin.com

0

My self modifying program (script file) strips the comments to cut the source down from over 500 kB to itself: half the size with just the programming instructions. Reason: to shorten the startup for 0.8 seconds. When the new source file is found, the stripdown takes 0.1 sec.

I have an area after the program with notes, so I mark it with #d- and that is the end of the program.

awk '{
    if ( $1 !~ /^#/ || $1 ~ /^#!/ )
    {
        if ( match($0,/[\t ]#/) ) { print substr($0, 0, RSTART) }
        else if ( NF ) { print }
    }
    else if ( /^#d-/ ) { exit }
}' source.sh > strip.sh

The above works flawlessly in a correctly written script (not for the question) as asdf="5"; echo "$asdf"#test demonstrates we need a space before the #, possibly some other markers apply, too. Limitation is a free character # in something like echo line or similar.

This solution leaves 154 characters # in my program which are used in the programming context, leaving even the ones in the above command (as it is a self modifying program, it parses the above lines, too).

if ( $1 !~ /^#/ || $1 ~ /^#!/ ) 

finds the lines not starting with # or #! (meaning: skip the lines starting with # or #!)...

if ( match($0,/[\t ]#/) )

... and check if they contain an inline comment: a # preceded by the tab or the space (here we could add markers).

{ print substr($0, 0, RSTART) }

when an inline comment is found, the above cuts the line dropping out the comment and writing out the line before the comment

else if ( NF ) { print }

simply prints out lines which are not empty (or: skip over empty lines)

else if ( /^#d-/ ) { exit }

can be omitted completely, it will finish stripping when the #d- is found at the beginning of a line.

The above is readable and complete version. It could be shortened if you want bare bones version to paste into the command line:

awk '/^[^#]/||/^#!/{if(match($0,/[\t ]#/))print substr($0,0,RSTART);else print}' source.sh > strip.sh
-1

I'm posting what works for me and seems to make the most sense, after reading through the others, with explanation. A couple of posts came close, but I could not comment yet (because I'm a newb):

grep -E -v "(^#.*|^$)" filename
  • -E = interpret the following pattern as a regular expression, similar to using egrep
  • -v = print the inversion of the pattern (lines that do not match the expression will be printed)
  • "(^#.*|^$)" = this has a pipe which designates an OR statement. This expression says to print any line that starts with a # (and anything else after it) OR any line with zero characters between the beginning and end of the line.

The -v will print on the screen the inversion of that, which will be any line with characters that does not start with a #.

5
  • It won't handle print "#tag" # Print a hashtag. Commented Oct 24, 2019 at 3:04
  • Ah, right... of course. Thanks for pointing that out. I was looking for an answer with regards to typical linux configuration files, such as pam.d configs, so I didn't think of that. I guess it would have to be adapted to find and remove any comments that lie on the same line as code. I just saw probably a better solution to my particular issue above: egrep -v "#|$^"
    – jackbmg
    Commented Oct 28, 2019 at 14:33
  • @RayButterworth, but why it should look for anything that doesn't starts with # , when its explicit that to look for anything that starts-with ^#, if you want to do that that then better to use egrep -v '^(#|"|$)' .
    – Karn Kumar
    Commented May 21, 2021 at 12:12
  • @KarnKumar, the question explicitly says that evenmorestuff#Or this should be stripped to evenmorestuff. Commented May 21, 2021 at 12:38
  • @RayButterworth, my bad just overlooked that :(, then sed 's/#.*$//;/^$/d' OR sed -n 's/#.*//;/^$/!p' this should work.
    – Karn Kumar
    Commented May 21, 2021 at 18:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .