Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I've two configuration files, the original from the package manager and a customized one modified by myself. I've added some comments to describe behavior.

How can I run diff on the configuration files, skipping the comments? A commented line is defined by:

  • optional leading whitespace (tabs and spaces)
  • hash sign (#)
  • anything other character

The (simplest) regular expression skipping the first requirement would be #.*. I tried the --ignore-matching-lines=RE (-I RE) option of GNU diff 3.0, but I couldn't get it working with that RE. I also tried .*#.* and .*\#.* without luck. Literally putting the line (Port 631) as RE does not match anything, neither does it help to put the RE between slashes.

As suggested in “diff” tool's flavor of regex seems lacking?, I tried grep -G:

grep -G '#.*' file

This seems to match the comments, but it does not work for diff -I '#.*' file1 file2.

So, how should this option be used? How can I make diff skip certain lines (in my case, comments)? Please do not suggest greping the file and comparing the temporary files.

share|improve this question
6  
The -I option causes a block to be ignored only if all its lines match the regexp. So you can ignore a comment-only change that way, but not the comment changes that are near a non-comment change. –  Gilles Jul 20 '11 at 14:22
    
@Gilles: Thanks, now I get it why diff -I does not behave as I expected. I updated my answer with an example which clarified this behavior for me. –  Lekensteyn Jul 20 '11 at 14:44

4 Answers 4

up vote 15 down vote accepted

According to Gilles, the -I option only ignores a line if nothing else inside that set matches except for the match of -I. I didn't fully get it until I tested it.

The Test

Three files are involved in my test (take the indented code, ignore File test*; I did it this way to prevent formatting making it less readable):

File test1:
    text
File test2:
    text
    #comment
File test3:
    changed text
    #comment

The commands:

$ # comparing files with comment-only changes
$ diff -u -I '#.*' file{1,2}
$ # comparing files with both comment and regular changes
$ diff -u -I '#.*' file{2,3}
--- file2       2011-07-20 16:38:59.717701430 +0200
+++ file3       2011-07-20 16:39:10.187701435 +0200
@@ -1,2 +1,2 @@
-text
+changed text
 #comment

The alternative way

Since there is no answer so far explaining how to use the -I option correctly, I'll provide an alternative which works in bash shells:

diff -u -B <(grep -vE '^\s*(#|$)' file1)  <(grep -vE '^\s*(#|$)' file2)
  • diff -u - unified diff
    • -B - ignore blank lines
  • <(command) - a bash feature called process substitution which opens a file descriptor for the command, this removes the need for a temporary file
  • grep - command for printing lines (not) matching a pattern
    • -v - show non-matching lines
    • E - use extended regular expressions
    • '^\s*(#|$)' - a regular expression matching comments and empty lines
    • ^ - match the beginning of a line
    • \s* - match whitespace (tabs and spaces) if any
    • (#|$) match a hash mark, or alternatively, the end of a line
share|improve this answer

I usually ignore this clutter by either:

  • Generating non-commented versions using grep -v "^#" | cat -s and diffing those or...
  • Using vim -d to look at the files. The syntax highlighting takes care of making comment vs. non-comment differences quite obvious. The diff highlighting of in-line difference so you can see what values or parts of values have been changed at a glance makes this my favorite.
share|improve this answer

Here is what I use to remove all commented lines -even those starting with a tab or space- and the blank ones:

egrep -v "^$|^[[:space:]]*#" /path/to/file

or you can do

sed -e '/^#.*/d' -e 's/#.*//g' | cat -s
share|improve this answer

After search around the web, alternative way of Lekensteyn is the better one I found.

But I want use dif output as patch... and there is a problem because line number are note keeped because of "grep -v".

So I purpose to improve this command line :

diff -u -B <(sed 's/^[[:blank:]]*#.*$/ /' file1)  <(sed 's/^[[:blank:]]*#.*$/ /' file2)

It's not perfect but line number are keeped in patch file.

However, if a new line is added instead of comment line... comment will be produce a Hunk FAILED when patching as we can see bellow.

File test1:
  text
  #comment
  other text
File test2:
  text
  new line here
  #comment changed
  other text changed

test now our command

$ echo -e "#!/usr/bin/sed -f\ns/^[[:blank:]]*#.*$/ /" > outcom.sed
$ echo "diff -u -B <(./outcom.sed \$1)  <(./outcom.sed \$2)" > mydiff.sh
$ chmod +x mydiff.sh outcom.sed
$ ./mydiff.sh file1 file2 > file.dif
$ cat file.dif
--- /dev/fd/63  2014-08-23 10:05:08.000000000 +0200
+++ /dev/fd/62  2014-08-23 10:05:08.000000000 +0200
@@ -1,2 +1,3 @@
 text
+new line

-other text
+other text changed

/dev/fd/62 & /dev/fd/63 are file produced by process substitution. Line between "+new line" and "-other text" is the default space character defined in our sed expression to replace comments.

And now, what is comming when we apply this patch :

$ patch -p0 file1 < file.dif 
patching file file1
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file file1.rej

Solution is to dont use unified diff format without -u

$ echo "diff -B <(./outcom.sed \$1)  <(./outcom.sed \$2)" > mydiff.sh
$ ./mydiff.sh file1 file2 > file.dif
$ cat file.dif
1a2
> new line
3c4
< other text
---
> other text changed
$ patch -p0 file1 < file.dif 
patching file file1
$ cat file1
text
new line
#comment
other text changed

now patch file working file (without warranty of result in very complex diff process).

share|improve this answer
    
Your unified diff fails to apply due to the context differences. You can use diff -U0 one two to disable the context. For patching, there are a bunch of tools that may be better suited such as kdiff3. –  Lekensteyn Aug 23 at 11:22
    
Thank you for -U0 option to disable context. Note : kdiff3 is a graphical tool. I need automatic tool for manage git merge attributes. –  syjust Aug 29 at 12:49
    
vimdiff supports three-way merges, might be worth looking at. –  Lekensteyn Aug 29 at 14:21
    
to be more precise, I need a script tool for automating the git merge process with exludes in an sql script. kdiff3 and vimdiff are interactives tools, not usable in my case. –  syjust Aug 29 at 14:37

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.