Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have to insert a line into a configuration files automatically, but with the added caveat that it should be inserted before a multi-line footer comment and any preceding empty or whitespace-only lines if the footer exists. That is, the new line should be inserted right after the last configuration line, making for a single-line diff from the original file. In pseudo-code:

  1. Go to the end of the file.
  2. Go backwards to the first (that is, last in the file) configuration line (that is, a line which is not empty, whitespace-only, comment only or whitespace followed by a comment).
  3. Insert text after the current line.

Extended regular expression for configuration line: ^\s*[^[:space:]#]

Any common *nix tools such as sed, awk, ed or ex should work.


Possible solutions and their problems:

  • Use tac twice to make this into a forward-searching problem rather than backward-searching. This means I'll have to store the result in a temporary file then replace the original, rather than doing this in a single command.
  • Use sed -i with the reversal trick. This means storing the entire file in memory.
  • ex -c '1' -c '?^\s*[^[:space:]#]?' -c $'a\nmy new line\n.' -c 'wq' /path, which I'm also given to understand stores the full file in memory.

Is there a solution which circumvents both of these issues?

Example starting file:

# Universe configuration
#

pi = 3 # A good #
e = mc**2 # To within a hair

[cut 200 trillion lines]

# 
# END
#

Example input:

sol { mass = 42, start = 9.2 }

Expected output:

# Universe configuration
#

pi = 3 # A good #
e = mc**2 # To within a hair

[cut 200 trillion lines]
sol { mass = 42, start = 9.2 }

# 
# END
#

General-purpose function based on @StephaneChazelas's solution

share|improve this question
1  
^\s*[^#] matches "<space>#", you want ^\s*[^\s#] here. –  Stéphane Chazelas Jun 7 '13 at 9:45

3 Answers 3

up vote 2 down vote accepted

You could do something along these lines:

file=/some/file
newtext='sol { mass = 42, start = 9.2 }'
tac -- "$file" | 
  NEWTEXT=$newtext awk -v size="$(wc -c < "$file")" '
    $1 ~ /^[^#]/ {
      system("dd bs=1 seek=" size - length(footer) " conv=notrunc if=/dev/null")
      printf "%s\n%s", ENVIRON["NEWTEXT"], footer
      exit
    }
    {footer=$0 "\n" footer}' 1<> "$file"

That overwrites the file in place and only stores the footer in memory. It needs the non-standard GNU tac command. The file has to be a regular text file.

share|improve this answer
    
Wow. My single ex command is almost three times slower than this, even on a 267 byte file. –  l0b0 Jun 7 '13 at 12:27
    
I think ex is not the solution, it must read the whole file before execute command. –  cuonglm Jun 7 '13 at 12:29
    
@l0b0, for such a short file, the difference is only going to be due to the loading time of the utilities. Note that ex will need to read the whole file, but not store the whole content in memory (though it will still need to store a fair amount of information in memory). YMMV depending on the ex implementation. –  Stéphane Chazelas Jun 7 '13 at 12:39
    
Sure, I was just surprised that the startup time of ex (even when reading the file) was much more than the combined startup time of tac, awk and dd plus a subshell. –  l0b0 Jun 7 '13 at 13:04
1  
PS: Works with GNU awk 3.1.4, but not with mawk 1.2. No output, and no changes to the file. –  l0b0 Jun 7 '13 at 13:27

awk only solution (tested with gawk):

$ awk '
    BEGIN   {   footer = ""; wl = ""; } 
    END     {   while(( getline line < "sol.txt") > 0 ) {
                    print(line)
                }
                footer = wl footer
                print substr(footer, 0, length(footer)-1); 
            }

    # Blank line
    /^[[:blank:]]*$/ {
                if (footer) {
                    footer = wl footer
                    print substr(footer, 0, length(footer)-1); 
                    footer = ""
                }
                wl = $0 "\n"
            }
    # Comment only
    /^[[:blank:]]*#/ {
                footer = footer $0 "\n"; 
            }

    # Configuration line
    /^[[:blank:]]*[^[:space:]#]/ {
                print(wl footer $0); wl = ""; footer = ""; 
            }
    ' <  universe.txt > universe2.txt

Producing:

# Universe configuration
#

pi = 3 # A good #
e = mc**2 # To within a hair

[cut 200 trillion lines]
sol { mass = 42, start = 9.2 }

# 
# END
# 

With one line diff:

sh$ diff universe*.txt
8d7
< sol { mass = 42, start = 9.2 }
share|improve this answer
    
This seems to simply place the text just before the last comment block, which is not the requested feature. For example, what if there are several comment blocks separated by empty lines? This also doesn't seem to handle whitespace before comment characters. –  l0b0 Jun 7 '13 at 19:49
    
I have difficulties to understand the specification of the footer. Sorry about that. If a footer is "a sequence of zero or more white lines followed by one or more comment lines", I've modified my answer to better match your specs. But maybe a footer could be made of several comments intermixed with several blank lines? –  Sylvain Leroux Jun 8 '13 at 14:33

This is a configuration file, so it fits comfortably in memory. There is no need to process it as a stream.

newline='
' tab=$(echo | tr '\n' '\t')
old=$(cat foo.config)
footer=${old##*"$newline[!$newline$tab #]"}
if [ "$footer" = "$old" ]; then
  footer=
else
  footer=${footer#*"$newline"}
fi
head=${old%"$footer"}
echo "$head$text_to_insert$footer" >foo.config.new
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.