2
#!/bin/bash
search_string="\/sbin\/iptables -A INPUT -p tcp --dport 12443 -j ACCEPT";
delimeters=$(cat /root/firewall/firewall.txt);
sed -i "s/$search_string/$delimeters$search_string/" /root/result.txt

I want to add the contents of the /root/firewall/firewall.txt into /root/result.txt file before a line which is saved in search_string variable.

If /root/firewall/firewall.txt contains one line above script works. But if the firewall.txt contains multiple lines, script breaks as:

sed: -e expression #1, char 64: unterminated `s' command

I think, new line characters causing the problem but I could not properly backslash it.

search_string="\/sbin\/iptables -A INPUT -p tcp --dport 12443 -j ACCEPT";
delimeters=$(cat /root/firewall/firewall.txt);
replaced= "$delimeters" | sed -r 's/\\n/\\\\n/g'
sed -i "s/$search_string/$replaced$search_string/" /root/result.txt 

How can I fix this issue?

  • what is the output you expect? – Fiisch May 14 '15 at 17:59
  • It should add the multiple line contents of firewall.txt into result.txt before /sbin/iptables -A INPUT -p tcp --dport 12443 -j ACCEPT line. – NecNecco May 14 '15 at 18:01
  • and the linebreaks should be still there the same way as in the firewall.txt? – Fiisch May 14 '15 at 18:03
  • yes, I just backslashed because the first script did not work. – NecNecco May 14 '15 at 18:08
  • you sure this is about linebreaks? maybe some other character in the firewall.txt could cause this? – Fiisch May 14 '15 at 18:14
0

(This overlaps somewhat with some of the other answers.)

  1. I’m somewhat confused.  You say,

    I want to add the contents of the firewall.txt file into the result.txt file before a line which is saved in search_string variable.

    OK, first of all, if they aren’t an essential part of the question, full pathnames (/root/…) clutter your question and add no value.  Just use simple filenames.  After all, I hope you’re debugging this in a local directory, not running as root.

    Secondly, what is useful is giving some example data.  Not the 2000 lines that you actually have in your actual files, but a handful of lines.  (Again: you are debugging this on test files, aren’t you?)  For example, say result.txt contains

    One, two,
    /sbin/iptables -A INPUT -p tcp --dport 12443 -j ACCEPT
    Buckle my shoe.
    

    and firewall.txt contains

    Humpty Dumpty sat on a wall.
    

    You say,

    If firewall.txt contains [only] one line, the script

    #!/bin/bash
    search_string="\/sbin\/iptables -A INPUT -p tcp --dport 12443 -j ACCEPT";
    delimeters=$(cat /root/firewall/firewall.txt);
    sed -i "s/$search_string/$delimeters$search_string/" /root/result.txt
    

    works.

    (By the way, you don’t need the semicolons at the ends of the lines, and the word “delimiter” is spelled with the word “limit” in the middle.)  Well, the above produces this result:

    One, two,
    Humpty Dumpty sat on a wall./sbin/iptables -A INPUT -p tcp --dport 12443 -j ACCEPT
    Buckle my shoe.
    

    Is that really what you want?  Because that’s not what most people think when you say, “add [text] into [a] file before a line”, especially when you start talking about the text to be inserted being more than one line, and especially since you said the linebreaks should be still there the same way as in the firewall.txt.  I’ll assume that you really want

    One, two,
    Humpty Dumpty sat on a wall.
    /sbin/iptables -A INPUT -p tcp --dport 12443 -j ACCEPT
    Buckle my shoe.
    

    If you want the last line of the firewall.txt file concatenated with the /sbin/iptables line, please explain more precisely what you want.

  2. You say,

    search_string="\/sbin\/iptables -A INPUT -p tcp --dport 12443 -j ACCEPT";
    delimeters=$(cat /root/firewall/firewall.txt);
    replaced= "$delimeters" | sed -r 's/\\n/\\\\n/g'
    sed -i "s/$search_string/$replaced$search_string/" /root/result.txt
    

    Well, that’s nonsense; the third line responds

    -bash: Humpty Dumpty sat on a wall.: command not found
    

    Perhaps you meant

    replaced=$(echo "$delimeters" | sed -r 's/\\n/\\\\n/g')

    ?

  3. OK, even if you had said

    replaced=$(echo "$delimeters" | sed -r 's/\\n/\\\\n/g')
    

    it wouldn’t have done any good, because sed normally works a line at a time, and it isn’t going to see newlines as characters in lines (even if its input is coming from a shell variable that has a multi-line value; that’s no different from a file with multiple lines).  What does work (eliminating a useless use of cat) is

    replaced=$(sed 's/$/\\/' firewall.txt)
    sed "s/$search_string/$replaced
    $search_string/" result.txt
    

    using sed 's/$/\\/' to add a backslash at the end of every line in firewall.txt.  You need to type a newline (Enter) after $replaced because the last newline gets stripped off when you do the replaced=$(…) command substitution.  And, just for simplicity’s sake, if you want to leave the /sbin/iptables command untouched, you might want to consider changing the final command to

    sed "s/$search_string/$replaced
    &/" result.txt
    

    using & in the replacement string to say “insert here the text that was found by the search string (regex)”.

  4. While the s command will allow you to insert entire lines, it wasn’t really meant for that.  There are a, i, and c commands for inserting one or more entire lines from the command string.  But, since your insertion text is coming from a file, it makes the most sense to look at the r (read) command.  As a first cut,

    sed "/$search_string/r firewall.txt" result.txt
    

    will do almost what you want.  Almost.  Unfortunately, it will inject the contents of the firewall.txt file after the /sbin/iptables line.  I was able to find a workaround (to get the contents of firewall.txt before the /sbin/iptables line), but it’s grievously complicated:

    sed -n -e "/$search_string/{s/^/+/; h; r firewall.txt
    n}" -e 'x; s/^+//p; s/.*//; x; p' result.txt
    

    Apparently sed won’t recognize ; as a delimiter to end a filename, so, when we say r firewall.txt, we must type Enter.

    Here we go:

    • -n: Don’t write output except as commanded by p or r commands.
    • /$search_string/{…}: For each line matching $search_string (/sbin/iptables …), do the following:
      • s/^/+/: Insert a + at the beginning of the line (creating +/sbin/iptables …).  This flags the line as being a match for the search string.  (I’ll get back to that.)
      • h: Copy the pattern space (+/sbin/iptables …) to the hold space.
      • r firewall.txt: Read the firewall.txt file (and write its contents).
      • n: Stop processing this line and read the next one.
    • Then, for every other line (those not matching $search_string), do the following:
      • x: Exchange the contents of the hold and pattern spaces.  I.e., the line we just read (the one not matching $search_string) goes into the hold space, and we copy into the pattern space the previously held line (which might be +/sbin/iptables … and might be blank).
      • s/^+//p: If the line is a saved match of the search string (i.e., it is a flagged line containing +/sbin/iptables …), strip off the + and print the rest.  Otherwise, print nothing.
      • s/.*//: Wipe out the line (replace everything with nothing).  I would have liked to do d (delete) here, but that terminates processing of the current line.
      • x: Exchange the contents of the hold and pattern spaces again.  Move the blank line from the pattern space into the hold space, and retrieve the line from result.txt that we just stashed there.  And finally …
      • p: Print the line from result.txt.

    In short,

    • When we find a line matching $search_string (i.e., /sbin/iptables …), we save it in the hold space (without printing it), and read (and print) the firewall.txt file.
    • For every other (i.e., non-matching) line, we pull the saved line (if any) out of the hold space and print it, and then print the current line.

    Argh!  This fails if /sbin/iptables … occurs on the last line, because it gets saved in the hold space, but there’s no subsequent non-matching line to trigger its extraction.

    So let’s just make sure that /sbin/iptables … never occurs on the last line, by adding a dummy line at the end, and then strategically removing it.

    echo >> result.txt
    sed -n -e "/$search_string/{s/^/+/; h; r firewall.txt
    n}" -e 'x; s/^+//p; $d; s/.*//; x; p' result.txt
    

    The $d causes the last line to be deleted.  (We could use $q and get the same effect.)

    This does work if there are multiple iptables lines.  But, yes, it is getting to be something of a kludge.  I guess the sed "s/$search_string/$replacednewline&/" answer isn’t looking so bad now.

  • Thank you very much @G-Man for your detailed answer. Yes, code had some flaws. I just could not figured out why new lines were eaten by sed. I could fix the new line issue by this answer. Your 3rd method also doing the job. It is good to learn sed is designed to work with line by line. – NecNecco May 15 '15 at 17:18
  • Thanks for having the patience to read a long, late answer.  I'm glad you found it helpful. – G-Man May 15 '15 at 18:24
0

While this is likely possible with some sed calisthenics, I would use another tool instead. Perl, for example:

search_string="\/sbin\/iptables -A INPUT -p tcp --dport 12443 -j ACCEPT";
delimeters=$(cat /root/firewall/firewall.txt);
perl -i -pe "s/$search_string/$delimeters$search_string/" /root/result.txt
  • your method works @terdon. But I would like to handle by sed if possible. – NecNecco May 14 '15 at 18:31
0

sed has more commands than just s/// -- there's an r command to read another file into the current stream. However that reads the file after the current line, and you want to place it before the current line, so we have to juggle a bit. Note that you do not have to escape the slashes in the search string, we can use alternate delimiters in sed.

$ cat result.txt
foo
bar
/sbin/iptables -A INPUT -p tcp --dport 12443 -j ACCEPT
baz

$ cat firewall.txt
firewall line 1
firewall line 2

$ search_string="/sbin/iptables -A INPUT -p tcp --dport 12443 -j ACCEPT"

$ sed "\%${search_string}% {
                     # we have hit the search string:
    x                # move the current line to hold space
    r firewall.txt   # queue the file for reading
    N                # append the next line from results
                     # -> this triggers the actual file read and printing
    s/\n//           # the "x" command left behind a blank line so we remove it
    H                # append the current line to the hold space   
    g                # then bring the hold space to the pattern space
}" result.txt                
foo
bar
firewall line 1
firewall line 2
/sbin/iptables -A INPUT -p tcp --dport 12443 -j ACCEPT
baz

The sed command can be shrunk to

sed "\#${search_string}#{x;r firewall.txt
N;s/\n//;H;g}" result.txt 

GNU sed seems to require a newline after the filename.

  • 1
    This doesn't seem to work if several lines match ${search_string}. Also, the real problem is when the last line matches (as you can't pull the Next line)... Definitely not a job for sed, unless you have gnu sed: sed "\|${search_string}|e cat firewall.txt" result.txt should work irrespective of the number of lines matching pattern and their position... – don_crissti May 14 '15 at 23:57
  • Glenn-jackman it did not work. I am using CentOS terminal, though. Thanks anyway. @don-crissti your method works. Thank you very much for it. – NecNecco May 15 '15 at 17:21
0
sed '/\-A INPUT -s 192.168.0.0\/24 -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT/a CLIENTSCRIPT2="hello.sh"' file.txt

In this case i added the variable CLIENTSCRIPT2="hello.sh"' on file.txt just under -A INPUT...etc Note the backslash escape in the syntax

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.