Multiline Text Processing : Editing sshd_config

Question

I have multiple blocks at the end of my sshd_config files like theses:

Match User FOO
    ChrootDirectory /srv/www/FOO
    AllowTCPForwarding no
    X11Forwarding no
    ForceCommand internal-sftp

How could I achieve the deletion of this block (and not other regarding FOO1, FOO2, etc.) in a bash script?

Answer 1

Assuming all your Match blocks are at the end of the sshd_config file.
If your blocks are separated by empty lines, e.g.:

Match User FOO1
    PasswordAuthentication no

Match User FOO2
    PasswordAuthentication yes

Match User FOO
    ChrootDirectory /srv/www/FOO
    AllowTCPForwarding no
    X11Forwarding no
    ForceCommand internal-sftp

then just delete from the line Match User FOO up to (and including) the first empty line:

sed '/^Match User FOO$/,/^$/d' sshd_config

If they're not separated, e.g.:

Match User BAZ
    PasswordAuthentication yes
Match User FOO
    ChrootDirectory /srv/www/FOO
    AllowTCPForwarding no
    X11Forwarding no
    ForceCommand internal-sftp
Match User FOO1
    PasswordAuthentication no
Match User FOO2
    PasswordAuthentication yes

then delete from Match User FOO up to (but not including) the first line that starts with Match:

sed '/^Match User FOO$/,/^Match.*/{//!d;/^Match User FOO$/d;}' sshd_config

Note you'll have to use sed -i ... to edit the file in-place. Check your sed manual for details/backup options.

Answer 2

You can try this script using send and awk tools:

matchedLine=$(awk '/Match User FOO1/{ print NR; exit }' $1) 


if [ "$matchedLine" = "" ];then
echo "Match not found"
exit 1;
fi

sed -e "$matchedLine,$(awk "BEGIN {print $matchedLine + 5 }";)d;" $1

This script takes the filename of the ssh config file as its first argument. It then searches the file for the first occurrence of the match string (in this case 'Match User FOO1'). If it finds the match it will save its line number to the variable $matchedLine. Lastly using sed it will delete the line of the matched text and the following 5 lines. This script will not edit your actual file, it will only show you the content of the file with the matched lines removed. This way you can check if the script is working without any risk. If you want to edit the actual file just redirect the sed command to your file.

Answer 3

sed '/^Match User FOO$/,/^  *F.*-sftp$/c\ 
    The whole of the above line range is \
    replaced with this one block of text.\
    Use an empty "c"hange command or a   \
    "d"elete command to replace it with  \
    nothing at all.' <infile >outfile

It looks like you can rely on indents, by the way. So probably you can do:

sed '$!N;/^Match User FOO\n/,/\n[[:upper:]]/!P;D
'    <infile >outfile

Which will start deleting current lines while watching the next input line when the Match block is encountered, and stop deleting when the next input line begins with an upper-case letter.

The above command might need better explanation. It is the easiest and most robust method of doing this of which I am aware.

The workflow is like this:

1. For every input line which is !not the $last, also append the Next input line to pattern space.

   • While sed is typically known only to work a single line at a time this is merely its default behavior - sed's editing stream is as customizable as you might wish it to be.
   • In this case the Next command introduces a one-line look-ahead which is sustained - with the help of two other commands - throughout the course of the script.

2. Then check if the current context falls within that defined by the /2/,/address/ range of lines specified.

   • /^Match User FOO\n/ - The range starts when the matched pattern has already been in pattern-space for an entire cycle and would otherwise be Printed to standard out. This is signified by the head-of-pattern-space ^anchor and the trailing \newline in the pattern. This pattern represents an entire input line - from head to tail - but only half pattern-space at any one time.
   • /\n[[:upper:]]/ - This is mostly a suggestion based on my reading of both your example's apparent indent-style and of man sshd_config which would indicate that all relevant command primitives begin with an upper-case letter.
   • That, combined with the leading \newline escape in the pattern instruct sed only to end the range's match context when a line pulled in with Next begins with one - and not with any white-space.

3. If it does !not fall within that range then Print up to the first \newline character in pattern space to standard output, but if it does, print nothing at all for the current cycle.

4. Last, Delete up to the first occurring \newline in pattern space and end the current cycle.

   • This is very important - Delete does not pull in a new input line when the next cycle begins unless there are 0 \newlines in the current pattern space.
   • When there is a \newline in pattern space, Delete clears only up to and including it before recursing into a new cycle with what remains. What it Deletes, in fact, is exactly only as much as Print will write to standard output in a context which does not match your range.

Now all of that is a very technical way of saying what I said before - from the start of your matched range until its indentation level ends and a new command beginning with an upper-case letter is found at the head of a line nothing is printed, while everything else is printed. And so your block is deleted from output in a very simple way which is indent-dependent and which enables you to control its selection merely by indenting your Match blocks as you already do.

Now, to fully understand all of that, you have to understand sed's cycle, but it is also quite easily understood given an introspective look at how it works:

seq 10 | sed '$!N;/^4\n/,/\n9/!P;l;D'

The above command prints for two different reasons - the first (when it applies) is the !P range match context - which is what would go to stdout as suggested above.

The second is our look into how sed works - I tell it to print for every cycle an unequivocal representation of the current contents of pattern space. The results are these:

1             #This is Printed before we look at pattern space.
1\n2$         #This is our look - though only the 1st line is printed,
2             #each time one is, pattern space is actually 2 lines
2\n3$         #all of the time - it's our window into future output.
3
3\n4$         #3 prints, though 4 is in pattern space. But when pattern
4\n5$         #space matches ^4\n as it does now, nothing prints.
5\n6$
6\n7$         #And continues not to print...
7\n8$
8\n9$         #until the range ends here - when \n9 matches our future -
9             #now our current pattern space.
9\n10$        
10
10$

This makes controlling ranges - which some can find unintuitive because they require special contexts for start and end and extra matches for controlling those - much easier overall because you can start a range in the current context, and end it in a future one.

In other words, rather than saying...

/this_line/,/that_line/command - Apply command beginning from this_line up to and including that_line

...you can phrase it a little more intuitively like...

N;/this_line\n/,/\nthat_line/command;D - Apply command beginning from this_line until that_line.

It is as robust as you could like - as there is no means of putting a \newline into a sed pattern space except as a result of an edit command (such as Next) - and among the most performant solutions you might hope for.

So I'll borrow don's example data:

sed '$!N;/^Match User FOO\n/,/\n[[:upper:]]/!P;D' <<\IN
Match User BAZ
    PasswordAuthentication yes
Match User FOO
    ChrootDirectory /srv/www/FOO
    AllowTCPForwarding no
    X11Forwarding no
    ForceCommand internal-sftp
Match User FOO1
    PasswordAuthentication no
Match User FOO2
    PasswordAuthentication yes
IN

...and...

sed '$!N;/^Match User FOO\n/,/\n[[:upper:]]/!P;D' <<\IN
Match User FOO1
    PasswordAuthentication no

Match User FOO2
    PasswordAuthentication yes

Match User FOO
    ChrootDirectory /srv/www/FOO
    AllowTCPForwarding no
    X11Forwarding no
    ForceCommand internal-sftp

Match User FOO2
    PasswordAuthentication yes

IN

...will print...

Match User BAZ
    PasswordAuthentication yes
Match User FOO1
    PasswordAuthentication no
Match User FOO2
    PasswordAuthentication yes

...and...

Match User FOO1
    PasswordAuthentication no

Match User FOO2
    PasswordAuthentication yes

Match User FOO2
    PasswordAuthentication yes

...respectively.

It doesn't quit the range context on a blank line, though - all blank-lines following the block are regarded as a part of the block as written. So the blanks between the output lines in the second example are the ones which preceded it. It could, like:

sed '$!N;/^Match User FOO\n/,/\n\([[:upper:]].*\)*$/!P;D' <infile >oufile

...which would break the range context and begin writing to output again for either a blank line one begun with an upper-case case character. It could be made more general still:

sed '$!N;/^Match User FOO\n/,/\n\([^[:blank:]].*\)*$/!P;D' <infile >oufile

...which would break the range for any blank line or one opened with any non-blank character. In all cases though the breaking pattern is one on which you wish out to begin again, rather than a pattern for the last in a series for which you wish it to remain stopped.

Answer 4

This can be done using sed by using the N command:

sed -ri 'N;N;N;N;N;/^.+FOO\n.+/d' input.txt

What N command does is that it reads/appends the next line into your pattern space, hence making multi-line regex possible using sed.

For more, see this tutorial.

Answer 5

While sed is a possibility here, it's easier to make this robust in awk. The following script skips lines starting at Match User FOO (or any variation in case and whitespace) and stops skipping at any other Match User line, and prints all non-skipped lines.

awk '
    tolower($1)=="match" && tolower($2) == "user" {skip = $3 == "foo"}
    !skip
'

In case your script is interrupted, make sure to write the output to a temporary file, then move that into place. Note that this could still result in data loss if someone else modifies the file at the same time.

awk … </etc/sshd_config >/etc/sshd_config.tmp &&
mv /etc/sshd_config.tmp /etc/sshd_config