0

I'm new to programming (/scripting?), as this will make clear.

Purpose:

Trying to create a re-usable script automate the install/config of new instances of linux.... as I keep breaking them. In this case I'm mucking around with /etc/pam.d/sshd

And learning... sed is definitely making me learn about quotes, breaks and all the basics.

I've got it working as you can see but have a few questions (and I suspect someone will point out how fragile this method is)

Question 1-2)

Variable file - being passed as an argument to the sedinsert.sh

sedfind="# Standard Un\*x authentication."
#sedfind=".* Standard Un\*x authentication"
sedinsert="I am a \nmulti line variable \nand I kind of work"
sedfile=sshd

I'm trying to get this to do the heavy lifting i.e. I'll escape any characters needed in the sed command or use wildcards here as then I just have to change the variable file on updates once and it's, to me at least, a bit cleaner.

It also massively helps my understanding as I can easily see how changes affect the output/ results.

Q1. Am I right with my quotes and new lines here. It seems to work but is there another way? I get a horrible feeling I'll hit a character soon that will break this - any examples?

Q2. Is it possible to split variables onto multiple lines to make this file easier to work with, when necessary, (e.g. similar to using \ at the end of the line) - bearing in mind the sourcing and sed coming up.
Perhaps another command within the script to deal with this? (Grep? or another sed?.

Question 2

sedinsert.sh

set -x
source $1
sed -i "\|$sedfind|a\ \n$sedinsert" $sedfile

I'm trying to keep this as clean as possible and also to not use quotes around the variables. I've currently another one for replacing at the moment but I'll probably look at passing the command as a variable next

I'm not absolutely wedded to that idea but hopefully the reasons will make sense... i.e. This will be used to alter all kinds of files with 'difficult' characters in them e.g. @#" etc

Q. Because of this I'm having to put the first new line in the command itself (|a\ /n ....). Would there be someway to put this in the variable itself in case I don't want it/ want to include multiple blank lines?

Question 3

It's nice in sed that you can set the pattern delimiter (correct term?) from \ to | as I have here.

But every time I think I've found a good one to use I sure enough find that's included in the next file I look at

Q. Is there some suggested/ standard obscure delimiter to use that's rarely used elsewhere - so I don't keep finding it in a file e.g. some russian character (apologies to anyone Russian) rather than #|, etc.

I'm just a little confused why everyone uses the common ones, which seem to cause people all kinds of confusion ( e.g. / in file paths) going by other answers

Question 4: Now how should I really be going about all this?

Few lines of pam.sshd as an example

# PAM configuration for the Secure Shell service

# Standard Un*x authentication.
@include common-auth

# Disallow non-root logins when /etc/nologin exists.
account    required     pam_nologin.so

Sorry for the length - thought it worth explaining the whole process

1

Since sedinsert.sh contain the source command I assume you use bash.

Question 1-2)

Q1.

You are right.

Q2.

Actually one can have a variable with newline symbol:

sedinsert="I am a 
multi line variable 
and I kind of work"

Sourcing will be OK as far as variable file is a valid bash script.

For your purposes, you can have this variable (with additional backslashes preceding every newline):

sedinsert="I am a \
multi line variable \
and I kind of work"

Backslashes are necessary for using with sed. Now you can use your command without changes:

sed -i "\|$sedfind|a\ \n$sedinsert" $sedfile

Question 2

The sed command a \ is supposed to be used with newline as here:

sed -i "\|$sedfind|a \
$sedinsert" $sedfile

For example, the command

sed "/ping/a \
pong" -i file.txt

Will add a string pong after each string that contains substring "ping" (without extra newlines and spaces).

If you don't like how it looks, you can use $'word' mechanism in order to represent newline:

sed "/ping/a \"$'\n'"pong" -i file.txt

or

newline=$'\n'
sed "/ping/a \${newline}pong" -i file.txt

Question 3

Firstly, every symbol goes as far as pattern (e. g. value of sedfind) doesn't contain it. The content of file $sedfile doesn't matter. Delimiter is only used for indication of end of pattern. Secondly, if it is necessary to use delimiter character in the pattern, one can escape all its occurrences with backslash:

sed "\|bar\|bar| a \
foo-foo"

will add a string foo-foo after each string that contains bar|bar

Is there some suggested/ standard obscure delimiter to use that's rarely used elsewhere?

I don't know nothing about this.

By the way, my version of sed doesn't accept russian characters as delimiter (error message delimiter character is not a single-byte character).

Summary

Now you can use this variable file:

sedfind="# Standard Un\*x authentication."  # all occurrences of symbol '|' must be escaped by backslash as far as '|' is sed pattern delimiter
#sedfind=".* Standard Un\*x authentication"
sedinsert="I am a \
multi line variable \
and I kind of work"
sedfile=sshd

And this sedinsert.sh:

set -x
source $1
sed -i "\|$sedfind|a \
$sedinsert" $sedfile

It is better to surround variables with double quotes (otherwise script will crash on filenames with special symbols):

set -x
source "$1"
sed -i "\|$sedfind|a \
$sedinsert" "$sedfile"

This script will add the string(s) $sedinsert after each line of file $sedfile that match pattern $sedfind.


Good tutorial about quoting, regular expressions and sed scripting, one may find it useful: http://www.grymoire.com/Unix/

  • Really appreciate you taking the time to look through that. I could have made it a lot easier to read so thanks. For the tutorial as well. – beardedfool Aug 28 '17 at 0:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.