I have a YAML file which includes a stanza looks this:

    password: '$6$oTQhLvN/4VFJPscD$8LYwUMSFi'
    password: '$6$JKOtLF0wHeZfIskt$W/M5.ugDS'

If the shell variable $ACCOUNT contains the account name (alice, bob, etc), and $YAML is the filename of the YAML file, I can add a new entry with sed, like this. It inserts the new account directly under the "admin::common::passwords:" line, so I know it's in the correct stanza:

  sed -i /'^admin::common::passwords:'/a"\  ${ACCOUNT}:\n    password: '${ENCRYPTED_PASS}'" $YAML

I can find an existing account like this with awk. Again, this finds the stanza, then finds the account within that stanza (it's probably possible to do this all in awk, but I'm not good enough at awk, and this will do the job):

  awk "BEGIN{RS=ORS="\n\n";FS=OFS="\n"}/admin::common::passwords:/" $YAML | grep -A1 "^[ ]*${ACCOUNT}:"

But to delete a line, the best I can come up with is this:

sed -i "/^  ${ACCOUNT}:/,+1d" $YAML

Which would delete any line "^ alice:" anywhere in the file, whether it's in the admin::common::passwords stanza or elsewhere.

I can't make any assumptions about the content of the file, except that it does contain exactly one admin::common:passwords stanza (if it did not, then it would have been created by another part of my script; that bit is easy).

So, my question is this: How can I improve this sed to say 'Search for "^ ${ACCOUNT}:" within the admin::common::passwords stanza, and delete that matching line as well as the line below it'?

(it doesn't have to be sed, of course; it may be that awk or even Perl are better suited to the task)

  • 1
    It would help if you clarified how sections are delimited in your file's format. Commented Nov 10, 2015 at 13:47

2 Answers 2

sed "
  /^admin::common::passwords:$/,/^[^ ].*:$/ {
     /^  $ACCOUNT:$/ {
  }" < "$YAML"

That is, enclose your deleting stanza in a section that's matched between admin::common... and the next whatever-section:.

Beware that the . character, common in user names is also a regular expression operator, so john.doe would match john.doe but also johnWdoe for instance.

Note that the above won't work if you have two consecutive admin::common::passwords sections and the account to delete is in the second one.

If, as Otheus indicates, sections run until the next line that has the same or less amount of leading space characters and your admin::common::passwords: section may have some leading spaces, then it's probably time to switch to another language like awk:

awk -v account="$ACCOUNT" '
  match($0, /[^ ]/) && RSTART <= n {n = 0}
  n && NF == 1 && $1 == account ":" {getline; next}
  !n && /^ *admin::common::passwords:$/ {
    match($0, /[^ ]/)
    n = RSTART
  {print}' < "$YAML"
  • I thought so too - but I wasn't sure about the ending bit for YAML or if it was supposed to be a part of a different script so I enclosed the handler in its own little loop. you use a : too? that is a sure thing then? i think i'll just delete mine. i'm too fuzzy there.
    – mikeserv
    Commented Nov 10, 2015 at 13:39
  • 1
    @mikeserv, I don't know anything about YAML, I just extrapolated from the OP's sample. Commented Nov 10, 2015 at 13:43
  • The problem is that the whitespace before each line is a variable amount, but that amount must be consistent between a stanza's first child node and subsequent child nodes (but grand-children may have a different amount of leading whitespace).
    – Otheus
    Commented Nov 10, 2015 at 13:45
  • Will $/ within double-quotes confuse the shell?
    – Otheus
    Commented Nov 10, 2015 at 13:48
  • @Otheus - no. it only expands expandables. S: the whitespace thing does indicate a closed loop. or the hold space, maybe.
    – mikeserv
    Commented Nov 10, 2015 at 13:50

It would be far safer to use pre-built libraries to parse the whole file (like this, for example https://stackoverflow.com/questions/1773805/how-can-i-parse-a-yaml-file) make your modifications and then re-save the modified file. Doing somewhat blind text modifications on a structured text file like this is just asking for something to go wrong.

  • The problem generally is: comments and subtle formatting will get excluded this way. Same goes for any answer including "augeas".
    – Otheus
    Commented Nov 10, 2015 at 13:39
  • @Otheus - what's augeas? i think its silly to require some behemoth like python to handle a formatted text file. there really aren't that many options.
    – mikeserv
    Commented Nov 10, 2015 at 13:41
  • 1
    Augeaus is an attempt to generalize the maintenance of configuration files and parameters; all directives/parameters are considered nodes in a tree. Different files are managed via different lenses, which describe how such directives/parameters are spliced into nodes. Clever. Almost usable.
    – Otheus
    Commented Nov 10, 2015 at 13:44
  • 1
    Bookmarking this one as an example why YAML-puppet configurations kinda suck. :)
    – Otheus
    Commented Nov 10, 2015 at 13:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .