I am trying to write a script for changing user password in dovecot user database and I can't understand how to replace a set of characters between delimiters for exact lines with sed.

Please check this line for example (this is the part from dovecot userdb):


How to replace the string between ":" delimeters starting with "{SHA512-CRYPT}" only for user "123@example.com" and not for user "abc@example.com" with sed?



sed 's/\(\(^\|:\)123@example\.com:\)\([^:]\+\)/\1foo/'

given there is no escaped delimiters in the value.

sed 's/\(\(^\|:\)123@example\.com:\)\([^:]\+\)/\1foo/'    
     |     |         |                |     | |  | |     
     |     |         |                |     | |  | +----- H. End of sub.
     |     |         |                |     | |  +------- G. Sub string
     |     |         |                |     | +---------- F. Match Group 1.
     |     |         |                |     +------------ E. End of Group 3.
     |     |         |                +------------------ D. Group 3.
     |     |         +----------------------------------- C. User
     |     +--------------------------------------------- B. Prefix Group 2.
     +--------------------------------------------------- A. Substitute
  • A: s/ Substitute command.
  • B: (^|:) Starts with start of line or delimiter :, Group 2, part of match group 1.
  • C: The user to match, part of match group 1.
  • D: ([^:]+) The part to remove, anything until :. Part of group 3. Everything until next delimiter. Should perhaps be \(:\|$\), but as it should end in : it should suffice.
  • E: \) Ending the removal grouping.
  • F: \1 Put back match group 1. User + delimiter(s).
  • G: foo What ever is to be inserted as crypt.
  • H: / Ending it all. Optioanlly an /g for global, but assume it is a once only.
  • Why you put this \(\( between substitution command and prefix group "B"? And thank you a lot - I ve tried this regex with GNU sed version 4.2.1 - works like a charm! – powerthrash Nov 28 '14 at 20:35
  • @powerthrash: Because it is an either or expression. (^|:) -> Either ^ (start of line) | (or) : (colon). If no grouping it would be (^|:user@examp...) as in: Either Start of line or Colon followed by user .... In this case it would be best to omit it completely though, and use (^user@...), as also mentioned by mikeserv. You want to match start of line. wiki2.dovecot.org/AuthDatabase/PasswdFile – Runium Nov 29 '14 at 18:44
  • @mikeserv: Yes, it replaces any :password: hash if present – not testing for scheme. Should not use (^|:) at start as it for one should be start and second I'm not sure if the user could be part of other (later) field (i.e. :gecos:). – Runium Nov 29 '14 at 19:03

You can just alternate addresses here -

sed '/^123@example.com:{SHA512-CRYPT}/s/{[^:]*/REPLACE/'

Here the s///ubstitution command is a function of the /regxp/ address and so the s///ubstitution is not even attempted unless the line first matches its parent address.

Or just w/ a single s///:

sed 's/^\(123@example.com:\){SHA512-CRYPT}[^:]*/\1REPLACE/'

Or by anchoring to the head-of-line only if 123 does not immediately follow a colon:

sed 's/^\(\(.*:\)*123@example.com:\){SHA512-CRYPT}[^:]*/\1REPLACE/'

While I am certainly no stranger to complicated sub-expressions, in this case such a thing is unnecessary - I usually prefer the address form when I can get it - I find it easier to read, it is more efficient, and fully portable. Admittedly though, at a second glance it would seem I initially over-estimated the inherent complication here.

Similarly, you might also split the back-references and the actual substitution like:

sed -e '
#other commands that can be sure to affect only a line matching the 1st address'

In the above example the address verifies a match by branching out of the script for any !not matching line, but it also pulls double-duty in saving the variable bit in \1 for the next s///ubstitution should any line pass its match test. Moreover, all subsequent commands can also only be executed on a line which matches the initial address.

And of course there is the less-ultimate form which specifies a match function context with curly braces:

sed -e '
    #other function commands
};/other match function/{
    #still more commands...

Using awk:

awk -F':' '/^123@example.com/ {$2="NEWPASSWORDHERE"}1' OFS=':' infile

  • I used awk with its -F option to setting colon(:) as field separator and if line start(^) with 123@example.com then replace column2($2) with new value(NEWPASSWODHERE).

  • OFS=':' Changes the output field separator from space(by default) to colon(:).

  • The 1 on the end, enables printing by default.

  • One question - it's possible to modify a file with awk inplace like using sed with "-i" option? – powerthrash Dec 19 '14 at 22:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.