I am new to shell scripting

What I have is a log file, each line begins like the below entry,

[2021.04.27.17:03:43.813]: ERROR -
[2021.04.27.17:04:36.790]: SUCCESS -

The desired output to a new file is

2021-04-27 17:03:43 813,ERROR,
2021-04-27 17:04:36 790,SUCCESS,

What I have coded is

while read a; do
    echo ${a//"["/""}
    echo ${a//"]"/""}
done < xms.log > resultlog.txt

The above code is removing square brackets for alternate lines meaning from the first line "[" opening square is removed and from second line "]" closing bracket is removed I want both to be removed from all the lines. Not only that the dot I want to change to hyphen has shown in the above example.

  • You say each line begins with the text that you show. That means there is more data on the line, right? Are you able to show a complete line, just so that we might se what other types of characters etc. there might be (so that nobody gives an answer that e.g. deletes all dashes or square brackets when these needs to be preserved in the rest of the line).
    – Kusalananda
    Commented Jun 10, 2021 at 11:38
  • 1

2 Answers 2


Use the stream editor sed to modify a text file:

sed 's/\./-/;s//-/;s// /;s// /;s/: /,/;s/ -/,/;s/\[//;s/]//' xms.log > resultlog.txt

The script contains only substitute commands of the form s/pattern/replacement/, separated by semicolons. s/\./-/ replaces the first dot with a dash. You also want to replace the next dot, so you don't need to repeat the pattern, but just leave the pattern empty and write s//-/. The same way you do all the other replacements required. If characters with a special meaning in regular expression syntax should be matched, you need to escape them with a backslash: \. and \[.

More readable, you can also write it as

sed 's/\./-/
     s/\./ /
     s/\./ /
     s/: /,/
     s/ -/,/
     s/]//' xms.log > resultlog.txt

To process text from within a shell script, you'd call a text processing tool such as sed, awk or perl to process the whole input, not several invocations for inadequate tools such as read or echo on each line of the output.

Here, I'd use regular expressions to match the input and extract the relevant fields and reformat as you like. perl is the most appropriate to regexp matching:

perl -lpe '
    $_ = "$1-$2-$3 $4 $5,$6," if
     m{^\[(\d\d\d\d)\.(\d\d)\.(\d\d)\.(\d\d:\d\d:\d\d)\.(\d\d\d)\]: (\w+)}
  ' < input.txt > output.txt

Where, in the regular expression syntax:

  • ^ matches at the start of the subject (lines here)
  • \[ matches a [ character.
  • \d matches a decimal digitΒΉ.
  • \. matches a literal . character.
  • \w+ matches one or more word characters (here lettersΒΉ or underscore) as many as possible.
  • Β  and : match themselves respectively.
  • (...) is used to capture what's matched in $1, $2...

That reformats the lines that match the pattern and leave the other ones (if any) alone.

ΒΉ limited to the ASCII ones (0123456789) as we've not told perl to work in Unicode mode nor as per the locale's encoding, so it won't include other decimal digits such as π–­π–­‘π–­’π–­“π–­”π–­•π–­–π–­—π–­˜π–­™πŸŽπŸπŸπŸ‘πŸ’πŸ“πŸ”πŸ•πŸ–πŸ—πŸ˜πŸ™πŸšπŸ›πŸœπŸπŸžπŸŸπŸ πŸ‘. Same goes for the letters matched by \w.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .