I'm trying to remove some patterns from a file input string (parsing file line by line). Here is a sample string:

1: (10/17 12:49:31.175) - CONSTANT ID1 - CONSTANT ID2: RAW DATA OUT     > [0x00,0xa2,…,0x00] 

And I want the output to be something like the following:

(12:49:31.175): RAW DATA OUT : [0x00,0xa2,…,0x00]

I have been trying to use egrep or sed but no luck so far, just keep getting errors or some "unterminated substitute pattern" error. Here is and example of what I have tried:

echo $line | sed -e 's/.*\s\([0-9]*:[0-9]*:[0-9]*.[0-9]*\)'

Any help would be appreciated. I'm usually more of a batch file guy instead of bash on my Mac OS/X.


I should mention, I'm reading a file as such:

while read line

Then I want to perform any actions on line for every line in the file.


You can also call perl from bash. The -n arg makes it loop for each line. -e means the script is one line.

cat in.txt | perl -ne 's{.*(.*\().*? (.*) -.*(:.*?)\s\s+.*(\[.*)}{$1$2$3 : $4};'
| improve this answer | |
  • Yup, this is exactly what I was looking for. I had an extra - in the CONSTANT ID so I had to add another -.* in there. Although, I was hoping not to use perl and more regular scripting commands. – Chef Pharaoh Oct 21 '14 at 17:25
  • Can I use this output to print to a file now? Haha, never mind, just a matter of using redirect. – Chef Pharaoh Oct 21 '14 at 17:27

This works on your example input line:

 sed 's/.* \([0-9].*)\) .*>/(\1: RAW DATA OUT:/
 ' <<\DATA
 1: (10/17 12:49:31.175) - CONSTANT ID1 - CONSTANT ID2: RAW DATA OUT     > [0x00,0xa2,…,0x00]

It prints:

(12:49:31.175): RAW DATA OUT : [0x00,0xa2,…,0x00]    
| improve this answer | |
sed '
  s/([^ ]\+ /(/
  s/ - CONSTANT ID//
  s/ \+>/ :/
' <<DATA
1: (10/17 12:49:31.175) - CONSTANT ID: RAW DATA OUT     > [0x00,0xa2,…,0x00]
(12:49:31.175): RAW DATA OUT : [0x00,0xa2,…,0x00]

The newlines in the sed script can be replaced by semicolons if you prefer

In a while-read loop:

while read line; do
    echo "$line" | sed 's/^[^(]*//; s/([^ ]\+ /(/; s/ - CONSTANT ID//; s/ \+>/ :/'
done < filename
| improve this answer | |
  • This looks good but I keep getting "No such file or directory" error. I'm reading a file like so: while read line and then want to parse line for each line in the file. – Chef Pharaoh Oct 21 '14 at 16:56
  • Or now I'm getting s/([^ ] ...": bad flag in substitute command: ')' – Chef Pharaoh Oct 21 '14 at 17:09
  • Anyone trying to replicate your approach is going to be in for an ugly surprise if the <<DATA they feed sed has any shell special characters in it. If you do not quote the heredoc limiter the shell will perform any expansions it can on the contents. – mikeserv Oct 21 '14 at 18:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.