0

I have a file where the lines are partitioned by internal carriage returns, like the below:

EMP|123|10\rINFO|JOHN|SMITH|M|01/12/1980\rADDR|125|BRIDGE RD|COLUMBUS|OH
EMP|456|10\rINFO|DAVID|PIRT|M|02/12/1980\rADDR|257|BRIDGE RD|COLUMBUS|OH
EMP|789|10\rINFO|JENN|MCKENZI|F|03/12/1980\rADDR|389|BRIDGE RD|COLUMBUS|OH

Note that the portions of the lines between the CRs are further delimited by | characters.

I want to break each line into separate lines at the CR characters.  Then each new line (i.e., a line formed by replacing a CR with a newline character) should start with the Id (the second |-delimited field) from the original line.

Expected Output:

EMP|123|10
123|INFO|JOHN|SMITH|M|01/12/1980
123|ADDR|125|BRIDGE RD|COLUMBUS|OH
EMP|456|10
456|INFO|DAVID|PIRT|M|02/12/1980
456|ADDR|257|BRIDGE RD|COLUMBUS|OH
EMP|789|10
789|INFO|JENN|MCKENZI|F|03/12/1980
789|ADDR|389|BRIDGE RD|COLUMBUS|OH

The Id is repeated so we can know that each group of three lines in the output belong to the same (one) input line.

I tried the below command, but it only replaced the CRs with newlines; I am not sure how to append the Id to new lines

tr '\r' '\n' < test.txt > new.txt

output:

EMP|123|10
INFO|JOHN|SMITH|M|01/12/1980
ADDR|125|BRIDGE RD|COLUMBUS|OH
EMP|456|10
INFO|DAVID|PIRT|M|02/12/1980
ADDR|257|BRIDGE RD|COLUMBUS|OH
EMP|789|10
INFO|JENN|MCKENZI|F|03/12/1980
ADDR|389|BRIDGE RD|COLUMBUS|OH

How can I do this?

0

2 Answers 2

2

How about

$ awk -F '\r' '{
  print $1; 
  split($1,a,"|"); 
  for(i=2;i<=NF;i++) print a[2] "|" $i;
}' file
EMP|123|10
123|INFO|JOHN|SMITH|M|01/12/1980
123|ADDR|125|BRIDGE RD|COLUMBUS|OH
EMP|456|10
456|INFO|DAVID|PIRT|M|02/12/1980
456|ADDR|257|BRIDGE RD|COLUMBUS|OH
EMP|789|10
789|INFO|JENN|MCKENZI|F|03/12/1980
789|ADDR|389|BRIDGE RD|COLUMBUS|OH
2
  • Thank you it helped , but i have 1 more question how do i add another string to line for example a[3] also like 123|10|.. Jun 10, 2019 at 23:13
  • @PhaneendraPavuluri you mean like a[2] "|" a[3] "|" $i ? you can just concatenate strings Jun 10, 2019 at 23:29
0

In sed:

sed ':loop; s/^\([^|]*|\)\([^|]*|\)\([^\r]*\)\r/\1\2\3\n\2/; t loop'

It’s a (somewhat) simple substitution: Match a pattern that consists of all the characters through (i.e., up to and including) the first \r in the buffer.  Break it into three capture groups:

  • Everything through (i.e., up to and including) the first | in the buffer.  This will be EMP| for every line in your file.
  • Everything after that, through (i.e., up to and including) the next (i.e., second) | in the buffer.  This will be 123|, 456| or 789| in your file.
  • Everything after that, up to (but not including) the first \r in the buffer.

Then it replaces the above with

  • The three capture groups (\1\2\3); i.e., everything up to (but not including) the first \r in the buffer,
  • A newline, and
  • The second capture group (\2); i.e., the id and the |.

Then the rest of the buffer (everything after the first \r in the buffer) is left in place, so it falls after the repeated id on the newly created line.

Then, if the above match and substitute succeeded (t is test and goto if success), go back and try it again.

An aspect of this that might not be obvious to the beginner is that, even after we have inserted a newline \n into the buffer with a substitute command, thereby creating a new line, sed still operates on the entire buffer (including the internal / embedded newline character) and not just the ‘new line’ (i.e., the portion after the newline).  So, sed reads the first line,

EMP|123|10\rINFO|JOHN|SMITH|M|01/12/1980\rADDR|125|BRIDGE RD|COLUMBUS|OH

into the buffer.  After the s command is executed for the first time, the buffer contains

EMP|123|10
123|INFO|JOHN|SMITH|M|01/12/1980\rADDR|125|BRIDGE RD|COLUMBUS|OH

and so the next iteration of the s command matches

  • \1 = EMP|,
  • \2 = 123|, and
  • \3 = 10 123|INFO|JOHN|SMITH|M|01/12/1980 (including the newline that was inserted in the first iteration).

Warning: This may make a mess if there is a \r before the second | in an input line.

1
  • Can you please let me know hot to validate if we are parsing correctly by using above answered commands Aug 29, 2019 at 19:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .