This is the file i want to transform:

john doe  
jane doe  
Beverly Hills

I want to result file to be like that:

john doe,555-666-333,john@gmail.com  
jane doe,Beverly Hills,444-333-111,jane@gmail.com

The word die is in the file and i want to use it to delimit my lines, that's way they will don't appear in the output as show above.

I have change the format of the file. The former format doesn't take in account the fact that number of words between die can vary.

$ awk -v OFS=',' '/^die$/ { print substr(lines,2); lines=""; next } { lines=lines OFS $0 }' file
john doe,555-666-333,john@gmail.com
jane doe,Beverly Hills,444-333-111,jane@gmail.com

Same thing here applies regarding data that contains commas (see end of my answer below). If the data contains commas, you may want to use this:

awk -v OFS=',' '
    /^die$/ { print substr(lines,2); lines=""; next }
    /,/     { $0=sprintf("\"%s\"", $0 ) }
            { lines=lines OFS $0 }' file

The code builds up a string in lines that is delimited by OFS (a comma). When the word die is found by itself on a line, the string in lines is outputted. Tho substr() call removes the comma that was added in front of the line when the first field of the record was appended to the string. Lines with commas are handled the same way as in my code below.

Using GNU awk or mawk, but not BSD awk, you could also do

mawk -v RS='\ndie\n' -v FS='\n' -v ORS='\n' -v OFS=',' '{$1=$1;print}' file

This would not produce quoted fields for data that contains commas.

The $1=$1 forces awk to re-form the record according to the OFS (output field separator) and ORS (output record separator) variables before outputting.

Answer before update to question:

paste -d, - - - - <file

This would produce

john doe,555-666-333,john@gmail.com,die
jane doe,444-333-111,jane@gmail.com,die

To remove the die lines (these are totally unnecessary):

paste -d, - - - - <file | cut -d, -f 1-3

The above works if the original data contains no commas.

You can also filter out the die lines from the start:

sed '/^die$/d' file | paste -d, - - -

This would work even if the original data contains commas.

If the data contains commas, you may want to preprocess it to add quotes around those lines:

awk '/^die$/ { next } /,/ { $0=sprintf("\"%s\"", $0 ) } 1' file | paste -d, - - -

Given the file

john doe
jane doe
Me, myself and I

that last command would generate

john doe,555-666-333,john@gmail.com
jane doe,444-333-111,jane@gmail.com
"Me, myself and I",000-000-000,myself@example.com
| improve this answer | |
  • Running your code above i obtain: john doe,555-666-333,john@gmail.com,janedoe,444-333-111 444-333-111,jane@gmail.com,"Me, myself and I",000-000-000, myself@example.com – Anicet Ebou Apr 20 '18 at 12:46
  • The lines are not clearly separeted. – Anicet Ebou Apr 20 '18 at 12:52
  • @AnicetEbou If the records contain different number of fields, how can you later distinguish what's what? See also updated answer. – Kusalananda Apr 20 '18 at 13:42

You can do it with a kind of idiomatic awk like this:

$ awk '$1=$1' RS='.die\n' OFS="," FS='\n' file1
john doe,555-666-333,john@gmail.com
jane doe,Beverly Hills,444-333-111,jane@gmail.com

In above awk, we define Record Separator RS to be the die entry that is used in your file to separate person's details.

$1=$1 forces awk to recalculate and print input fields using "," as Output Field Separator OFS

PS: When i'am suspicious of bad file endings like \r i use to call tr to remove possible \r characters: tr -d '\r' file1 |awk .....

By the way, you use also sed like this:

$ sed -z 's/\n/,/g; s/,die,/\n/g'

This will produce the same output as awk, by fooling sed to use null character as record separator.

As soon as there are not real null chars in the input file, sed will treat the whole input file as a big record == a big line.

| improve this answer | |

One way is using paste and sed:

paste -sd, <infile |sed 's/,die,\?/\n/g'
| improve this answer | |

This can be done using the "sed" editor in a POSIX manner

sed -e '
' input_file


  • Setup a loop and append the next line into the pattern space.
    • N command
  • Change the newline to comma and then try to strip off the ",die"
    • y/// s/// commands
  • In case you succeed, then you"re all set and do no further processing for this.
    • t command without a label
  • Otherwise, go back for more and just in case we are eof, we bail out.
    • b and q commands.

We could also use Perl for this:

perl -lne '
    push @A, $_ unless /^die$/;
    print join ",", splice @A if /^die$/ || eof;
' input_file

Where we accumulate lines in an array until we see the "die" line. At which point we join the array contents by means of comma (also empty out the array).

We could also slurp the file and then invoke Perl to get the results:

perl -lF'/^die\n/m' -0777nae 'print join ",", split /\n/ for @F' input_file
  • -F'/^die\n/m' will split the file slurped as a string on the regex BOL die followed by a newline.
  • -0777 will turn on slurping. -n shall disable autoprint of lines and -a will split the slurped lines (in our case just one line) based on the -F value.
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.