Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have an issue where I am getting a flatfile source with a huge number of records with PIPE delimiters and one of the fields is getting carriage returns (in multiple lines) and starts with a newline (\n). So how can I remove the \n character in the file?

Example:

-000123456|1654321|6/12/2002 8:49:20 AM|
tt Cynthia L Eggleston E456585 remove move the funds adv account in fcle flagged on 710091 pmt due 12-16- 15|

Can anyone suggest how to proceed?

share|improve this question
add comment

2 Answers

If you know, that this field begins with \n in all of the records, you can use sed as follows:

sed "N;s/|\n/|/"

to get rid of the new-line. Note, that since sed uses \n as line delimiters, you first have to join the next line to the one processed (the N command), during this process sed inserts the newline character between the joined lines.

If this problem arises randomly (not in every records), you'll have to resort to a stronger tool to parse the file content - basically you need to count the fields. Either awk or perl can do the trick elegantly (I think it could be accomplished in sed as well as the, but you probably don't want go that direction).

share|improve this answer
add comment

You can use many tools, but sed was built for exactly these kinds of situations:

 sed -i 's,\\n,,g' Flatfile

To show you how it works, here's the same expression with the input you provided just echoed:

$ echo '-000123456|1654321|6/12/2002 8:49:20 AM|\n tt Cynthia L Eggleston E456585 remove move the funds adv account in fcle flagged on 710091 pmt due 12-16- 15|' | sed 's,\\n,,g'
-000123456|1654321|6/12/2002 8:49:20 AM| tt Cynthia L Eggleston E456585 remove move the funds adv account in fcle flagged on 710091 pmt due 12-16- 15|

It just looks for the string \n and replaces it with an empty, zero-length string (effectively removing it). There are two \, so \n loses its special (newline character) meaning and can be matched as two characters.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.