21

I have a text file with following data and each row ends with |END|.

T|somthing|something|END|T|something2|something2|END|

I am tryig to replace |END| with \n new line with sed.

 sed 's/\|END\|/\n/g' test.txt

But it's producing wrong output like below:

 T
 |
 s
 o
 m
 e
 ...

But what I want is this:

T|somthing|something
T|something2|something2

I also tried with tr. It didn't work either.

1
22

Use this:

sed 's/|END|/\n/g' test.txt

What you attempted doesn't work because sed uses basic regular expressions, and your sed implementation has a \| operator meaning “or” (a common extension to BRE), so what you wrote replaces (empty string or END or empty string) by a newline.

3
  • Need to comment out the \ in \n: sed 's/|END|/\\n/g
    – Baazigar
    Apr 2 '15 at 18:42
  • @Baazigar No, what A.B. wrote is correct (for Linux at least, some sed implementations would emit \n). The question asks how to replace |END| by a newline, not by \n. Apr 3 '15 at 12:08
  • The characters for newline are '\n'. The \\n is needed because \ also is an escape character, so if you do only \n, you are saying 'escape this n character'. When you do \\n you are saying 'don't treat this next \ as an escape.'.
    – Baazigar
    Apr 17 '15 at 13:28
8

The following worked fine for me:

$ sed 's/|END|/\
/g' foobar
T|somthing|something
T|something2|something2

Notice that I just put a backslash followed by the enter key.

2
  • 3
    That's the standard syntax. Using \n as in @A.B.'s answer wouldn't work with some sed implementations. Apr 2 '15 at 16:33
  • @StéphaneChazelas What sed implementation supports \| for alternation in a regexp but not \n meaning newline in an s replacement? Apr 3 '15 at 12:11
5

You can use awk:

$ awk -F'\\|END\\|' '{$1=$1}1' OFS='\n' file
T|somthing|something
T|something2|something2
  • -F'\\|END\\|' set field separator to |END|
  • OFS='\n' set ouput field separator to newline
  • $1=$1 cause awk reconstruct $0 with OFS as field separator
  • 1 is a true value, causeawk print the whole input line
3

Another possibly command and using its RS option would be:

awk '$1=$1' RS="\|END\|" file

Will print those records (based on awk's Record Separator) which are not empty( has at least one field) to prevent printing empty lines.

Tested on this input:

T|somthing|something|END|T|something2|something2|END|
Test|END|
|END|

Gives this output:

T|somthing|something
T|something2|something2
Test

That cleared all empty lines :) If you want to have newlines too, replace $1=$1 with $0 in command:

awk '$0' RS="\|END\|" file
1
  • $1=$1 condenses sequences of blanks into one space character and returns false if the first field is 0. Doesn't make sense. You probably want awk 1 RS='\\|END\\|' or awk NF RS='\\|END\\|' or awk length RS='\\|END\\|' here. Note that a regexp RS requires gawk or mawk Apr 2 '15 at 21:49
3

Another way with sed that doesn't print empty lines:

sed 's/|END|/\
/g;/^$/!P;D' infile

e.g. input:

T|one|two|END|T|three|four|END|
T|five|six|END|T|seven|eight|END|
T|nine|ten|END|T|eleven|twelve|END|

output:

T|one|two
T|three|four
T|five|six
T|seven|eight
T|nine|ten
T|eleven|twelve

same thing with ed:

ed -s infile <<'IN'
1,$j
s/|END|/\
/g
,p
q
IN
1

I had a same issue in strict posix shell I did it in two pass with a unused char

cat data.json|tr '§' '?'|sed -e 's/"[^"]":/§&/g'|tr '§' '\n'
1

As mentioned here by Walter Mundt, we can achieve this using an ANSI C quoted string

sed $'s/|END|/\\\n/g'

~ $ echo 'T|somthing|something|END|T|something2|something2|END|' | sed 
$'s/|END|/\\\n/g'
T|somthing|something
T|something2|something2

~ $

Check out the above here link for other alternatives.

You can use the following syntax as well, I'm not sure if it works on all flavours of Unix/Linux

sed 's/|END|/\'$'\n''/g'

~ $ echo 'T|somthing|something|END|T|something2|something2|END|' | sed 
's/|END|/\'$'\n''/g'
T|somthing|something
T|something2|something2

~ $
1
  • Working on FreeBSD v10. Actually, only method that worked for me. Thanks you. Dec 14 '19 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.