33

I have a text file with following data and each row ends with |END|.

T|somthing|something|END|T|something2|something2|END|

I am tryig to replace |END| with \n new line with sed.

 sed 's/\|END\|/\n/g' test.txt

But it's producing wrong output like below:

 T
 |
 s
 o
 m
 e
 ...

But what I want is this:

T|somthing|something
T|something2|something2

I also tried with tr. It didn't work either.

1

7 Answers 7

40

Use this:

sed 's/|END|/\n/g' test.txt

What you attempted doesn't work because sed uses basic regular expressions, and your sed implementation has a \| operator meaning “or” (a common extension to BRE), so what you wrote replaces (empty string or END or empty string) by a newline.

3
  • Need to comment out the \ in \n: sed 's/|END|/\\n/g
    – Baazigar
    Apr 2, 2015 at 18:42
  • @Baazigar No, what A.B. wrote is correct (for Linux at least, some sed implementations would emit \n). The question asks how to replace |END| by a newline, not by \n. Apr 3, 2015 at 12:08
  • The characters for newline are '\n'. The \\n is needed because \ also is an escape character, so if you do only \n, you are saying 'escape this n character'. When you do \\n you are saying 'don't treat this next \ as an escape.'.
    – Baazigar
    Apr 17, 2015 at 13:28
8

The following worked fine for me:

$ sed 's/|END|/\
/g' foobar
T|somthing|something
T|something2|something2

Notice that I just put a backslash followed by the enter key.

2
  • 4
    That's the standard syntax. Using \n as in @A.B.'s answer wouldn't work with some sed implementations. Apr 2, 2015 at 16:33
  • @StéphaneChazelas What sed implementation supports \| for alternation in a regexp but not \n meaning newline in an s replacement? Apr 3, 2015 at 12:11
5

You can use awk:

$ awk -F'\\|END\\|' '{$1=$1}1' OFS='\n' file
T|somthing|something
T|something2|something2
  • -F'\\|END\\|' set field separator to |END|
  • OFS='\n' set ouput field separator to newline
  • $1=$1 cause awk reconstruct $0 with OFS as field separator
  • 1 is a true value, causeawk print the whole input line
3

Another possibly command and using its RS option would be:

awk '$1=$1' RS="\|END\|" file

Will print those records (based on awk's Record Separator) which are not empty( has at least one field) to prevent printing empty lines.

Tested on this input:

T|somthing|something|END|T|something2|something2|END|
Test|END|
|END|

Gives this output:

T|somthing|something
T|something2|something2
Test

That cleared all empty lines :) If you want to have newlines too, replace $1=$1 with $0 in command:

awk '$0' RS="\|END\|" file
1
  • $1=$1 condenses sequences of blanks into one space character and returns false if the first field is 0. Doesn't make sense. You probably want awk 1 RS='\\|END\\|' or awk NF RS='\\|END\\|' or awk length RS='\\|END\\|' here. Note that a regexp RS requires gawk or mawk Apr 2, 2015 at 21:49
3

Another way with sed that doesn't print empty lines:

sed 's/|END|/\
/g;/^$/!P;D' infile

e.g. input:

T|one|two|END|T|three|four|END|
T|five|six|END|T|seven|eight|END|
T|nine|ten|END|T|eleven|twelve|END|

output:

T|one|two
T|three|four
T|five|six
T|seven|eight
T|nine|ten
T|eleven|twelve

same thing with ed:

ed -s infile <<'IN'
1,$j
s/|END|/\
/g
,p
q
IN
2

As mentioned here by Walter Mundt, we can achieve this using an ANSI C quoted string

sed $'s/|END|/\\\n/g'

~ $ echo 'T|somthing|something|END|T|something2|something2|END|' | sed 
$'s/|END|/\\\n/g'
T|somthing|something
T|something2|something2

~ $

Check out the above here link for other alternatives.

You can use the following syntax as well, I'm not sure if it works on all flavours of Unix/Linux

sed 's/|END|/\'$'\n''/g'

~ $ echo 'T|somthing|something|END|T|something2|something2|END|' | sed 
's/|END|/\'$'\n''/g'
T|somthing|something
T|something2|something2

~ $
2
  • Working on FreeBSD v10. Actually, only method that worked for me. Thanks you. Dec 14, 2019 at 15:37
  • Worked on macos, while the other answers did not work.
    – psmith
    Feb 11, 2022 at 4:33
1

I had a same issue in strict posix shell I did it in two pass with a unused char

cat data.json|tr '§' '?'|sed -e 's/"[^"]":/§&/g'|tr '§' '\n'

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .