Sed or awk - Insert a new line after Matching pattern

Question

I have a file which contains multiple URLs. But unfortunately, all the URLs are in one line.

cat url_file

http://transfer.sh/PIGfk/my-file.002554http://transfer.sh/Ep9Md/my-file.002555http://transfer.sh/Ep9Md/my-file.002556http://transfer.sh/Ep9Md/my-file.002557

Expected output:

http://transfer.sh/PIGfk/my-file.002554
http://transfer.sh/Ep9Md/my-file.002555
http://transfer.sh/Ep9Md/my-file.002556
http://transfer.sh/Ep9Md/my-file.002557

Answer 1

GNU grep

grep -oP 'http://.+?(?=http://|$)' url_file

Answer 2

Using perl:

perl -pe 's#(?<=.)(?=http://)#\n#g' url_file

Explanation

This uses a positive lookahead to find substrings that begin with http:// and place a newline (\n) before them.

It also uses a positive lookbehind to only match when there is a character before the http://. In this way, no newline is insterted before the first url on a line. This will be extra handy if you end up with multiple lines.

Update

Prior to @steeldriver's awesome comment, a lookbehind wasn't used and I'd relied on sed '1d' to delete the first line.

Answer 3

You can use this GNU sed command:

sed 's,http://,\n&,g' url_file | tail -n +2

It looks for the pattern http:// and insert a CR before it.

The tail -n +2 skips the first (empty) line inserted by this sed command.

Answer 4

Here's a way to do it all within POSIX sed:

$ sed -e '
   s|http://|\
&|2;P;D
' input.file

This places a newline before the 2nd http:// substring to be found in the current line. Then we perform the action "print upto 1st newline, chop upto 1st newline, rinse & repeat" till you run out of the pattern space. When only 1 http:// is left then the substitution does nothing, and that's the last print and delete action for the current record.

You can use Perl arrays to do the job:

perl -F'http:\/\/' -lane 'print "http://$_" for @F[1..$#F]' input.file

The first field $F[0] is empty so is skipped over while printing.

Answer 5

I have done by below 3 methods

python

    #!/usr/bin/python
    import re
    k=open('filename','r')
    for i in k:
        print re.sub("http","\nhttp",i)



perl

perl -pne "s/http/\nhttp/g" filename



sed command

sed "s/http/\n&/g" filename

output

http://transfer.sh/PIGfk/my-file.002554
http://transfer.sh/Ep9Md/my-file.002555
http://transfer.sh/Ep9Md/my-file.002556
http://transfer.sh/Ep9Md/my-file.002557