2

I have a file which contains multiple URLs. But unfortunately, all the URLs are in one line.

cat url_file

http://transfer.sh/PIGfk/my-file.002554http://transfer.sh/Ep9Md/my-file.002555http://transfer.sh/Ep9Md/my-file.002556http://transfer.sh/Ep9Md/my-file.002557

Expected output:

http://transfer.sh/PIGfk/my-file.002554
http://transfer.sh/Ep9Md/my-file.002555
http://transfer.sh/Ep9Md/my-file.002556
http://transfer.sh/Ep9Md/my-file.002557
1
  • 4
    Are you sure there's no invisible NUL character in there? Check with sed -n l < url_file Feb 27, 2019 at 22:50

5 Answers 5

3

GNU grep

grep -oP 'http://.+?(?=http://|$)' url_file
2

Using perl:

perl -pe 's#(?<=.)(?=http://)#\n#g' url_file

Explanation

This uses a positive lookahead to find substrings that begin with http:// and place a newline (\n) before them.

It also uses a positive lookbehind to only match when there is a character before the http://. In this way, no newline is insterted before the first url on a line. This will be extra handy if you end up with multiple lines.

Update

Prior to @steeldriver's awesome comment, a lookbehind wasn't used and I'd relied on sed '1d' to delete the first line.

0
2

You can use this GNU sed command:

sed 's,http://,\n&,g' url_file | tail -n +2

It looks for the pattern http:// and insert a CR before it.

The tail -n +2 skips the first (empty) line inserted by this sed command.

5
  • or sed 's,\(.\)http://,\1\nhttp://,g'
    – Jeff Schaller
    Feb 27, 2019 at 22:19
  • Thanks @JeffSchaller. I've never been able to do positive lookaheads with sed. That's super handy. You could also do sed 's,\(.\)\(http://\),\1\n\2,g' url_file This way you don't have to re-write the http:// and it's more universal.
    – Crypteya
    Feb 27, 2019 at 22:32
  • @JeffSchaller, why does your sed match the first URL? There are no characters before the first url to be matched by \(.\)
    – Crypteya
    Feb 27, 2019 at 22:53
  • 1
    It doesn't match the first URL, as there's no character before the first one. That's why I mentioned it.
    – Jeff Schaller
    Feb 27, 2019 at 22:54
  • 5
    That \n is a GNU extension. With GNU sed, you can also do sed 's|http://|\n&|2g' Feb 27, 2019 at 22:55
0

Here's a way to do it all within POSIX sed:

$ sed -e '
   s|http://|\
&|2;P;D
' input.file

This places a newline before the 2nd http:// substring to be found in the current line. Then we perform the action "print upto 1st newline, chop upto 1st newline, rinse & repeat" till you run out of the pattern space. When only 1 http:// is left then the substitution does nothing, and that's the last print and delete action for the current record.

You can use Perl arrays to do the job:

perl -F'http:\/\/' -lane 'print "http://$_" for @F[1..$#F]' input.file

The first field $F[0] is empty so is skipped over while printing.

0

I have done by below 3 methods

python

    #!/usr/bin/python
    import re
    k=open('filename','r')
    for i in k:
        print re.sub("http","\nhttp",i)



perl

perl -pne "s/http/\nhttp/g" filename



sed command

sed "s/http/\n&/g" filename

output

http://transfer.sh/PIGfk/my-file.002554
http://transfer.sh/Ep9Md/my-file.002555
http://transfer.sh/Ep9Md/my-file.002556
http://transfer.sh/Ep9Md/my-file.002557

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .