2

I have a file which contains multiple URLs. But unfortunately, all the URLs are in one line.

cat url_file

http://transfer.sh/PIGfk/my-file.002554http://transfer.sh/Ep9Md/my-file.002555http://transfer.sh/Ep9Md/my-file.002556http://transfer.sh/Ep9Md/my-file.002557

Expected output:

http://transfer.sh/PIGfk/my-file.002554
http://transfer.sh/Ep9Md/my-file.002555
http://transfer.sh/Ep9Md/my-file.002556
http://transfer.sh/Ep9Md/my-file.002557
  • 4
    Are you sure there's no invisible NUL character in there? Check with sed -n l < url_file – Stéphane Chazelas Feb 27 at 22:50
2

Using perl:

perl -pe 's#(?<=.)(?=http://)#\n#g' url_file

Explanation

This uses a positive lookahead to find substrings that begin with http:// and place a newline (\n) before them.

It also uses a positive lookbehind to only match when there is a character before the http://. In this way, no newline is insterted before the first url on a line. This will be extra handy if you end up with multiple lines.

Update

Prior to @steeldriver's awesome comment, a lookbehind wasn't used and I'd relied on sed '1d' to delete the first line.

2

GNU grep

grep -oP 'http://.+?(?=http://|$)' url_file
2

You can use this GNU sed command:

sed 's,http://,\n&,g' url_file | tail -n +2

It looks for the pattern http:// and insert a CR before it.

The tail -n +2 skips the first (empty) line inserted by this sed command.

  • or sed 's,\(.\)http://,\1\nhttp://,g' – Jeff Schaller Feb 27 at 22:19
  • Thanks @JeffSchaller. I've never been able to do positive lookaheads with sed. That's super handy. You could also do sed 's,\(.\)\(http://\),\1\n\2,g' url_file This way you don't have to re-write the http:// and it's more universal. – Crypteya Feb 27 at 22:32
  • @JeffSchaller, why does your sed match the first URL? There are no characters before the first url to be matched by \(.\) – Crypteya Feb 27 at 22:53
  • 1
    It doesn't match the first URL, as there's no character before the first one. That's why I mentioned it. – Jeff Schaller Feb 27 at 22:54
  • 5
    That \n is a GNU extension. With GNU sed, you can also do sed 's|http://|\n&|2g' – Stéphane Chazelas Feb 27 at 22:55
0

Here's a way to do it all within POSIX sed:

$ sed -e '
   s|http://|\
&|2;P;D
' input.file

This places a newline before the 2nd http:// substring to be found in the current line. Then we perform the action "print upto 1st newline, chop upto 1st newline, rinse & repeat" till you run out of the pattern space. When only 1 http:// is left then the substitution does nothing, and that's the last print and delete action for the current record.

You can use Perl arrays to do the job:

perl -F'http:\/\/' -lane 'print "http://$_" for @F[1..$#F]' input.file

The first field $F[0] is empty so is skipped over while printing.

0

I have done by below 3 methods

python

    #!/usr/bin/python
    import re
    k=open('filename','r')
    for i in k:
        print re.sub("http","\nhttp",i)



perl

perl -pne "s/http/\nhttp/g" filename



sed command

sed "s/http/\n&/g" filename

output

http://transfer.sh/PIGfk/my-file.002554
http://transfer.sh/Ep9Md/my-file.002555
http://transfer.sh/Ep9Md/my-file.002556
http://transfer.sh/Ep9Md/my-file.002557

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.