17

Suppose I have a list of URLs in a text file:

google.com/funny
unix.stackexchange.com/questions
isuckatunix.com/ireallydo

I want to delete everything that comes after '.com'.

Expected Results:

google.com
unix.stackexchange.com
isuckatunix.com

I tried

sed 's/.com*//' file.txt 

but it deleted .com as well.

  • Is there a specific reason for which you want to search for .com only instead of removing everything after and including the first / character? What if you had a URL like en.wikipedia.org/wiki/Ubuntu in your list? – Byte Commander Feb 1 '16 at 18:55
13

To explicitly delete everything that comes after ".com", just tweak your existing sed solution to replace ".com(anything)" with ".com":

sed 's/\.com.*/.com/' file.txt

I tweaked your regex to escape the first period; otherwise it would have matched something like "thisiscommon.com/something".

Note that you may want to further anchor the ".com" pattern with a trailing forward-slash so that you don't accidentally trim something like "sub.com.domain.com/foo":

sed 's/\.com\/.*/.com/' file.txt
7

You can use awk's field separator (-F) following way:

$ cat file
google.com/funny
unix.stackexchange.com/questions
isuckatunix.com/ireallydo

$ cat file | awk -F '\\.com' '{print $1".com"}'
google.com
unix.stackexchange.com
isuckatunix.com

Explanation:

NAME
       awk - pattern scanning and processing language

-F fs
       --field-separator fs
              Use fs for the input field separator (the value of the FS predefined variable).

As you want to delete every things after .com, -F '.com' separates line with .com and print $1 gives output only the part before .com. So, $1".com" adds .com and gives you expected output.

4

The best tool for non-interactive in-place file editing is ex.

ex -sc '%s/\(\.com\).*/\1/ | x' file.txt

If you've used vi and if you've ever typed a command that begins with a colon : you've used an ex command. Of course many of the more advanced or "fancy" commands you can execute this way are Vim extensions (e.g. :bufdo) and are not defined in the POSIX specifications for ex, but those specifications allow for a truly astonishing degree of power and flexibility in non-visual text editing (whether interactive or automated).

The command above has several parts.

-s enables silent mode to prepare ex for batch use. (Suppress output messages et. al.)

-c specifies the command to execute once the file (file.txt, in this case) is opened in a buffer.

% is an address specifier equivalent to 1,$—it means that the following command is applied to all lines of the buffer.

s is the substitute command that you are likely familiar with already. It is commonly used in vi and has essentially identical features to the s command of sed, though some of the advanced regex features may vary by implementation. In this case from ".com" to the end of the line is replaced with just ".com".

The vertical bar separates sequential commands to be executed. In many (most) ex implementations you can also use an additional -c option, like so:

ex -sc '%s/\(\.com\).*/\1/' -c x file.txt

However, this is not required by POSIX.

The x command exits, after writing any changes to the file. Unlike wq which means "write and quit", x only writes to the file if the buffer has been edited. Thus if your file is unaltered, the timestamp will be preserved.

  • +1 for using ex – Jeff Schaller Jan 25 '16 at 19:23
  • 1
    It doesnt edit in-place. At least, it doesnt any more than Gnu sed's bogus -i does. It reads/writes to on-disk buffers. See for yourself w/ ex -r and the preserve command. – mikeserv Feb 1 '16 at 16:59
2

Very quick, simple and dirty python way:

#!/usr/bin/env python
import sys
with open( sys.argv[1]  ) as file:
    for line in file:
        print line.split("/")[0]

Sample run

skolodya@ubuntu:$ chmod +x removeStrings.py                                   

skolodya@ubuntu:$ ./removeStrings.py strings.txt                              
google.com
unix.stackexchange.com
isuckatunix.com


skolodya@ubuntu:$ cat strings.txt                                             
google.com/funny
unix.stackexchange.com/questions
isuckatunix.com/ireallydo
  • 2
    May i please know reason for downvote ? – Sergiy Kolodyazhnyy Feb 1 '16 at 18:33
  • 3
    It works, but it does not care about .com, it just removes everything starting with the first / in the line. (which is in my opinion even the better approach!) – Byte Commander Feb 1 '16 at 18:50
  • 1
    @ByteCommander exactly right ! If domain name is .net, in other approaches the part that comes after domain and extension wouldn't get deleted, so it's safer to use / as separator. – Sergiy Kolodyazhnyy Feb 1 '16 at 18:53
  • +1 for answer and comments that makes me feel like I'm in AskUbuntu.com :D – WinEunuuchs2Unix Jun 17 '18 at 18:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.