What should I use when cut doesn't cut it?

Question

I have a file cities like this:

[1598] San Diego, US (inactive)
[4517] St Louis, US (inactive)
[6346] Orlando, US (inactive)

I want to cut out the city names, so that I have:

San Diego
St Louis
Orlando

This is the best I could come up with:

cut -d ',' -f1 cities | cut -d ']' -f2

But that still leaves me with a space before the names. Is there a cut like command that I can use that accept delimiters of several characters so I can cut on ]?

Answer 1

Awk (also check Awk Info) is beautiful with that sort of question. Try:

awk -F'[],] *' '{print $2}' cities

This defines a field separator -F as [],] * - which means one occurence of either a closing square bracket or a comma, followed by zero or any number of spaces. Of course you can change that to suit any requirement. Read up on regular expressions.

Once the line is split, you can do what you want with the split result. Here, I decided to print out the second field only with print $2. Note that it is important to use single quotes around the awk instructions otherwise $2 gets substituted by the shell.

Answer 2

You can modify the last cut in your pipeline to this:

cut -d ' ' -f2-

The above means that field separator is whitespace, and we want to select all fields starting from the second. The complete sequence becomes:

cut -d ',' -f1 cities | cut -d ' ' -f2-

Answer 3

For more complex parsing, you should use sed(1):

sed -e 's/\[[0-9]\+\] \([^,]\+\),.*/\1/' cities

Or using -r to simplify the regular expression, as suggested by pepoluan:

sed -re 's/\[[0-9]+\] ([^,]+),.*/\1/' cities

Answer 4

I normally use Perl when things get too hard for sed and grep.

There are a number of ways you could write it in Perl. For example, you might prefer it to be fast, or you might prefer it to handle slight unexpected problems in the input (e.g. two spaces where one was expected).

One obvious way (assumes id is numeric, city is alphabetic, status is alphabetic):

while (<>) {
    if (/^\[\d+\] (\w+(?: \w+)*), \w+ \(\w*\)$/) {
        my $city = $1;
        print "$city\n";
    }
}

Or slower but more permissive (does more backtracking):

while (<>) {
    if (/^.*\]\s+(.*),.*$/) {
        my $city = $1;
        print "$city\n";
    }
}

Or faster (field stops at first occurrence of closing bracket):

while (<>) {
    if (/^\[[^]]*\] ([^,]*), \S+ \([^)]*\)$/) {
        my $city = $1;
        print "$city\n";
    }
}

From the command line rather than a script, you could use the -n option, which basically adds the while (<>) { BLOCK } loop:

perl -ne '/^\[[^]]*\] ([^,]*), \S+ \([^)]*\)$/ and print $1, "\n";' cities

or if you want the usage to resemble cut, you can use the -F option, which is similar to awk's -F option, for example:

perl -a -n -F'/[],]\s+/' -e 'print $F[1], "\n"' cities

This way obviously assumes that no field will contain any of the delimiters.