Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am working on a movie database problem to improve regular expressions, this is the problem I'm running into. My dataset looks like this:

Movie Name (variable space and tabs) year
Movie1(can have spaces or multiple spaces between them)(variable spaces and tabs could be \t+ or multiple space or single space> Year1
Movie2(can have spaces or multiple spaces between them)(variable spaces and tabs could be \t+ or multiple space or single space> Year2
Movie3(can have spaces or multiple spaces between them)(variable spaces and tabs could be \t+ or multiple space or single space> Year3
Movie4(can have spaces or multiple spaces between them)(variable spaces and tabs could be \t+ or multiple space or single space> Year4

I want to extract names of all of the movies. These are the challenges I'm facing while doing it:

1: The delimiter is variable. If it was colon or something unique, I would have used an awk command to extract them like this awk -F 'separator' '{print $1}'
In this case, it can be single space, two or more spaces or combination of \t or spaces.

2: For those rows where delimiter is \t, I can use a \t to extract it, because that does not come in movie names. But what if the delimiter is one space or two spaces. They can very easily appear in the movie's name. In those cases, I don't know what to do.

I know the question is very rigid and specific. But as I described earlier, I'm very much blocked here. I can't think of any way around this problem.

Is there any combination of grep/sed/awk with reg-ex that can be used to achieve the objective?

share|improve this question
1  
I'm having trouble understanding your description. Could you post a few actual sample lines that illustrate the variety? –  Gilles Jul 5 at 11:29

5 Answers 5

up vote 7 down vote accepted

Using gawk and assuming that the year always ends the record:

awk -F"[0-9]{4}$" '{print $1}' movies
share|improve this answer

bash:

while read -r line; do
    if [[ $line =~ (.*)[[:blank:]]+[0-9]{4}$ ]]; then
        echo "${BASH_REMATCH[1]}"
    fi
done < data

sed:

sed 's/[[:blank:]]\+[0-9]\{4\}$//' < data
share|improve this answer
    
I like the bash approach. It is really good. :) –  Ramesh Jul 4 at 19:58

This is really quite simple. As long as the last field, the year, does not contain any whitespace (this is not clear from your question but I am assuming it is the case), all you need to do is remove the last field. For example:

$ cat movies
Casablanca  1942
Eternal Sunshine        of the Spotless Mind            2004
He Died with a Felafel in His Hand                       2001
The Blues Brothers 1980

So, if you want to print just the title, you can use:

$ perl -lpe 's/[^\s]+$//' movies
Casablanca  
Eternal Sunshine        of the Spotless Mind            
He Died with a Felafel in His Hand                       
The Blues Brothers 

$ sed 's/[^ \t]*$//' movies 
Casablanca  
Eternal Sunshine        of the Spotless Mind            
He Died with a Felafel in His Hand                       
The Blues Brothers 

or, to collapse the whitespace in the titles as well:

$ sed -r 's/[\t ]+/ /g;s/[^ \t]*$//' movies 
Casablanca 
Eternal Sunshine of the Spotless Mind 
He Died with a Felafel in His Hand 
The Blues Brothers 

$ perl -lpe 's/\s+/ /g;s/[^\s]+$//' movies
Casablanca 
Eternal Sunshine of the Spotless Mind 
He Died with a Felafel in His Hand 
The Blues Brothers 

$ awk '{for(i=1;i<NF-1;i++){printf "%s ",$i} print $(NF-1)}' movies
Casablanca 
Eternal Sunshine of the Spotless Mind 
He Died with a Felafel in His Hand 
The Blues Brothers 

If the year is always 4 digits, you can use

$ perl -lpe 's/....$//' movies 
Casablanca 
Eternal Sunshine of the Spotless Mind 
He Died with a Felafel in His Hand 
The Blues Brothers 

or

$ perl -lpe 's/\s+/ /g;s/....$//' movies 
Casablanca 
Eternal Sunshine of the Spotless Mind 
He Died with a Felafel in His Hand 
The Blues Brothers 

or

$ while read line; do echo ${line%%????}; done < movies|od -c 
Casablanca 
Eternal Sunshine of the Spotless Mind 
He Died with a Felafel in His Hand 
The Blues Brothers 
share|improve this answer
    
+1 for your choice of films :) –  jasonwryan Jul 5 at 10:32
    
@terdon, for the while loop what happens when the movie name is 2012? –  Ramesh Jul 6 at 2:59
    
@Ramesh it will fail. Thanks for pointing it out, I fixed it and made some other improvements too. –  terdon Jul 6 at 19:38

I assume the movie data will look something like below.

cat movies
one flew over the cuckoo's nest          1975
taxi driver      1976
the shining    1980

Now, I also assume the years in the movie data will always be 4 characters at the end.

So, now if you use the commands as below,

 awk '{ gsub (" ", "", $0); print}' movies | rev | cut -c 5- | rev

The output would be,

oneflewoverthecuckoo'snest
taxidriver
theshining

EDIT:

However, the best approach would be,

rev movies | cut -c5- | rev
one flew over the cuckoo's nest          
taxi driver      
the shining   

Of course, I assume the year in your data will always be 4 characters. If it is always the same number of characters you could follow the second approach as it even maintains the spaces in the movie names.

share|improve this answer

This should remove the last numeric characters and the tabs and spaces before it:

sed -e 's#[\t ]*[0-9]*$##' movies.txt
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.