6

I'm trying to create an .sh script where if any keyword is found in a file then copy that file to the directory /pathtofolder/keywords/$thefirstkeywordfound (overwrite the file if it already exists). Any help would be appreciated with the copy code ( cp -rf filename /pathtofolder/keywords/$thefirstkeywordfound ) which is incorrect. It probably also needs a line of code to exclude the path it copies to (/pathtofolder/keywords/)

Note that I want to copy the file into a directory called /pathtofolder/keywords/$keyword, creating it if necessary.

0

3 Answers 3

5

There are various issues here. First, I have no idea what you're trying to do with IFS='\n' so I'll just ignore it. Second, you seem to be copying filename not a variable called filename and in any case, you are not setting it to anything.

If I understand what you are trying to do correctly, you are looking for something like this:

#!/usr/bin/env bash

## Pass the directory to search in as an argument, 
## easier to use and avoids including /pathfolder/
## as long as your run the script from a different
## directory.
dir="$1";

## Destination directory
dest="$2"

## Don't use capitalized variables in bash,
## environmental vars are CAPS and that could cause
## problems if you use something like $USER. 
keywords=("Florida" "FL" "Miami-Dade" "Aventura" "Bal Harbour" "Bay Harbor Islands")

## Find all files of the right size and pass their names
## to the while loop. Skip any files matching $dest
find "$dir" -size +1c -path "$dest" -prune -o  -type f -print0 | 

## This is to make sure it works with file names containing strange
## characters like newlines. If they don't, you can remove the -print0
## from find and simplify to `while read file name`
while IFS= read -r -d $'\0' filename
do
    for keyword in "${keywords[@]}";
    do
        ## -m 1: stop at first match
        grep -qm 1 -Fw "$keyword" "$filename" && 

        ## Create a dir for the keyword if it does not exist
        mkdir -p "$dest"/"$keyword" &&

        ## copy the file if grep found a match (that's what && above means)
        cp -vf "$filename" "$dest"/"$keyword"/ &&

        ## move to the next filename if the copy succeds
        break 
    done
done

Save the script somewhere, foo.sh and run it giving it the folder to be searched as an argument:

./foo.sh "/source/dir" "/target/dir"
11
  • I made an edit above explaining my current issues with your code. Any help would be greatly appreciated!
    – Guest
    Mar 11, 2014 at 3:35
  • @Guest please don't edit my answer to ask for more information. Edit your post instead. You might want to have a look at our help center or take the tour to understand how the stack exchange sites work. Anyway, sorry and please see updated answer, I had a silly syntax error.
    – terdon
    Mar 11, 2014 at 3:58
  • root@death:~# ./sort-miami.sh "/root/unsorted" "/root/sorted" grep: : No such file or directory grep: : No such file or directory grep: : No such file or directory grep: : No such file or directory grep: : No such file or directory grep: : No such file or directory grep: : No such file or directory grep: : No such file or directory grep: : No such file or directory
    – Guest
    Mar 11, 2014 at 5:05
  • @Guest argh! I seem to have left the error in there again, sorry. Try again, make sure the while ends in filename and not in file name as I had it. It is correct now.
    – terdon
    Mar 11, 2014 at 5:07
  • It works perfect. Just one more question sir. Will it copy the same file to multiple /paths/ example /sorted/fl /sorted/miami /sorted/florida or only one time for the first key word discovered... then the script will move onto the next file.
    – Guest
    Mar 11, 2014 at 5:09
0

You can do that with a somewhat complex awk command:

timp@helez:~/tmp/cp_find_test$ ls
command.sh  command.sh~  test.1  test.2  test.3
timp@helez:~/tmp/cp_find_test$ cat test.1 
Aventura
Whatever
Florida
timp@helez:~/tmp/cp_find_test$ cat test.2
Random stuff
Floridaasdklfj
timp@helez:~/tmp/cp_find_test$ cat test.3
FL
timp@helez:~/tmp/cp_find_test$ ./command.sh
cp -rf ./test.3 /pathtofolder/keywords/FL
cp -rf ./command.sh /pathtofolder/keywords/Florida
cp -rf ./test.1 /pathtofolder/keywords/Aventura
cp -rf ./command.sh~ /pathtofolder/keywords/Florida
timp@helez:~/tmp/cp_find_test$ cat command.sh
KEYWORDS=("Florida" "FL" "Miami-Dade" "Aventura" "Bal Harbour" "Bay Harbor Islands")
IFS=$'\n'
find . -size +1c -type f ! -exec grep -oHwF "${KEYWORDS[*]}" {} \; | awk 'BEGIN {FS=":"; last_line=""} {if (last_line!=$1) {print "cp -rf", $1, "/pathtofolder/keywords/"$2}; last_line=$1}'

I added a -o to your grep so that it would only print the matching parts of the line, and then used awk to build the cp commands.

The reason I didn't use -m 1 in grep instead of the if (last_line!=$1) {...}; last_line=$1 was because if there's multiple matches on the same line it would print them all out as separate lines:

timp@helez:~/tmp/cp_find_test$ ./command.sh
cp -rf ./test.3 /pathtofolder/keywords/FL
cp -rf ./command.sh /pathtofolder/keywords/Florida
cp -rf ./command.sh /pathtofolder/keywords/FL
cp -rf ./command.sh /pathtofolder/keywords/Miami-Dade
cp -rf ./command.sh /pathtofolder/keywords/Aventura
cp -rf ./command.sh /pathtofolder/keywords/Bal Harbour
cp -rf ./command.sh /pathtofolder/keywords/Bay Harbor Islands
cp -rf ./test.1 /pathtofolder/keywords/Aventura
cp -rf ./command.sh~ /pathtofolder/keywords/Florida
cp -rf ./command.sh~ /pathtofolder/keywords/FL
cp -rf ./command.sh~ /pathtofolder/keywords/Miami-Dade
cp -rf ./command.sh~ /pathtofolder/keywords/Aventura
cp -rf ./command.sh~ /pathtofolder/keywords/Bal Harbour
cp -rf ./command.sh~ /pathtofolder/keywords/Bay Harbor Islands
timp@helez:~/tmp/cp_find_test$ cat command.sh
KEYWORDS=("Florida" "FL" "Miami-Dade" "Aventura" "Bal Harbour" "Bay Harbor Islands")
IFS=$'\n'
#find . -size +1c -type f ! -exec grep -oHwF "${KEYWORDS[*]}" {} \; | awk 'BEGIN {FS=":"; last_line=""} {if (last_line!=$1) {print "cp -rf", $1, "/pathtofolder/keywords/"$2}; last_line=$1}'
find . -size +1c -type f ! -exec grep -m 1 -oHwF "${KEYWORDS[*]}" {} \; | awk 'BEGIN {FS=":"} {print "cp -rf", $1, "/pathtofolder/keywords/"$2}'
0

I Think you need more than a simple pipeline to do that. So, I propose something like this:

#!/bin/bash
KEYWORD_PATTERN='Florida|FL|Miami-Dade|Aventura|Bal Harbour|Bay Harbor Islands'
find . -type f |
while read FNAME
do
    if grep -Ew -q "$KEYWORD_PATTERN" $FNAME
    then
        KEYWORD=$(grep -Ew -o "$KEYWORD_PATTERN" $FNAME  | head -1)
        echo mv $FNAME /pathtofolder/keywords/$KEYWORD
    fi

done

This takes advantage of extended (-E) GNU grep flags, -w and -o. Without those two flags, you'd have to put something like a little Perl program that can break "words" out of lines of text, select a word that matches of the keywords and uses that as the file name.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .