Search for specified keywords in files and copy to a new filename based on the keyword

Question

I'm trying to create an .sh script where if any keyword is found in a file then copy that file to the directory /pathtofolder/keywords/$thefirstkeywordfound (overwrite the file if it already exists). Any help would be appreciated with the copy code ( cp -rf filename /pathtofolder/keywords/$thefirstkeywordfound ) which is incorrect. It probably also needs a line of code to exclude the path it copies to (/pathtofolder/keywords/)

Note that I want to copy the file into a directory called /pathtofolder/keywords/$keyword, creating it if necessary.

Answer 1

There are various issues here. First, I have no idea what you're trying to do with IFS='\n' so I'll just ignore it. Second, you seem to be copying filename not a variable called filename and in any case, you are not setting it to anything.

If I understand what you are trying to do correctly, you are looking for something like this:

#!/usr/bin/env bash

## Pass the directory to search in as an argument, 
## easier to use and avoids including /pathfolder/
## as long as your run the script from a different
## directory.
dir="$1";

## Destination directory
dest="$2"

## Don't use capitalized variables in bash,
## environmental vars are CAPS and that could cause
## problems if you use something like $USER. 
keywords=("Florida" "FL" "Miami-Dade" "Aventura" "Bal Harbour" "Bay Harbor Islands")

## Find all files of the right size and pass their names
## to the while loop. Skip any files matching $dest
find "$dir" -size +1c -path "$dest" -prune -o  -type f -print0 | 

## This is to make sure it works with file names containing strange
## characters like newlines. If they don't, you can remove the -print0
## from find and simplify to `while read file name`
while IFS= read -r -d $'\0' filename
do
    for keyword in "${keywords[@]}";
    do
        ## -m 1: stop at first match
        grep -qm 1 -Fw "$keyword" "$filename" && 

        ## Create a dir for the keyword if it does not exist
        mkdir -p "$dest"/"$keyword" &&

        ## copy the file if grep found a match (that's what && above means)
        cp -vf "$filename" "$dest"/"$keyword"/ &&

        ## move to the next filename if the copy succeds
        break 
    done
done

Save the script somewhere, foo.sh and run it giving it the folder to be searched as an argument:

./foo.sh "/source/dir" "/target/dir"

Answer 2

You can do that with a somewhat complex awk command:

timp@helez:~/tmp/cp_find_test$ ls
command.sh  command.sh~  test.1  test.2  test.3
timp@helez:~/tmp/cp_find_test$ cat test.1 
Aventura
Whatever
Florida
timp@helez:~/tmp/cp_find_test$ cat test.2
Random stuff
Floridaasdklfj
timp@helez:~/tmp/cp_find_test$ cat test.3
FL
timp@helez:~/tmp/cp_find_test$ ./command.sh
cp -rf ./test.3 /pathtofolder/keywords/FL
cp -rf ./command.sh /pathtofolder/keywords/Florida
cp -rf ./test.1 /pathtofolder/keywords/Aventura
cp -rf ./command.sh~ /pathtofolder/keywords/Florida
timp@helez:~/tmp/cp_find_test$ cat command.sh
KEYWORDS=("Florida" "FL" "Miami-Dade" "Aventura" "Bal Harbour" "Bay Harbor Islands")
IFS=$'\n'
find . -size +1c -type f ! -exec grep -oHwF "${KEYWORDS[*]}" {} \; | awk 'BEGIN {FS=":"; last_line=""} {if (last_line!=$1) {print "cp -rf", $1, "/pathtofolder/keywords/"$2}; last_line=$1}'

I added a -o to your grep so that it would only print the matching parts of the line, and then used awk to build the cp commands.

The reason I didn't use -m 1 in grep instead of the if (last_line!=$1) {...}; last_line=$1 was because if there's multiple matches on the same line it would print them all out as separate lines:

timp@helez:~/tmp/cp_find_test$ ./command.sh
cp -rf ./test.3 /pathtofolder/keywords/FL
cp -rf ./command.sh /pathtofolder/keywords/Florida
cp -rf ./command.sh /pathtofolder/keywords/FL
cp -rf ./command.sh /pathtofolder/keywords/Miami-Dade
cp -rf ./command.sh /pathtofolder/keywords/Aventura
cp -rf ./command.sh /pathtofolder/keywords/Bal Harbour
cp -rf ./command.sh /pathtofolder/keywords/Bay Harbor Islands
cp -rf ./test.1 /pathtofolder/keywords/Aventura
cp -rf ./command.sh~ /pathtofolder/keywords/Florida
cp -rf ./command.sh~ /pathtofolder/keywords/FL
cp -rf ./command.sh~ /pathtofolder/keywords/Miami-Dade
cp -rf ./command.sh~ /pathtofolder/keywords/Aventura
cp -rf ./command.sh~ /pathtofolder/keywords/Bal Harbour
cp -rf ./command.sh~ /pathtofolder/keywords/Bay Harbor Islands
timp@helez:~/tmp/cp_find_test$ cat command.sh
KEYWORDS=("Florida" "FL" "Miami-Dade" "Aventura" "Bal Harbour" "Bay Harbor Islands")
IFS=$'\n'
#find . -size +1c -type f ! -exec grep -oHwF "${KEYWORDS[*]}" {} \; | awk 'BEGIN {FS=":"; last_line=""} {if (last_line!=$1) {print "cp -rf", $1, "/pathtofolder/keywords/"$2}; last_line=$1}'
find . -size +1c -type f ! -exec grep -m 1 -oHwF "${KEYWORDS[*]}" {} \; | awk 'BEGIN {FS=":"} {print "cp -rf", $1, "/pathtofolder/keywords/"$2}'

Answer 3

I Think you need more than a simple pipeline to do that. So, I propose something like this:

#!/bin/bash
KEYWORD_PATTERN='Florida|FL|Miami-Dade|Aventura|Bal Harbour|Bay Harbor Islands'
find . -type f |
while read FNAME
do
    if grep -Ew -q "$KEYWORD_PATTERN" $FNAME
    then
        KEYWORD=$(grep -Ew -o "$KEYWORD_PATTERN" $FNAME  | head -1)
        echo mv $FNAME /pathtofolder/keywords/$KEYWORD
    fi

done

This takes advantage of extended (-E) GNU grep flags, -w and -o. Without those two flags, you'd have to put something like a little Perl program that can break "words" out of lines of text, select a word that matches of the keywords and uses that as the file name.