Bash command find with exec and grep not working properly, anything wrong?

Question

I want to filter my log files with the keyword ALARM/FATAL in lines into another backup files, suppose I have the following two files

[root@TENCENT64 /tmp/test]# cat test_ccd.log 
ccd0.log:01-28 09:33:18:461 1254 F NORMAL
ccd0.log:01-28 09:33:18:461 1254 F FATAL 
ccd0.log:01-28 09:34:30:827 1254 E ALARM
[root@TENCENT64 /tmp/test]# cat test_mcd.log 
mcd0.log:01-29 09:33:18:461 1254 F NORMAL
mcd0.log:01-29 09:33:18:461 1254 F FATAL 
mcd0.log:01-29 09:34:30:827 1254 E ALARM

and I want to generate the following backup files

[root@TENCENT64 /tmp/test]# cat test_ccd.log.txt
ccd0.log:01-28 09:33:18:461 1254 F FATAL 
ccd0.log:01-28 09:34:30:827 1254 E ALARM
[root@TENCENT64 /tmp/test]# cat test_mcd.log.txt
mcd0.log:01-29 09:33:18:461 1254 F FATAL 
mcd0.log:01-29 09:34:30:827 1254 E ALARM

I'd tried the following commands to achieve that purpose but neither of them are succeed, please anybody help to correct anything I did wrong?

[root@TENCENT64 /tmp/test]# cat /etc/redhat-release 
CentOS Linux release 7.2 (Final)

# this commands hangs for ever
[root@TENCENT64 /tmp/test]# find . -name "*.log" -type f -exec sh -c 'grep -E "ALARM|FATA:" 1>>"$0.txt"' {} \;

# this commands reports the error and the output was missed up
[root@TENCENT64 /tmp/test]# find . -name "*.log" -type f -exec sh -c 'grep -R -E "ALARM|FATA:" 1>>"$0.txt"' {} \;
grep: input file ‘test_mcd.log.txt’ is also the output
grep: input file ‘test_ccd.log.txt’ is also the output

# the output for each file is not correct
[root@TENCENT64 /tmp/test]# cat test_mcd.log.txt
test_mcd.log:mcd0.log:01-29 09:34:30:827 1254 E ALARM
test_ccd.log:ccd0.log:01-28 09:34:30:827 1254 E ALARM
[root@TENCENT64 /tmp/test]# cat test_ccd.log.txt
test_mcd.log:mcd0.log:01-29 09:34:30:827 1254 E ALARM
test_ccd.log:ccd0.log:01-28 09:34:30:827 1254 E ALARM
test_mcd.log.txt:test_mcd.log:mcd0.log:01-29 09:34:30:827 1254 E ALARM
test_mcd.log.txt:test_ccd.log:ccd0.log:01-28 09:34:30:827 1254 E ALARM

Also thing get complicated when I tried to use the following codes to persist the backed alarms to another location, I have a NFS share to store the backup alarms and each of them are stored separately into its folder (named with IP address) on the share, below is my command

# create the backed folder on the NFS share, like /data/backup/1.1.1.1/alarm_history
ip=`hostname -I|awk '{print $1}'` && mkdir -p /data/backup/${ip}/alarm_history

# iterate through /data/backup/1.1.1.1 to find all the alarms and persist into /data/backup/1.1.1.1/alarm_history, /data/backup is a shared base and 1.1.1.1 is mounted on the current server
ip=`hostname -I|awk '{print $1}'` && find /data/backup/${ip} -name '*.log' -type f -exec sh -c '
    grep -E "ALARM|FATAL" "$1" >> "/data/backup/${ip}/alarm_history/$1.alarm" && sort -o "/data/backup/${ip}/alarm_history/$1.alarm" -u "/data/backup/${ip}/alarm_history/$1.alarm"
' 'find-sh' {} \;

it yield two erros:

find-sh: line 1: /data/backup//alarm_history//data/backup/11.62.17.249/manager/manager_mcd0.log.alarm: No such file or directory
find-sh: line 1: /data/backup//alarm_history//data/backup/11.62.17.249/manager/manager_mcd0_stat.log.alarm: No such file or directory

they are two issue, 1, the ${ip} passed into the command "find" can not be recongized within the grep. 2. $1.alarm, the $1 is the absoulte pass in the grep, how can I get the basename of it?

Answer 1

The argument to grep is missing and : should be an L to get the expected result, thus

grep -E "ALARM|FATA:" 1>> "$0.txt"

should be

grep -E "ALARM|FATAL" "$0" > "$0.txt"

(or use >> or 1>> if you want to append the result to an existing file)

Since $0 should be the command name and positional parameters start with $1, you could add a name and pass the result as $1 to the script:

find . -name '*.log' -type f -exec sh -c '
    grep -E "ALARM|FATAL" "$1" > "$1.txt"
' 'find-sh' {} \;

Or loop the results using as few sh invocations as possible with {} +:

find . -name '*.log' -type f -exec sh -c '
    for file; do
        grep -E "ALARM|FATAL" "$file" > "${file}.txt"
    done
' 'find-sh' {} +

---

Edit:

I added a variable for the target directory which is passed as first argument to the script. To be able to sort the resulting logfile, I used a temporary file.

ip=$(hostname -I | awk '{ print $1 }')
targetdir="/data/backup/$ip/alarm_history"
mkdir -p "$targetdir"

find "/data/backup/$ip" -name '*.log' -type f -exec sh -c '
    targetdir=$1; shift

    for file; do
            # extract the log filename from the path, append ".alarm", prepend directory
            targetfile=${targetdir}/${file##*/}.alarm

            grep -E "ALARM|FATAL" "$file" >> "$targetfile" \
                && mv "$targetfile" "${targetfile}.tmp" \
                && sort -o "$targetfile" -u "${targetfile}.tmp" \
                && rm "${targetfile}.tmp"
    done
' 'find-sh' "$targetdir" {} +

Answer 2

This may be easier as a for loop

eg

for file in *.log
do
  grep -E -- "ALARM|FATAL" "$file" > "$file.txt"
done