File deletion under special conditions, recursively if possible

Question

I am looking for a unix command sequence, which is able to perform a quite complex, conditional deleting action.

All files with the extensions "[NAME].cut" and "[NAME].cut.bak" shall be deleted, IF there exists no corresponding file named "[NAME].rec" or "[NAME].mpg" in the same folder.

The command shall be run on an integrated device with some minimal kernel, and therefore not use "special tools". e.g. 'ls' and 'rm' can be used, 'find' is not present (i.e. only via an external tool, which I would prefer not to depend on)

If possible, the command shall delete recursively in subdirectories as well...

Background is following: I am developing a tool for a Linux-based PVR, that allows cutting of recorded programmes. For each recording (extension .rec or .mpg) the segment markers are stored in a .cut-file. When some recording gets moved/renamed/deleted, then the corresponding cut-file remains on the disk useless. I already have implemented the removing of those useless cut-files in C. But I am wondering, whether there may be a (simple) system based solution. In this case it could be run via 'system' and & in the background, which would make my application more responsive...

Here are the commands, which are included within the PVR's firmware:

affinity, arp, awk, basename, bash, busybox, cat, chgrp, chmod, chown, cmp, cp, cramfsck, cut, date, dd, df, dirname, dmesg, du, echo, egrep, env, expr, false, fgrep, flash_erase, flash_eraseall, flash_info, free, getty, grep, head, hexdump, hostname, id, ifconfig, install, kill, killall, ln, logger, login, ls, md5sum, mkcramfs, mkdir, mknod, mktemp, more, mount, mv, nice, norcleanmark, od, passwd, pidof, ping, pmtest, portmap, printenv, printf, ps, pwd, readlink, renice, rm, rmdir, route, sed, seq, sh, shutdown, sleep, sort, strings, stty, sync, tail, tee, telnet, tftp, time, tinylogin, touch, tr, true, tty, twin, umount, uname, uniq, usleep, vi, wc, which, whoami, xargs, yes

Do you have any tips for me?

Answer 1

I couldn't keep this simpler; this works but it assumes there are no files whose filename contains newlines in the target directory; first test the command using this:

find . -type f \( -name "*.cut" -o -name "*.cut.bak" \) -exec bash -c '[ -f "$(<<< "{}" sed "s/\(.*\/[^.]*\).*/\1/").rec" -o -f "$(<<< "{}" sed "s/\(.*\/[^.]*\).*/\1/").mpg" ] && echo "{}"' \;

If the files listed are those expected to be deleted, you can go ahead and run this:

find . -type f \( -name "*.cut" -o -name "*.cut.bak" \) -exec bash -c '[ -f "$(<<< "{}" sed "s/\(.*\/[^.]*\).*/\1/").rec" -o -f "$(<<< "{}" sed "s/\(.*\/[^.]*\).*/\1/").mpg" ] && rm "{}"' \;

Test on a directory hierarchy created ad hoc:

~/tmp$ tree
.
└── dir1
    ├── file1.cut
    ├── file1.cut.bak
    ├── file1.rec
    ├── file2.cut
    ├── file2.cut.bak
    ├── file2.mpg
    ├── file3.cut
    ├── file3.cut.bak
    └── subdir1
        ├── file1.cut
        ├── file1.cut.bak
        ├── file1.rec
        ├── file2.cut
        ├── file2.cut.bak
        ├── file2.mpg
        ├── file3.cut
        └── file3.cut.bak

2 directories, 16 files
~/tmp$ find . -type f \( -name "*.cut" -o -name "*.cut.bak" \) -exec bash -c 'if [ -f "$(<<< "{}" sed "s/\(.*\/[^.]*\).*/\1/").rec" -o -f "$(<<< "{}" sed "s/\(.*\/[^.]*\).*/\1/").mpg" ]; then rm "{}"; fi' \;
~/tmp$ tree
.
└── dir1
    ├── file1.rec
    ├── file2.mpg
    ├── file3.cut
    ├── file3.cut.bak
    └── subdir1
        ├── file1.rec
        ├── file2.mpg
        ├── file3.cut
        └── file3.cut.bak

2 directories, 8 files

As you can see, all files with extension .cut or .bak for which a file with the same name and extension .rec or .mpg exists are deleted recursively (file1.cut and file1.cut.bak are deleted because of file1.rec, file2.cut and file2.cut.bak are deleted because of file2.mpg; file3.cut and file3.cut.bak are not deleted because there's no file3.rec or file3.mpg in the same directory)

Answer 2

If the command is to be run on some small embedded device with very limited commands and shells (e.g. missing find, minimal shell à la sash), a possible approach would be to write some tiny C (or perhaps C++) program doing that, and compile that program to an executable (it would be nice to make that program a free software).

You'll use nftw to recursively traverse the file tree, and build an array (or a vector, or a list) of paths to be removed. Then a second pass would effectively remove the files. Relevant functions are nftw(3) (to traverse the file tree), snprintf(3) (to construct file names), access(2) (to test existence of file), stat(3) (to get type, size and other meta-data about a file), remove(3) (to remove a file), basename(3), etc.

If you have GNU find, you might make a shell script with it (but details could depend upon your actual shell, especially if it is small enough to not be POSIX compliant). Collect the filenames to be deleted in some temporary file. Then remove them later.

If you insist on using shell solutions, and assuming that GNU find is available, you might have several shell scripts:

First, a shell script (invoked from outside), let's name it clean-the-mess.sh which would create a temporary file, then invoke another internal shell script consider-cleaning.sh for every .cut & .cut.bak file:

#!/bin/sh
## untested clean-the-mess.sh script
## a temporary file listing files to remove
cleanlistfile=$(tempfile -s .cleanlist)
## on exit, or SIGQUIT, SIGTERM, or SIGINT signal, remove it
trap "rm $cleanlistfile" EXIT QUIT TERM INT
## find every file name .cut | .cut.bak; run consider-cleaning.sh on it
find -type f \( -name '*.cut' -o 'name '*.cut.bak' \) \
    -exec consider-cleaning.sh '{}' $cleanlistfile \;
## remove the collected file list
for f in $(cat $cleanlistfile) ; do rm $f ; done

^{That won't work if some files have spaces in their name. Perhaps consider xargs(1)}

Then you need to code the internal consider-cleaning.sh script, which adds file paths to the list of files to be cleaned. It could be

#!/bin/sh
# internal untested consider-cleaning.sh script
file=$1
collectlist=$2
case $file in 
 *.cut) 
   bas=$(basename $file .cut)
   if [ ! -f "$bas.rec" -a ! -f "$bas.mpg" ]; then
      echo $file >> $collectlist
   fi
 ;;
 *.cur.bak)
   bas=$(basename $file .cut.bak)
   if [ ! -f "$bas.rec" -a ! -f "$bas.mpg" ]; then
      echo $file >> $collectlist
   fi
 ;;
 esac

My scripts are not proven, untested, so are probably wrong. Put #!/bin/sh -vx as their first line to test & debug them (you'll get some execution trace)

Alternatively, embed a tiny Lua interpreter with the few basic primitive file related operations, and write your script in Lua. Lua is very easily embeddable, and can be made to be a tiny interpreter.