1

I'm trying to use ffmpeg's signature function to perform a duplicate analysis on several thousand video files that are listed in the text file vids.list. I need to have it so that every file is compared with every other file then that line of the list is removed. The following is what I have so far:

#!/bin/bash

home="/home/user/"
declare -i lineno=0

while IFS="" read -r i; do
    /usr/bin/ffmpeg -hide_banner -nostats -i "${i}" -i "$(while IFS="" read -r f; do
        echo "${f}"
    done < ${home}/vids.list)" \
    -filter_complex signature=detectmode=full:nb_inputs=2 -f null - < /dev/null
    let ++lineno
    sed -i "1 d" ${home}/vids.list
done < vids.list 2> ${home}/out.log

ffmpeg is outputting a "too many arguments" because the inside while loop is dumping all the filenames into the second -i. I'm not sure if I need a wait somewhere (or a formatting option) to hold to loop open while the top while loop finishes. Just to clarify, I would need the loop to start at line 1 of the text file with paths, compare that file with the file from line 2, 3, 4...2000 (or whatever), remove line 1, and continue.

1

Sidestepping the exact command, I take it you want something like this (with the obvious four-line input)?

$ bash looploop.sh 
run ffmpeg with arguments 'alpha' and 'beta'
run ffmpeg with arguments 'alpha' and 'charlie'
run ffmpeg with arguments 'alpha' and 'delta'
run ffmpeg with arguments 'beta' and 'charlie'
run ffmpeg with arguments 'beta' and 'delta'
run ffmpeg with arguments 'charlie' and 'delta'

We already know how to make a loop, so let's just add another, nested inside the first. That by itself would match all input lines with themselves and all pairs twice, so count the lines to skip the pairs that will already have been processed.

#!/bin/bash

i=0
while IFS= read a; do 
        i=$((i + 1))
        j=0
        while IFS= read b; do
                j=$((j + 1))
                if [ "$j" -le "$i" ]; then continue; fi

                # insert the actual commands here
                printf "run ffmpeg with arguments '%s' and '%s'\n" "$a" "$b"
        done < vids.list
done < vids.list

Or like you did, removing the lines as they are processed by the outer loop, this is actually shorter:

#!/bin/bash
cp vids.list vids.list.tmp
while IFS= read a; do 
        while IFS= read b; do
                if [ "$a" = "$b" ]; then continue; fi
                # insert the actual commands here
                printf "run ffmpeg with arguments '%s' and '%s'\n" "$a" "$b"
        done < vids.list.tmp
        sed -i '1d' vids.list.tmp
done < vids.list.tmp
rm vids.list.tmp

I'm not sure what exactly causes "too many arguments" in your script, but the argument to -i is a double-quoted string with just a command substitution inside, so it will be passed as a single argument to ffmpeg (with the newlines from the echo embedded). It shouldn't result in too many arguments.

  • Yup that does seems to work nicely, thanks! – bcHelix Aug 5 at 19:02
0

What about this:

readarray -t arr < file
counter=1
for i in "${arr[@]}"; do 
  for k in "${!arr[@]}"; do 
    if [[ ! -z "${arr[$k+$counter]}" ]]; then 
      f1="${i}"
      f2="${arr[$k+$counter]}"
      ffmpeg -hide_banner -nostats -i "${f1}" -i "${f2}" -filter_complex signature=detectmode=full:nb_inputs=2 -f null - < /dev/null
    else
      break
    fi
  done
  (( counter++ ))
done
  • @ikkachu if you don't mind, what do you think of this approach? – guillermo chamorro Aug 5 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.