I'm looking for a method that reliably gives me a "mathematical distance" of two videos from one another. Similar to how the Levenshtein distance can be used to get the distance from a string to another. Is there a command or sequence of commands that provide me such. I consecutively want to automatically delete video duplicates that range over multiple video codecs and file types.
What I have
For example, I have a lot of videos that may only vary in codec (duplicates). The file formats range from webm, mp4, flv, mkv to avi. Comparing those videos with sha256sum does of course not work, similar to certain frames of the video which vary by slight changes in artifacts. One of my approaches included the usage of
ffmpeg to get pictures for every frame each x seconds, then use
compare from ImageMagic to get a numerical distance and use
awk to calculate the average of the total value over the amount of images. This has a few disadvantages that I would like to get rid of:
it only works for images
I have to manually tweak the threshold of the
comparemetric until I'm technically below that of artifacts to be confident enough to allow automatic deletions
I have to manually review every video with it's potential duplicates to know what the differences are (from start to finish)
In other words:
Is there a command, algorithm or tool that I can chain in a shell script to automatically delete video duplicates? or an improvement to my approach that spans over multiple video formats and which is less subjective? I would like this tool to be abstract enough (therefore rather a tool that calculates a mathematical index) to match future comparison tasks that involve around video contents. As I may not know, this could be a known problem and knowing it's name may help me find a tool.