6

I have a bunch of videos which I want to check if they are complete or not. Some of them may be downloaded partially, but they are not faulty. How can I efficiently check if these video are completely downloaded?

If I had the links, I would have checked the size of them, but I don't.

I tried to use ffprobe and mediainfo. ffprobe reports minor problems on partially downloaded files, but it also reports similar problems with some of completely downloaded files. Should I use ffmpeg to read the whole files and compare the length of the videos to check if they are downloaded? Is there a better solution?

  • Are you talking about files that are currently being downloaded, i.e. the program that's downloading them is still writing to the file as it receives more data? Or are you talking about files whose download was aborted at some point in the past? – Gilles Mar 12 '16 at 12:27
  • @Gilles The are downloaded in the past, and now some of these files are incomplete. I want to find incomplete video files, so that I can manually find the links and resume the download. – Ho1 Mar 12 '16 at 12:32
  • @Ho1 did my answer satisfy your question? – J363 May 5 '16 at 0:40
  • @J363 Thanks for your answer. Could you please respond my comment? – Ho1 May 7 '16 at 8:03
9
+25

ffmpeg is an OS agnostic tool that is capable of determining if a video file has been completely downloaded. The command below instructs ffmpeg to read the input video and encode the video to nothing. During the encoding process, any errors such as missing frames are output to test.log.

ffmpeg -v error -i FILENAME.mp4 -f null - 2>test.log

If a video file is not totally downloaded, there will be many lines in the test.log file. For example, .1 MB missing from a video file produced 71 lines of errors. If the video is fully downloaded and hasn’t been corrupted, no errors are found, and no lines are printed to test.log.


Edit

In the example I gave above, I tested the entire file because the test video I downloaded was a torrent, which can have missing chunks throughout the file.

Adding -sseof -60 to the list of arguments will check the last 60 seconds of the file, which is considerably faster.

ffmpeg -v error -sseof -60 -i FILENAME.mp4 -f null - 2>test.log

You'll need a newer version of ffmpeg, 2.8 was missing the sseof flag, so I used 3.0.

  • 1
    This works somehow, but it requires processing all the video. As an alternate method, I found that opening the video in VLC, clicking somewhere near the end takes less time and is more practical. Is it possible to define similar thing in ffmpeg? I mean seeking to 99% without processing all the video, and checking if it works fine without error. – Ho1 May 7 '16 at 8:02
  • @Ho1 see the updated command. – J363 May 8 '16 at 12:41
  • 1
    I am positive the command works because I tested it several times with and without the -sseof flag. The original command checks the whole file, which is the suggested behavior. The amended command you requested asserts the assumption that data is missing from the end of the file into the command, which may not be the case. My test file is 99.9% of a 137MB mp4 downloaded with a torrent client. When I use the -sseof flag and progressively increase the amount of time it looks backwards, I eventually get the exact same output as if I ran the command searching the entire file. – J363 May 11 '16 at 7:40
  • 1
    With -ss and -sseof, ffmpeg does read and process the whole file. So, anyway, opening the video in VLC and clicking near the end of it would be faster. – Ho1 May 12 '16 at 14:54
  • 1
    You're right, but I am downloading over http. – Ho1 May 30 '16 at 7:33
6

MediaInfo displays a line:

Truncated: Yes

if a file is not complete as expected by the format specifications.

As there is technically no difference between a file wrongly (not meeting the specs about file boundaries) muxed and partially downloaded files, it is technically impossible to do the difference between a buggy file and a partially downloaded file.

Another (advanced) test could be done e.g. reading the index of an .mp4 file and checking that the file offset + frame size of the last frame is within the file size of the file you get, but it is not exactly what your are looking for (if there is metadata e.g. posters in the source file, at the end of the file, and the file is truncated just before this poster, the partial download is still not detected in every case). It is not implemented in MediaInfo but you can add a MediaInfo feature request.

In all cases, it is very difficult to detect all partial downloads, because the total file size in not indicated in most video file formats so you can be sure a file is truncated, but you can not be sure a file is not truncated. The only way to be sure you downloaded the complete file is to get the file size from somewhere else (and better: have its hash e.g. MD5).

PS: this question is not specific to any operating system.

  • Thanks for your answer. I don't want an exact answer, an approximate answer would be enough for me. Do you think this question should be moved to superuser.com? – Ho1 Mar 14 '16 at 13:03
  • I couldn't get such field even by using mediainfo version v0.7.83 using -f (full information display). Could you please help on getting Truncated field? – Ho1 Mar 14 '16 at 13:17
  • Just remove 1 byte in your file and you should have this "truncated" field displayed (no need of -f, default output is enough). FYI it is implemented for the most used formats today: MP4/MOV, Matroska, MXF. are you testing with another format? – Jérôme Martinez Mar 14 '16 at 16:45
  • 1
    Thanks a lot for the answer. To check many files at once, I used the following: mediainfo * | grep -E "^(IsTruncated|Complete name)" where the filename of the corresponding file is being displayed directly above the "IsTruncated: Yes" line. – 0x01 May 13 '18 at 11:47
2

I had a file where ffmpeg (v3.1.1) with -sseof -60 did not show any warnings and just exited with no indication that something was wrong. Mediainfo did not show the file was truncated either. Only leaving out the -sseof -60 would report any error with ffmpeg. So I went back to trying ffprobe. It seemed slightly faster than ffmpeg.

ffprobe -v error -count_frames -i filename.mp4

produced this output

[h264 @ 00000000004e6a60] Invalid NAL unit size.
[h264 @ 00000000004e6a60] Error splitting the input into NAL units.
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000004e5280] stream 1, offset 0x1350135: partial file

I did not yet encounter any false positives (as reported by the OP).

Edit: ffprobe was on my testfiles about 10% faster than ffmpeg, but only if you instruct it to use all cores with the option '-threads 0'. Otherwise it uses only one core/thread and is slower.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.