1

I noticed if I extract frames with this command:

ffmpeg -i sample_nosound.mp4 $filename%03d.jpg

It will extract depends on fps by default. ffmpeg -i sample_nosound.mp4 shows this video has 6 fps, so it extracted 1630 jpg frame files, which 1630/6 = 271.6 seconds equivalent to the 4:32 total video duration.

But 1630 jpg frames total size is 13 MB:

$ du -h extracted_jpg_folder
13M      extracted_jpg_folder

, while the file size of the mp4 is 1.8 MB, far lower than the total frames size:

$ ls -la sample_nosound.mp4
-rw-rw-r-- 1 xiaobai xiaobai 1814889 Feb  13 15:42 'sample_nosound.mp4'

That means ffmpeg extract frames by referring fps info with duplicated frames.

Therefore my question is, how to make ffmpeg extract frames by "stored frames" without rely on fps ?

I expect I can get total frames size which almost equivalent with the mp4 files size.

I don't expect exact match file size, since mp4 can contains some metadata.

Output of ffprobe -i sample_nosound.mp4:

ffprobe version 3.4.4-0ubuntu0.18.04.1 Copyright (c) 2007-2018 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-16ubuntu3)
  configuration: --prefix=/usr --extra-version=0ubuntu0.18.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  WARNING: library configuration mismatch
  avcodec     configuration: --prefix=/usr --extra-version=0ubuntu0.18.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared --enable-version3 --disable-doc --disable-programs --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'sample_nosound.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.83.100
  Duration: 00:04:32.00, start: 0.000000, bitrate: 53 kb/s
    Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, bt470bg/bt709/bt709), 640x330 [SAR 1:1 DAR 64:33], 53 kb/s, 6 fps, 6 tbr, 12288 tbn, 12 tbc (default)
    Metadata:
      handler_name    : VideoHandler
  • 1
    You seem to assume that your MP4 container yields a sequence of JPEGs. This would be true if the video was compressed with MJPEG. However, h264 is very common in MP4 containers. What does ffprobe -i sample_nosound.mp4 say? – Hermann Feb 14 at 15:27
  • @Hermann Thanks, I've attached the output. – 林果皞 Feb 14 at 15:32
2

As far as I understand the question, you want to extract the frames from the video. Each frame should be stored a separate file. The sum of all file's sizes is expected to match the video's file size. This is only true for some specific videos. I'll try to explain things broadly.

TL;DR

Extracting frames from h264 encoded video with the same visual quality and file-size is not possible.

Video container formats are confusing

The video file in this case is an MP4 file. MP4 is a container for video data. The type of the container however does not really say anything about the actual content. In fact, many different kinds of video formats can reside inside a MP4 file – just like a zip archive (or a PDF file).

There are different kinds of video

A video is a sequence of images. There are many ways to store these images into a video stream (encode) and how to read them afterwards (decode). The algorithms are typically referred to as codecs.

Keep in mind that not all codecs do compression. In this example, h264 is the codec. By default, the h264 encoder calculates the difference from one frame to the next one. In case of a small difference, the encoder stores the difference only. The actual frame is discarded. Only the first¹ frame is stored as a complete image. This saves much space and is one of the compression strategies. The h264 decoder will apply the stored difference to the previous frame, recreating the original.

As you see, the frames in your video are depending on each other. If you want single files, you want them to be independent. This means you always need to store the complete information for each single frame. This means, you cannot simply take the existing data and copy them to files, but you must re-encode the video. Along the way, the sum of filesizes must increase.

You can read of the various picture types in video compression, specifically the "difference-based" inter-frames or an overview of video compression in gerneral.

h264 is not JPEG

Even if we are talking of individual images. JPEG uses a compression method known as DCT. H.264 uses a similar, but improved version. This means JPEG cannot possibly compress as efficient as h264. By the way, you can put a h264 compressed image into a file using HEIF (This essentially behaves like a one-frame video).


¹This is not entirely true, but I want to keep it simple for now. It is actually more like "the first frame of a scene". If you want to know the details:
The encoder notices the start of individual scenes (in cinematography, this is usually called a "cut"). The difference from one frame to another is very high and therefore not good to compress. The encoder decides not to use a "difference-based" inter-frame. Instead, it uses the complete picture (this is called an "intra-frame", also known as "key-frame").
There also is a technical reason: Only to intra-frames you may jump quicky when seeking through the video. Consequently, intra-frames are also put into the stream every now and then (regardless of the actual video content). Commonly, a video has one intra-frame per second.

Now we learned much about video compression. This video demonstrates some of the things:
Interesting ouput of damaged video file
Due to file corruption, this video lost an intra-frame. The decoder more or less successfully manages to play it back. The lost frame probably showed the woman looking to the side. Now she turns her head back, the decoder only has the data from inter-frames which include some movement information. It looks like the woman ended up with her face on the side of her head. Meanwhile, a person walks through the background. This person was not present in the lost intra-frame and therefore looks pretty okay.

  • Thanks, now I understand why download some streaming in half way is not possible to recover without original frame. But can you give more info or reference link in which cases Only the first¹ frame is stored as a complete image is not true ? – 林果皞 Feb 14 at 18:52
  • 1
    Glad to be of service. I expanded upon my anwser. – Hermann Feb 14 at 19:33
  • Thanks a million, seeking concerns is interesting. – 林果皞 Feb 14 at 19:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.