I've got a 720p video file that is about 20 minutes long. I want to compose each frame in the video using a single operator (e.g. lighten, darken, etc.) to produce a single image output.

My thought was that I could use Imagemagick's convert function to do this, using something like this:

convert video.mp4 -compose Lighten output.jpg

But after running for a few minutes it runs out of memory (and I've got a lot of memory and I've set policy to allow most of it to be used). Under the hood it appears to be calling ffmpeg, but whatever is happening it seems to be very inefficient, requiring tens of gigabytes of RAM for what should be loading a couple 720p frame into RAM at a time (one for the composite, and one for the new frame).

What's a quicker and more efficient way to compose video into an image?

  • Please detail how dit you set your policy to allow most of your RAM to be used, and how much RAM you do have even if it's "tens of gigabytes". – X.LINK Feb 22 at 0:43
  • @X.LINK I've got 128GB of RAM; I'm allowing convert half of it. The policy is irrelevant. I can extract video much faster than real time (source: having used ffmpeg for other things) and it is not necessary to buffer every frame in memory what is apparently what it is doing. There is no reason for it to be taken so much RAM when only two frames need to be in RAM at once. The naive usage is wrong so I need something that is less naive and uses resources commensurate with the actual complexity of the operation. – Michael Feb 22 at 0:59
  • 1
    Imagemagic's convert is a bit known for using a lot of RAM, even for seemingly simple tasks. But it does have a -limit switch: askubuntu.com/a/726827 which might help you, but also this: serverfault.com/questions/97340/… . I've asked about the policy because I myself am interested. – X.LINK Feb 22 at 15:01
  • @X.LINK Thanks... it seems my suspicion is correct about IM. I've tried finding a way to do this directly in ffmpeg, but so far my google-fu has failed me - if it's possible at all. I don't know if there is another Linux utility that would do this efficiently, but I'm hoping somebody will know. – Michael Feb 22 at 15:45

This can be done effectively by ffmpeg, although it requires knowing exactly how many frames need to be combined. A tblend filter must be set up to combine adjacent frames; each filter will divide the number of frames in half.

In my case, I had 72,000 frames (20 minutes @ 60fps). I picked 16 stages as this will divide the video down by 65536. The tblend documentation says:

By default, the output terminates when the longest input terminates.

From this, I would expect that I should divide by 131072, as the last division should result in a short input which should be combined with the full length one properly. However, using 17 stages results in the output Output file is empty, nothing was encoded so I dropped it to 16 at which point I got one image output.

The final command was:

ffmpeg -i $I -vf "tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,setpts=1*PTS" -f image2 $O.png

Where $I is the input file and $O is the input file stripped of its extension.

For comparison with convert, using ffmpeg using the above command on a 20 minute video file took slightly over a minute and negligible memory to process.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.