Re-encoding video library in x265 (HEVC) with no quality loss

Question

I am trying to convert my video library to HEVC format to gain space. I ran the following command on all of the video files in my library:

#!/bin/bash
for i in *.mp4;
do 
    #Output new files by prepending "X265" to the names
    avconv -i "$i" -c:v libx265 -c:a copy X265_"$i"
done

Now, most videos convert fine and the quality is the same as before. However, a few videos which are of very high quality (e.g. one movie print which is of 5GB) loses quality -- the video is all pixelated.

I am not sure what to do in this case. Do I need to modify the crf parameter in my command line? Or something else?

The thing is, I am doing a bulk conversion. So, I need a method where avconv automatically adjusts whatever parameter needs adjustment, for each video.

UPDATE-1

I found that crf is the knob I need to adjust. The default CRF is 28. For better quality, I could use something less than 28. For example:

avconv -i input.mp4 -c:v libx265 -x265-params crf=23 -c:a copy output.mp4

However, the problem is that for some videos CRF value of 28 is good enough, while for some videos, lower CRF is required. This is something which I have to check manually by converting small sections of the big videos. But in bulk conversion, how would I check each video manually? Is their some way that avconv can adjust CRF according to the input video intelligently?

UPDATE-2

I found that there is a --lossless option in x265: http://x265.readthedocs.org/en/default/lossless.html.

However, I don't know how to use it correctly. I tried using it in the following manner but it yielded opposite results (the video was even more pixelated):

avconv -i input.mp4 -c:v libx265 -x265-params lossless -c:a copy output.mp4

Answer 1

From my own experience, if you want absolutely no loss in quality, --lossless is what you are looking for.

Not sure about avconv but the command you typed looks identical to what I do with FFmpeg. In FFmpeg you can pass the parameter like this:

ffmpeg -i INPUT.mkv -c:v libx265 -preset ultrafast -x265-params lossless=1 OUTPUT.mkv

Most x265 switches (options with no value) can be specified like this (except those CLI-only ones, those are only used with x265 binary directly).

With that out of the way, I'd like to share my experience with x265 encoding. For most videos (be it WMV, or MPEG, or AVC/H.264) I use crf=23. x265 decides the rest of the parameters and usually it does a good enough job.

However often before I commit to transcoding a video in its entirety, I test my settings by converting a small portion of the video in question. Here's an example, suppose an mkv file with stream 0 being video, stream 1 being DTS audio, and stream 2 being a subtitle:

ffmpeg -hide_banner \
-ss 0 \
-i "INPUT.mkv" \
-attach "COVER.jpg" \
-map_metadata 0 \
-map_chapters 0 \
-metadata title="TITLE" \
-map 0:0 -metadata:s:v:0 language=eng \
-map 0:1 -metadata:s:a:0 language=eng -metadata:s:a:0 title="Surround 5.1 (DTS)" \
-map 0:2 -metadata:s:s:0 language=eng -metadata:s:s:0 title="English" \
-metadata:s:t:0 filename="Cover.jpg" -metadata:s:t:0 mimetype="image/jpeg" \
-c:v libx265 -preset ultrafast -x265-params \
crf=22:qcomp=0.8:aq-mode=1:aq_strength=1.0:qg-size=16:psy-rd=0.7:psy-rdoq=5.0:rdoq-level=1:merange=44 \
-c:a copy \
-c:s copy \
-t 120 \
"OUTPUT.HEVC.DTS.Sample.mkv"

Note that the backslashes signal line breaks in a long command, I do it to help me keep track of various bits of a complex CLI input. Before I explain it line-by-line, the part where you convert only a small portion of a video is the second line and the second last line: -ss 0 means seek to 0 second before starts decoding the input, and -t 120 means stop writing to the output after 120 seconds. You can also use hh:mm:ss or hh:mm:ss.sss time formats.

Now line-by-line:

1. -hide_banner prevents FFmpeg from showing build information on start. I just don' want to see it when I scroll up in the console;
2. -ss 0 seeks to 0 second before start decoding the input. Note that if this parameter is given after the input file and before the output file, it becomes an output option and tells ffmpeg to decode and ignore the input until x seconds, and then start writing to output. As an input option it is less accurate (because seeking is not accurate in most container formats), but takes almost no time. As an output option it is very precise but takes a considerable amount of time to decode all the stream before the specified time, and for testing purpose you don't want to waste time;
3. -i "INPUT.mkv": Specify the input file;
4. -attach "COVER.jpg": Attach a cover art (thumbnail picture, poster, whatever) to the output. The cover art is usually shown in file explorers;
5. -map_metadata 0: Copy over any and all metadata from input 0, which in the example is just the input;
6. -map_chapters 0: Copy over chapter info (if present) from input 0;
7. -metadata title="TITLE": Set the title of the video;
8. -map 0:0 ...: Map stream 0 of input 0, which means we want the first stream from the input to be written to the output. Since this stream is a video stream, it is the first video stream in the output, hence the stream specifier :s:v:0. Set its language tag to English;
9. -map 0:1 ...: Similar to line 8, map the second stream (DTS audio), and set its language and title (for easier identification when choosing from players);
10. -map 0:2 ...: Similar to line 9, except this stream is a subtitle;
11. -metadata:s:t:0 ...: Set metadata for the cover art. This is required for mkv container format;
12. -c:v libx265 ...: Video codec options. It's so long that I've broken it into two lines. This setting is good for high quality bluray video (1080p) with minimal banding in gradient (which x265 sucks at). It is most likely an overkill for DVDs and TV shows and phone videos. This setting is mostly stolen from this Doom9 post;
13. crf=22:...: Continuation of video codec parameters. See the forum post mentioned above;
14. -c:a copy: Copy over audio;
15. -c:s copy: Copy over subtitles;
16. -t 120: Stop writing to the output after 120 seconds, which gives us a 2-minute clip for previewing trancoding quality;
17. "OUTPUT.HEVC.DTS.Sample.mkv": Output file name. I tag my file names with the video codec and the primary audio codec.

Whew. This is my first answer so if there is anything I missed please leave a comment. I'm not a video production expert, I'm just a guy who's too lazy to watch a movie by putting the disc into the player.

PS. Maybe this question belongs to somewhere else as it isn't strongly related to Unix & Linux.

Answer 2

I've recently gone through the trouble of Transcoding my whole video catalog over to HEVC. I use https://github.com/FallingSnow/h265ize with the following settings.

h265ize -v -m medium -q 20 -x --no-sao --aq-mode 3 --delete --stats

-v - Verbose Output
-m medium - Medium encode speed (smaller higher quality, anything slower I find is not worth the time/quality dif)
-q 20 - the CRF used, 20 is similar to 18 or so in x264 but hey. This is for 1080p content (90% of my TV) I tend to use 22 for my 4K movies
-x - Use x265 central defined commands
--no-sao turns off Sample Adaptive Offset (improves speed of encode)
--aq-mode 3 - use Adaptive Quantisation with auto variance, helps 8bit encodes, especially in dark areas, stops most of the banding that can happen (at expense of encode time though)
--delete - replace encoding file with encoded file (test before using this one)
--stats - Write stats out to a csv file in the root of the path you ran from.

Encode speeds are around 30fps (for most 1080p stuff) on my rig. Dual Xeon E5 2687W v2, but I force the FFMPEG process to not use the first side of one of the processors (It's my Plex server, so have to make sure there's overhead for transcode if needed on playback etc)

Yes it took a while to convert most of it over, and now I have a scheduled task that runs twice a day to encode the stuff from that day over to x265.

The space savings have been enormous. My initial SAN was at 20Tb use, now it's around 12 but obviously has been added too with 6 months more content.

I've started to transcode all my Movies over too, however, that's an ongoing process, as I have to ID quality levels (Radarr fortunately labels then nicely) and use one of three transcode settings:

-m slower -q 18 -x --no-sao --aq-mode 3 for 720p transcodes
-m medium -q 20 -x --no-sao --aq-mode 3 for 1080p
-m medium -q 22 -x --no-sao for 2160p

Hope that helps some people. Shout if anyone needs a hand setting it all up. And before you encode everything to x265, think about playback, if the client doesn't support x265 native, then the transcade can be expensive in terms of CPU and Quality.

Answer 3

The correct syntax to enable lossless mode for x265 encoder in ffmpeg is -x265-params lossless=1 (you need to append =1).

However, for lossless coding there are better codec choices. I found by testing that FFV1 compresses vastly better (file size = ~80% of x265) at least on some kinds of video (if best settings are chosen for both codecs). And it also works faster, and (AFAIK) is not encumbered by patents. That is, it's superior to lossless H.265 in every way for video archiving. The compromise however is compatibility with current playback software and hardware.