7

I've been using x265 to encode some video on my workstation lately, but I have a problem: even though I launch it using nice -n 20 x265 to deprioritize it, it still slows the computer to a crawl while it's running. Everything still works, it's just... slow! I even see delays before the characters appear while typing in a terminal.

Do I have to live with this, or are there some other things I can try?

EDIT: Perhaps the following serves as proof that the nice value really is getting applied to x265? Look at the NI column.

% ps -awux -O nice | egrep "x265|PID"
USER      PID  %CPU %MEM     VSZ     RSS TT  STAT STARTED       TIME COMMAND            PID NI TT  STAT       TIME COMMAND
nobody  56654 789.3  3.7  785656  623384 11  SN+J 11:56PM    6:05.80 x265 --input-csp 56654 20 11  SN+J    6:05.80 x265 --input-csp i420 --bframes 5 -
  • 2
    This might be an I/O problem. Where does the data live? A fast SSD drive might help. – michas May 22 '15 at 5:20
  • I strongly suspect it is a CPU problem rather than IO. That is because I can reproduce the problem with a 4MB input video (about 60 frames). This entire video can be read from the SSD in well under a second. It takes much longer than that for x265 to encode the video (at less than one frame per second). And yet, the problem persists during the entire operation! If that's not enough, the videos (source and destination both) are stored on a dedicated SSD that is not used for anything else on the system. – gravitation May 22 '15 at 6:36
  • 2
    Does it maintain its nice level after you set it? How many threads does it launch and how does this compare to the number of cpu cores? If it starts so many threads or fork other processes. Nice might not help here... – Bichoy May 23 '15 at 8:04
  • 1
    Hey @BrandonThomson, check this Ubuntu question to get the number of threads: askubuntu.com/questions/88972/… – Bichoy May 25 '15 at 5:08
  • 3
    Also, check x265 documentation x265.readthedocs.org/en/latest/threading.html , you might just need to control the number of threads started by x265, say leave out one of your 8 cores for other applications ... – Bichoy May 25 '15 at 5:11
3

The FreeBSD kernel does implement something like an I/O scheduler, namely gsched. Reviewing the manual page, it seems to be a device-specific IO scheduler. Personally, I think it's a nice keynote toward realtime applications of FreeBSD, and a great reason to search for the existing FreeBSD documentation, candidly.

Albeit speculatively, perhaps with the root partition's block device configured to use an rr scheduler, using gsched, and the media files stored on a separate block device, perhaps it might serve to allow for the operating system to function more responsively, in I/O, even with the I/O bottleneck?

Perhaps, with gsched applied along with a configuration for processor priorities -- applying such as rtprio and/or idprio -- it might serve to improve the responsiveness of the primary operating system, even under heavy load from the media file processing.

Alternately, perhaps it might be possible to gain some more processor bandwidth, with ports compiled under CPU-specific optimizations. Towards that effect, there are the MACHINE_CPUARCH and CPUTYPE fields, such that may be applied in /etc/make.conf, and that would be applied during the ports build process [manual page]. The Handbook, of course, provides a lot of pointers about building ports with FreeBSD [ch.5]. Myself, I've been using MACHINE_CPUARCH?=amd64 and CPUTYPE?=core2 on an older Toshiba laptop. It seems to work out OK as a LAN gateway, though albeit I've not benchmarked it under high load in the processor or block I/O features.

2

Sometimes a single heavy I/O operation may affect kernel performance with respect to all I/O taks, including those not directly operating on the device of the first one.

  • The first and indirect way to control priorities on I/O scheduling is the tuning of process' nice level you already mentioned. In modern Linuxes, a process with a nice value of 19 (i.e., the maximum) is by default in the best effort class with priority equals to (19 + 20) / 5 = 7, which is the lowest priority available within the class. More generally, it ranges in [0,7] according such a mapping function.

  • A second, direct and more powerful way to control I/O scheduling is to manually intervene on the I/O scheduling class assigned to processes. This allow us to put a process also in two additional classes: real-time class, with higher priority than best effort level 0, and idle class, with lower priority than best effort level 7. This last theoretically guarantees that no other I/O operation could ever wait for a idle-scheduled process operation. Similarly to the nice command, the ionice allows one to spawn a process with a given priority or to change priorities of an existing process. More information about this tool and I/O scheduling on Linux kernels in general may be found in ionice man page.

That being said, have you tried to launch your process with ionice -c 3 x265 ...?

P.S. Sorry, I noticed the 'FreeBSD' tag in the question after posting my answer, which perhaps collapses into the following.

I don't think FreeBSD has any I/O scheduler. You may consider perform your work on a Linux box, which has it and it is quite is easy to play with it.

  • 1
    Not a worry mate, I am sure the details will be useful for many others who are browsing this thread =) – gravitation Jun 15 '15 at 22:11
0

You want "rctl"

see:

man rctl

which lets you allocate resources per user, per process, jail or login class. E.G.

# user root, maximum reads of 400 transfers per sec (tps) per the whole user :
rctl -a user:root:readiops:throttle=400/user

# user root, maximum reads of 30Mb per sec (31,457,280 bytes) per the whole user :
rctl -a user:root:readbps:throttle=31457280/user

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.