3

Update

They don't! At least, not for me.

See my answer.

Original question

According to last year's Phoronix benchmarks, applications on FreeBSD mostly run slower than on Debian (including Stockfish chess engine, Node.js, FLAC encoding and other computational tasks).

Phoronix article itself attributes some of the performance differences to use of Clang instead of GCC compiler. Some other opinions say that use of ZFS makes FreeBSD slower, as ZFS is inherently slower than ext4.

But even purely computational tasks on FreeBSD compiled with GCC8 performed slower than on Linux.

What is the cause of that? Is it inherent to differences between FreeBSD and Linux kernels, might it be caused by worse quality of drivers or is there some other reason?

P.S. To make it more specific, here is a fairly simple purely computational program that runs slower on FreeBSD than on Linux according to Phoronix: m-queens 1.2. Compiled like this:

gcc -o m-queens.bin main.c -O2 -march=native -mtune=native -std=c99 -fopenmp

Since this a multithreaded task that was run on two 20 core CPUs, I suspect the performance difference boils down to how well OS handles multiple threads.

P.P.S. To make it more clear, I am aware that FreeBSD has good networking capabilities and that it is used by Netflix. The question is specifically about computational tasks, like the one above.

P.P.P.S. After installing FreeBSD (TrueOS) on my 6-core desktop alongside Ubuntu and trying to run the queens benchmark myself, I didn't notice any significant difference in multithreaded performance. While Phoronix claims that it ran 39% slower on FreeBSD, in my tests it was only 3.7% slower, which could be attributed to slight difference in compiler version (gcc 7.4 on TrueOS, gcc 7.2 on Ubuntu). I will test more later.

  • 2
    Note to people who may consider answering: Unless answered by someone with real in-depth knowledge of the two systems (and of the actual benchmarks used), this question may generate pure speculation and personal opinion. Let's try to avoid as much of that as possible. – Kusalananda Jul 12 at 7:44
  • 2
    I don't see why your question should be downvoted either. So I've upvoted it. If you care enough, you should be able to answer the question yourself: boot the same machine into FreeBSD and into Linux. Take one of the computational tasks that is slower on FreeBSD and start disecting it into smaller parts until you have only the few lines left that behave differently. A few things that might cause the difference between the OSes: CPU cache management, system call overhead, slightly different function call conventions, scheduler overhead, security (stack canaries, ASLR, page zeroing, etc.) – Tomáš Pospíšek Jul 12 at 8:07
  • 1
    Take benchmarks with a grain of salt. Often what it matters in the real world does not translate directly in benchmarks be it user friendliness, capability of scaling, support and politics. The question in itself is interesting, but might be too broad as it is. I would look a bit into Netflix and why they go with FreeBSD, for examples in the real world. – Rui F Ribeiro Jul 12 at 8:20
  • 1
    Too board. Does program run slower on FreeBSD? That's actually non-sense without mentioning what program we're referring to. The same piece of C/Rust/Go or whatever language will compiled to different machine code on different platform, so they doom to have performance difference. So-called purely computational tasks usually also contain IO and OS task scheduling, which is platform dependent. It is simply not possible that every piece of code run slower on FreeBSD. – 炸鱼薯条德里克 Jul 12 at 8:30
  • 2
    @AndriyMakukha Far more than the network. It scales way much better under very heavy loads. – Rui F Ribeiro Jul 14 at 16:49
5

So many downvotes stimulated me to install FreeBSD (TrueOS) on my 6-core desktop computer to test it myself. (NOTE: I do not recommend trying to install TrueOS alongside other operating systems, because this installation wiped one of my hard drives, even though I tried to install it on a USB drive... Not a user-friendly experience.)

As a result, after running some tests from the Phoronix test suite on both Ubuntu and FreeBSD, I couldn't see the “slow applications on FreeBSD” effect. Quite the contrary, some applications ran significantly (10–25%) faster on FreeBSD:

    Test                                    FreeBSD 13        Ubuntu 17
Fhourstones, kpos/s                       16753             13336
m-queens, multithreaded, user time, s     18.08             17.38
7zip 1 GB text file, user time, s         994               1096

As you can see, the only task that performed slower on FreeBSD was multithreaded N queens problem, taking 3.7% more time than on Ubuntu.

Potential pitfalls:

  • gcc on Ubuntu was version 7.2, on FreeBSD – 7.4
  • Ubuntu was running with KDE, FreeBSD in a shell (shouldn't make much difference)
  • Phoronix used 80-thread server, I used 6-thread Intel i5 computer.

In conclusion, when testing OS performance, you should:

  • run benchmarks on your setup yourself instead of trusting results that were obtained by someone else.
  • try to use the same compiler.
  • beware that performance of scripting languages like Perl and Python are not a good indicator of OS performance, since different installations of the interpreters behave differently.
  • Are those averaged across multiple runs? How many? – muru Jul 13 at 4:42
  • @muru, three runs, best performance reported. In all cases the difference between runs was insignificant. – Andriy Makukha Jul 13 at 4:45
  • Anecdotal cautionary tale, they dub it "Moronix" for some reason. While useful, take what you read there with a pinch of salt. Upvoted both answer and question for you taking the trouble to investigating the issue. jonimoose.net/2013/… and lxer.com/module/forums/t/31292 – Rui F Ribeiro Jul 14 at 16:53
  • 1
    (coming back: see Brendan Gregg or the DragonFlyBSD site for benchmarking done right) – Rui F Ribeiro Jul 14 at 17:00
  • 1
    @RuiFRibeiro, wow! This controversy with FreeBSD and Phoronix goes back like 10 years in time! However, if Phoronix gets these bad results for FreeBSD so consistently, doesn't that mean that there is at least partial truth to these allegations? I would be very interested to see a new modern "anti-Phoronix" benchmark that would either explain why Phoronix gets such bad results for FreeBSD or just show that FreeBSD doesn't perform inferior when retested in similar setting. – Andriy Makukha Jul 15 at 8:41
2

I see, the benchmark involves using OpenMP. This is where difference may come from. Check what OpenMP runtime library is being used.

Another thing worth looking at - disassembly code or intermediate representation. It can allow you to quickly find differences in instructions being executed.

To sum this up, there is no quick and definitive answer to this, and you have to do some research.

  • Yeah, I'll research. I hoped that someone had an answer already, because this is only one of multitude tasks that seem to run slower on FreeBSD. – Andriy Makukha Jul 12 at 10:10
  • I also heard controversial opinions on Phoronix benchmarks quality. – arrowd Jul 12 at 10:14
  • But what is the controversy? I tried to find explanation on FreeBSD forums, but couldn't find anything of substance. – Andriy Makukha Jul 12 at 10:16
  • For example, here is an independent benchmark that comes to a similar conclusion: PostgreSQL benchmark on FreeBSD, CentOS, Ubuntu Debian and openSUSE. PostgreSQL works slower even for read-only operations. – Andriy Makukha Jul 12 at 10:19
  • @AndriyMakukha "Seems to run slower"? Are you referring to the benchmark results or to something you are running? – Kusalananda Jul 12 at 13:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.