Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

As of november 2010, Linux is used on 459 out of the 500 supercomputers of the TOP500.

What are the reasons behind this massive use of Linux in the supercomputer space?

share|improve this question
3  
And 19 more are Unix and 16 mixed leaving 1 BSD and 5 Windoze :) –  Caleb Jun 4 '11 at 21:57
1  
Some more reasons. –  Tshepang Jun 4 '11 at 22:21
20  
'Why is air commonly used for breathing?' I am amazed that anyone would want to build a supercomputer and then put Windows on it. What are the reasons for that? A really big Excel spreadsheet? Millions of layers in Photoshop? Quickly scanning pron collection with Norton Anti-virus? Playing Crysis with all options on? –  Mathew Jun 5 '11 at 1:37
5  
@Mathew Probably that last one. –  Maxpm Jun 5 '11 at 2:07
3  
What I find most curious is... what's up with that BSD computer? –  Ishpeck Jun 5 '11 at 4:19
show 3 more comments

4 Answers

  • Linux has wide support for lots of different hardware architectures and platforms from tiny embedded boards to massive computing arrays. While other good kernels are available, the coverage and quality of hardware drivers available for Linux far surpass any other platform.
  • The Linux kernel source is open and can easily be modified to run on various custom platforms. For any vendor creating a new piece of hardware, providing Linux drivers is one of the easiest ways to make it accessible. They don't have to work from scratch because they can modify existing drivers for similar pieces of hardware and build on their success.
  • Some of the other OS candidates rack up licensing fees per-CPU. Those become prohibitive at the supercomputer level.
  • Since Linux has been used by everybody in this space before, it has the best support and the widest selection of available software packages and libraries.
share|improve this answer
1  
Are you sure about all these reasons or are you guessing? If you are guessing, at least indicate so. Either that, or mention where you got this info, or even provide links if you have any. –  Tshepang Jun 4 '11 at 22:37
6  
's assertions are fairly self-evident. Here's what IBM said about using Linux on their BlueGene supercomptuers which backs up at least the openness of the kernel reason. –  Amazed Jun 4 '11 at 23:23
1  
Yeah the open source and driver base is probably what really sets Linux apart from everything else. There are plenty of capable open source kernels out there -- but without the large base of hardware support. I see this as the principle reason to go with a Linux kernel. –  Ishpeck Jun 5 '11 at 4:21
1  
Also, over the last years Linux has been carefully optimized for supercomputers by IBM and others. That BlueGene article is from 2002. –  starblue Jun 5 '11 at 8:49
2  
Linux also gained a lot of its supercomputer capability from the integration of SGI's NUMALink technology via the MIPS and Itanium2 architecture trees. One of the first large multiprocessor systems booted with Linux was a 32-cpu Origin 2000. It was later booted on a 128-cpu Origin 2000, and held that record for over two years. Source. –  Kumba Jun 5 '11 at 22:52
add comment

I work in the HPC industry.

If you're asking why most people today use Linux on their cluster, it's what you listed in your question: more than 90% of the biggest clusters run Linux. It's the de-facto standard - almost any cluster library, tool or application is ready-to-run on Linux. It is more work to setup a cluster using any other operating system.

If you're asking how Linux became the de-facto standard, then Caleb has the answers ;)

share|improve this answer
1  
It may be worth saying a few words about the distinction between different types of clusters and old fashioned big iron in this context. –  dmckee Jun 4 '11 at 23:37
add comment

For almost any question of the form: "Why is x the predominant choice in the y market segment?" the answers cluster around two factors.

At some critical juncture during the emergence and growth of that market segment or niche the product in question had some advantages in cost and features which encouraged its adoption by a critical mass. Once that critical mass has been achieved then all of the ancillary products for that segment will support it and all of the key personnel in that industry/niche will be familiar with it as the premier choice.

At some point back in the '90s Donald Becker released some code and information regarding the Beowulf cluster that he and Thomas Sterling had built for a project at NASA. This used commodity hardware, running Linux and incorporating the MPI (message passing interface) and PVM (parallel virtual machine) libraries for distribution of computational tasks across a network of nodes.

At the time the alternatives required much more expensive hardware (mostly Sun workstations), had proprietary software licensing with per/node or per/CPU costs, and typically were closed source or had significant closed source components.

Thus Linux had advantages in all three of these factors. That Becker released some code and documentation (and did so under a cool name) gave Linux a tremendous boost in credibility for that sort of supercomputing application. (That it was used by a project at NASA was also a huge boost to its credibility).

From there colleges and universities picked up the approach for their own labs. Within a couple years after that an entire generation of scientists were familiar with Beowulf clusters and a wide array of tools were readily available to support many applications across them.

share|improve this answer
add comment

One more reason. In the old days for serious work there were no Linux, no Windows, but UNIX and VMS (MSDOS and similar were not contenders, they lacked too many features), and maybe few less known things like lisp machines...

Of those, only UNIX-derived platforms survived. And Linux was a cheap alternative for UNIX-like OSes: more-or-less compatible, open source and free. This made it possible to reuse scientific software that was written before Linux.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.