Tag Info

Hot answers tagged

37

Linux has wide support for lots of different hardware architectures and platforms from tiny embedded boards to massive computing arrays. While other good kernels are available, the coverage and quality of hardware drivers available for Linux far surpass any other platform. The Linux kernel source is open and can easily be modified to run on various custom ...


18

I work in the HPC industry. If you're asking why most people today use Linux on their cluster, it's what you listed in your question: more than 90% of the biggest clusters run Linux. It's the de-facto standard - almost any cluster library, tool or application is ready-to-run on Linux. It is more work to setup a cluster using any other operating system. If ...


14

For almost any question of the form: "Why is x the predominant choice in the y market segment?" the answers cluster around two factors. At some critical juncture during the emergence and growth of that market segment or niche the product in question had some advantages in cost and features which encouraged its adoption by a critical mass. Once that ...


10

I'd use awk, but not store the whole content of L.txt in memory and do unnecessary hash look ups ;-). list=L.txt file=F.txt LIST="$list" awk ' function nextline() { if ((getline n < list) <=0) exit } BEGIN{ list = ENVIRON["LIST"] nextline() } NR == n { print nextline() }' < "$file"


8

I'd use awk: awk 'NR==FNR {a[$1]; next}; FNR in a' L.txt F.txt Update: I've done performance measures; it seems this version scales even better with very large data sets (as is the case with the stated requirements), since the comparison is very fast and overcompensates the effort necessary to build up the hash table.


8

grep -n | sort | sed | cut ( export LC_ALL=C grep -n '' | sort -t: -nmk1,1 ./L - | sed /:/d\;n | cut -sd: -f2- ) <./F That should work pretty quickly (some timed tests are included below) with input of any size. Some notes on how: export LC_ALL=C Because the point of the following operation is to get the entire file of ./F stacked ...


7

With C omitting meaningful error messages: #include <stdio.h> #include <stdlib.h> int main (int argc, char *argv[]) { FILE *L; FILE *F; unsigned int to_print; unsigned int current = 0; char *line = NULL; size_t len = 0; if ((L = fopen(argv[1], "r")) == NULL) { return 1; } else if ((F = fopen(argv[2], ...


6

One more reason. In the old days for serious work there were no Linux, no Windows, but UNIX and VMS (MSDOS and similar were not contenders, they lacked too many features), and maybe few less known things like lisp machines... Of those, only UNIX-derived platforms survived. And Linux was a cheap alternative for UNIX-like OSes: more-or-less compatible, open ...


5

A chroot should not impact performance. I haven't heard of any benchmark on it because it's really just a mapping between two spaces. Anyway, for your problem, maybe it should be better to use something like LXC. Instead of moving somethings working, you can simply create a LXC and do your stuff inside this LXC. LXC is far better than chroot and has really ...


3

I wrote a simple Perl script to do that: Usage: script.pl inputfile_f inputfile_f #!/usr/bin/env perl $number_arguments = $#ARGV + 1; if ($number_arguments != 2) { die "Usage: script.pl inputfile_f inputfile_l\n"; } open($f, '<', $ARGV[0]) or die "$ARGV[0]: Not found\n"; open($l, '<', $ARGV[1]) or die "$ARGV[1]: Not found\n"; ...


3

Just for completeness: we can merge the excellent awk script in the answer by St├ęphane Chazelas, and the perl script in the answer by kos but without keeping the entire list in memory, in the hope that perl might be faster than awk. (I've changed the order of args to match the original question). #!/usr/bin/env perl use strict; die "Usage: $0 l f\n" if ...


2

You should be aware that the "latency" might include the stdio system buffering enough output for an "efficient" filesystem write. See man 3 setbuf for some further information. The buffering is on a per FILE * basis, so stderr is typically unbuffered, while stdio is typically buffered. That's why output to stderr and output to stdout end up appearing ...


1

This perl solution is faster than the other awk or perl solutions by 20% or so, but oviously not as fast as the solution in C. perl -e ' open L, shift or die $!; open F, shift or die $!; exit if ! ($n = <L>); while (1) { $_ = <F>; next if $. != $n; print; exit if ! ($n = <L>); } ' -- L F


1

I would try to diagnose if this is a problem with your VM environment or your code. Here are some things to consider about your VM. Do you have vmware configured to allow access to multiple CPU's? I don't have experience with recent versions, but it used to be something you had to specifically allow. In VirtualBox, I have an option in the virtual machine ...


1

Can some one explain me what the following code means if possible line-by-line. The commands beginning with "#PBS" are directives for the batch job summiting resource manager. Your cluster should have an administrator who can point you at the PBS documentation. Or, here's some I found on the internet: ...


1

Instead of chrooting, why don't you take advantage of environment modules (something you'll see on nearly every linux cluster these days), and install the software to an alternative location?



Only top voted, non community-wiki answers of a minimum length are eligible