Tag Info

New answers tagged


You can use the usual awk de-duplicating technique, on the first field only (fields are separated by spaces): awk '!count[$1]++'


This perl solution is faster than the other awk or perl solutions by 20% or so, but oviously not as fast as the solution in C. perl -e ' open L, shift or die $!; open F, shift or die $!; exit if ! ($n = <L>); while (1) { $_ = <F>; next if $. != $n; print; exit if ! ($n = <L>); } ' -- L F


For completeness, another attempt at the join solution: sed -r 's/^/00000000000000/;s/[0-9]*([0-9]{15})/\1/' /tmp/L | join <( nl -w15 -nrz /tmp/F ) - | cut -d' ' -f2- This works by formatting the the line-number column that join works on as fixed length with leading zeros, so that the numbers are always 15 digits long. This circumvents the problem of ...


Just for completeness: we can merge the excellent awk script in the answer by St├ęphane Chazelas, and the perl script in the answer by kos but without keeping the entire list in memory, in the hope that perl might be faster than awk. (I've changed the order of args to match the original question). #!/usr/bin/env perl use strict; die "Usage: $0 l f\n" if ...


With C omitting meaningful error messages: #include <stdio.h> #include <stdlib.h> int main (int argc, char *argv[]) { FILE *L; FILE *F; unsigned int to_print; unsigned int current = 0; char *line = NULL; size_t len = 0; if ((L = fopen(argv[1], "r")) == NULL) { return 1; } else if ((F = fopen(argv[2], ...


I wrote a simple Perl script to do that: Usage: script.pl inputfile_f inputfile_f #!/usr/bin/env perl $number_arguments = $#ARGV + 1; if ($number_arguments != 2) { die "Usage: script.pl inputfile_f inputfile_l\n"; } open($f, '<', $ARGV[0]) or die "$ARGV[0]: Not found\n"; open($l, '<', $ARGV[1]) or die "$ARGV[1]: Not found\n"; ...


cat <<! >L.txt 1 3 ! cat <<! >F.txt Hello World Hallo Welt Hola mundo ! cmd(){ L=$1 F=$2 cat -n $F | join $L - | sed 's/[^ ]* //' } cmd L.txt F.txt Hello World Hola mundo Since L.txt is sorted you can use join. Just number each line in F.txt, join the two files, then remove the line number. No large intermediate files are needed. ...


grep -n | sort | sed | cut ( export LC_ALL=C grep -n '' | sort -t: -nmk1,1 ./L - | sed /:/d\;n | cut -sd: -f2- ) <./F That should work pretty quickly (some timed tests are included below) with input of any size. Some notes on how: export LC_ALL=C Because the point of the following operation is to get the entire file of ./F stacked ...


I'd use awk, but not store the whole content of L.txt in memory and do unnecessary hash look ups ;-). list=L.txt file=F.txt LIST="$list" awk ' function nextline() { if ((getline n < list) <=0) exit } BEGIN{ list = ENVIRON["LIST"] nextline() } NR == n { print nextline() }' < "$file"


I'd use awk: awk 'NR==FNR {a[$1]; next}; FNR in a' L.txt F.txt Update: I've done performance measures; it seems this version scales even better with very large data sets (as is the case with the stated requirements), since the comparison is very fast and overcompensates the effort necessary to build up the hash table.

Top 50 recent answers are included