Hot answers tagged

13

Another POSIX one: awk -F , 'NF == 11' <file If the line has 10 commas, then there will be 11 fields in this line. So we simply make awk use , as the field delimiter. If the number of fields is 11, the condition NF == 11 is true, awk then performs the default action print $0.


11

I'd use awk, but not store the whole content of L.txt in memory and do unnecessary hash look ups ;-). list=L.txt file=F.txt LIST="$list" awk ' function nextline() { if ((getline n < list) <=0) exit } BEGIN{ list = ENVIRON["LIST"] nextline() } NR == n { print nextline() }' < "$file"


9

grep -n | sort | sed | cut ( export LC_ALL=C grep -n '' | sort -t: -nmk1,1 ./L - | sed /:/d\;n | cut -sd: -f2- ) <./F That should work pretty quickly (some timed tests are included below) with input of any size. Some notes on how: export LC_ALL=C Because the point of the following operation is to get the entire file of ./F stacked ...


9

I'd use awk: awk 'NR==FNR {a[$1]; next}; FNR in a' L.txt F.txt Update: I've done performance measures; it seems this version scales even better with very large data sets (as is the case with the stated requirements), since the comparison is very fast and overcompensates the effort necessary to build up the hash table.


8

With C omitting meaningful error messages: #include <stdio.h> #include <stdlib.h> int main (int argc, char *argv[]) { FILE *L; FILE *F; unsigned int to_print; unsigned int current = 0; char *line = NULL; size_t len = 0; if ((L = fopen(argv[1], "r")) == NULL) { return 1; } else if ((F = fopen(argv[2], ...


8

Using egrep (or grep -E in POSIX): egrep "^([^,]*,){10}[^,]*$" file.csv This filters out anything not containing 10 commas: it matches full lines (^ at the start and $ at the end), containing exactly ten repetitions ({10}) of the sequence "any number of characters except ',', followed by a single ','" (([^,]*,)), followed again by any number of characters ...


5

OK, found it myself. The answer is /proc/sys/net/ipv4/conf/*/rp_filter. The option is documented at the Linux Foundation. While this is an old setting, it seems that Ubuntu changed the default value sometime between 07.04 and 14.04. Changing the value from 1 back to 0 fixed my problem.


5

If your files are not too large to fit in memory, you could use perl to slurp the file: perl -0777pe 's/.*?PAT[^\n]*\n?//s' file Just change PAT to whatever pattern you're after. For example, given these two input files and the pattern 5: $ cat file 1 2 3 4 5 11 12 13 14 15 $ cat file1 foo bar $ perl -0777pe 's/.*?5[^\n]*\n?//s' file 11 12 13 14 15 $ ...


5

sed -ne's/,//11;t' -e's/,/&/10p' <in >out That first branches out any line with 11 or more commas, and then prints of what remains only those that match 10 commas. Apparently I answered this before... Here's a me-plagiarism from a question looking for exactly 4 occurrences of some pattern: You can target [num]th occurrence of a pattern with ...


4

The simplest grep code that will work: grep -xE '([^,]*,){10}[^,]*' Explanation: -x ensures that the pattern must match the entire line, rather than just part of it. This is important so you don't match lines with more than 10 commas. -E means "extended regex", which makes for less backslash-escaping in your regex. Parentheses are used for grouping, ...


4

GNU grep; cat: { grep -m1 'pattern' && cat || ! cat ./infile } <./infile POSIX sed; cat: { sed -ne'/PATTERN/q;H;1h;$!d;x;p'; cat; } <infile GNU sed; cat: { sed -une'/PATTERN/q;H;1h;$!d;x;p'; cat; } <infile (just add -u) sharing is nice All of the above commands work because the file-descriptor from which they read() is ...


4

Throwing some short python: #!/usr/bin/env python2 with open('file.csv') as f: print '\n'.join(line for line in f if line.count(',') == 10) This will read each line and check if the number of commas in the line is equal to 10 line.count(',') == 10, if so print it will the line.


3

I wrote a simple Perl script to do that: Usage: script.pl inputfile_f inputfile_f #!/usr/bin/env perl $number_arguments = $#ARGV + 1; if ($number_arguments != 2) { die "Usage: script.pl inputfile_f inputfile_l\n"; } open($f, '<', $ARGV[0]) or die "$ARGV[0]: Not found\n"; open($l, '<', $ARGV[1]) or die "$ARGV[1]: Not found\n"; ...


3

Just for completeness: we can merge the excellent awk script in the answer by Stéphane Chazelas, and the perl script in the answer by kos but without keeping the entire list in memory, in the hope that perl might be faster than awk. (I've changed the order of args to match the original question). #!/usr/bin/env perl use strict; die "Usage: $0 l f\n" if ...


2

You can use the usual awk de-duplicating technique, on the first field only (fields are separated by spaces): awk '!count[$1]++'


2

The pstree program seems to be quite nice for this e.g. $ pidof bash | xargs -n 1 pstree -sp init(1)───lightdm(1284)───lightdm(1577)───init(2017)───gnome-terminal(2595)───bash(18001)───man(10946)───pager(10955) init(1)───lightdm(1284)───lightdm(1577)───init(2017)───gnome-terminal(2595)───bash(12895) ...


2

The xmlstarlet tool will do this: xmlstarlet sel -t -m /A -o ID, -v id -n -o C, -v //C -n -o D, -v //D -n test.xml For each A under the root element (-m /A), it prints the string "ID," (-o ID,), the contents of id (-v id), a newline (-n), and likewise for children C (-v //C)and D (-v //D) with their respective headers. The double slashes are the XPath ...


2

Using GNU sed, you can do this: :x;/PATTERN/{s/.*//;:z;N;bz};N;bx For example is we use 7 as the pattern we want to match and input data generated by seq, this will print the numbers 8 to 20 (including 17): seq 20 | sed ':x;/7/{s/.*//;:z;N;bz};N;bx' And this will print 1 to 6: seq 6 | sed ':x;/7/{s/.*//;:z;N;bz};N;bx' As noted in the comments, this ...


2

And here's a Perl way: perl -F, -ane 'print if $#F==10' The -n causes perl to read its input file line by line and execute the script given by -e on each line. The -a turns on automatic splitting: each input line will be split on the value given by -F (here, a comma) and saved as the array @F. The $#F (or, more generally $#array), is the highest index ...


1

To filter out either message in one grep, pull in the -E extended regular expression flag (for the "zero or once" meaning of ? below), and the -v inverse match: grep -Ev '^Remote:( Checking segments [[:digit:]]{1,3}\.[[:digit:]]%)?$' which says that the lines should (inverse) match: beginning of line ^ the string Remote: the grouped ( ... ) set of ...


1

If fields can contain commas or newlines your code needs to understand csv. Example (with three columns): $ cat filter.csv a,b,c d,"e,f",g 1,2,3,4 one,two,"three ...continued" $ cat filter.csv | python3 -c 'import sys, csv > csv.writer(sys.stdout).writerows( > row for row in csv.reader(sys.stdin) if len(row) == 3) > ' a,b,c d,"e,f",g ...


1

Pipe the input (cat file in this case) into a basic single-pass of awk first. Then, if pattern is not found (ie. nothing was printed), the process continues on with cat file cat file | { awk -v pat='^a.c$' ' { if( m ) print; else{ if( $0 ~ pat ) m=1 } } END{ exit !m }' || cat file } input: 1 2 abc 4 aXc 6 output ...


1

I temporarily replaced my .forward file with yours and confirmed that it doesn't work. There are two problems. contains performs a substring match, and does not understand regular expressions. For regexes you want matches rather than contains. The \d PCRE-style character class appears to be broken as does the {N} syntax! I tried all kinds of combinations. ...


1

This perl solution is faster than the other awk or perl solutions by 20% or so, but oviously not as fast as the solution in C. perl -e ' open L, shift or die $!; open F, shift or die $!; exit if ! ($n = <L>); while (1) { $_ = <F>; next if $. != $n; print; exit if ! ($n = <L>); } ' -- L F



Only top voted, non community-wiki answers of a minimum length are eligible