Hot answers tagged

13

Another POSIX one: awk -F , 'NF == 11' <file If the line has 10 commas, then there will be 11 fields in this line. So we simply make awk use , as the field delimiter. If the number of fields is 11, the condition NF == 11 is true, awk then performs the default action print $0.


11

I'd use awk, but not store the whole content of L.txt in memory and do unnecessary hash look ups ;-). list=L.txt file=F.txt LIST="$list" awk ' function nextline() { if ((getline n < list) <=0) exit } BEGIN{ list = ENVIRON["LIST"] nextline() } NR == n { print nextline() }' < "$file"


9

I'd use awk: awk 'NR==FNR {a[$1]; next}; FNR in a' L.txt F.txt Update: I've done performance measures; it seems this version scales even better with very large data sets (as is the case with the stated requirements), since the comparison is very fast and overcompensates the effort necessary to build up the hash table.


8

With C omitting meaningful error messages: #include <stdio.h> #include <stdlib.h> int main (int argc, char *argv[]) { FILE *L; FILE *F; unsigned int to_print; unsigned int current = 0; char *line = NULL; size_t len = 0; if ((L = fopen(argv[1], "r")) == NULL) { return 1; } else if ((F = fopen(argv[2], ...


7

Using egrep (or grep -E in POSIX): egrep "^([^,]*,){10}[^,]*$" file.csv This filters out anything not containing 10 commas: it matches full lines (^ at the start and $ at the end), containing exactly ten repetitions ({10}) of the sequence "any number of characters except ',', followed by a single ','" (([^,]*,)), followed again by any number of characters ...


5

If your files are not too large to fit in memory, you could use perl to slurp the file: perl -0777pe 's/.*?PAT[^\n]*\n?//s' file Just change PAT to whatever pattern you're after. For example, given these two input files and the pattern 5: $ cat file 1 2 3 4 5 11 12 13 14 15 $ cat file1 foo bar $ perl -0777pe 's/.*?5[^\n]*\n?//s' file 11 12 13 14 15 $ ...


5

OK, found it myself. The answer is /proc/sys/net/ipv4/conf/*/rp_filter. The option is documented at the Linux Foundation. While this is an old setting, it seems that Ubuntu changed the default value sometime between 07.04 and 14.04. Changing the value from 1 back to 0 fixed my problem.


4

Throwing some short python: #!/usr/bin/env python2 with open('file.csv') as f: print '\n'.join(line for line in f if line.count(',') == 10) This will read each line and check if the number of commas in the line is equal to 10 line.count(',') == 10, if so print it will the line.


4

The simplest grep code that will work: grep -xE '([^,]*,){10}[^,]*' Explanation: -x ensures that the pattern must match the entire line, rather than just part of it. This is important so you don't match lines with more than 10 commas. -E means "extended regex", which makes for less backslash-escaping in your regex. Parentheses are used for grouping, ...


3

Just for completeness: we can merge the excellent awk script in the answer by Stéphane Chazelas, and the perl script in the answer by kos but without keeping the entire list in memory, in the hope that perl might be faster than awk. (I've changed the order of args to match the original question). #!/usr/bin/env perl use strict; die "Usage: $0 l f\n" if ...


3

I wrote a simple Perl script to do that: Usage: script.pl inputfile_f inputfile_f #!/usr/bin/env perl $number_arguments = $#ARGV + 1; if ($number_arguments != 2) { die "Usage: script.pl inputfile_f inputfile_l\n"; } open($f, '<', $ARGV[0]) or die "$ARGV[0]: Not found\n"; open($l, '<', $ARGV[1]) or die "$ARGV[1]: Not found\n"; ...


3

wget includes features to support this directly: wget -r -A "*.ai,*.cdr" 'address-of-page-with-hyperlinks' -r enables recursive mode so it will download more than simply the given URL, and -A limits the files it will download and keep in the end.


3

There is no need to mix many instruments. Task can be done by sed only sed '/^INFO\|^DEBUG\|^TRACE\|^ERROR/{ /Logger2/{ :1 N /\nINFO\|\nDEBUG\|\nTRACE\|\nERROR/!s/\n// $!t1 D } }' log.entry


2

Based on one answer at http://stackoverflow.com/questions/9605232/merge-two-lines-into-one this seems to fit the bill #!/usr/local/bin/bash PATTERN1='TRACE *'; PATTERN2='DEBUG *'; PATTERN3='INFO *'; PATTERN4='ERROR *'; LINEOUT="" while read line; do case $line in $PATTERN1) echo $LINEOUT LINEOUT="$line" ...


2

perl filter for multiline log records (record begin mark) Use the following perl script as a working prototype. Usage script_path regular_expression log_files e.g. script_path "line \d" log_file_1 log_file_2 #!/usr/bin/perl $pattern = qr/(?^s)$ARGV[0]/; shift; # process filtering expression # (?^s) - treats matched string as single line my $line = ''; # ...


2

Which Unix? It will be quite difficult to make it so that file creation fails when an "invalid" name is passed to it, but there are some platform-specific ways of doing this. First of all, there is inotify and its relatives, which can at least observe the filesystem and notify you immediately when a file is created. I'm not sure you can actually stop the ...


2

You can use the usual awk de-duplicating technique, on the first field only (fields are separated by spaces): awk '!count[$1]++'


2

Using GNU sed, you can do this: :x;/PATTERN/{s/.*//;:z;N;bz};N;bx For example is we use 7 as the pattern we want to match and input data generated by seq, this will print the numbers 8 to 20 (including 17): seq 20 | sed ':x;/7/{s/.*//;:z;N;bz};N;bx' And this will print 1 to 6: seq 6 | sed ':x;/7/{s/.*//;:z;N;bz};N;bx' As noted in the comments, this ...


2

And here's a Perl way: perl -F, -ane 'print if $#F==10' The -n causes perl to read its input file line by line and execute the script given by -e on each line. The -a turns on automatic splitting: each input line will be split on the value given by -F (here, a comma) and saved as the array @F. The $#F (or, more generally $#array), is the highest index ...


2

The pstree program seems to be quite nice for this e.g. $ pidof bash | xargs -n 1 pstree -sp init(1)───lightdm(1284)───lightdm(1577)───init(2017)───gnome-terminal(2595)───bash(18001)───man(10946)───pager(10955) init(1)───lightdm(1284)───lightdm(1577)───init(2017)───gnome-terminal(2595)───bash(12895) ...


2

The xmlstarlet tool will do this: xmlstarlet sel -t -m /A -o ID, -v id -n -o C, -v //C -n -o D, -v //D -n test.xml For each A under the root element (-m /A), it prints the string "ID," (-o ID,), the contents of id (-v id), a newline (-n), and likewise for children C (-v //C)and D (-v //D) with their respective headers. The double slashes are the XPath ...


1

If fields can contain commas or newlines your code needs to understand csv. Example (with three columns): $ cat filter.csv a,b,c d,"e,f",g 1,2,3,4 one,two,"three ...continued" $ cat filter.csv | python3 -c 'import sys, csv > csv.writer(sys.stdout).writerows( > row for row in csv.reader(sys.stdin) if len(row) == 3) > ' a,b,c d,"e,f",g ...


1

Pipe the input (cat file in this case) into a basic single-pass of awk first. Then, if pattern is not found (ie. nothing was printed), the process continues on with cat file cat file | { awk -v pat='^a.c$' ' { if( m ) print; else{ if( $0 ~ pat ) m=1 } } END{ exit !m }' || cat file } input: 1 2 abc 4 aXc 6 output ...


1

This perl solution is faster than the other awk or perl solutions by 20% or so, but oviously not as fast as the solution in C. perl -e ' open L, shift or die $!; open F, shift or die $!; exit if ! ($n = <L>); while (1) { $_ = <F>; next if $. != $n; print; exit if ! ($n = <L>); } ' -- L F


1

As long as you're interested in the last column, you can do it with sort and uniq: $ sort -k3n test.txt | uniq -f2 -D 1+1 = 2 2x1 = 2 BLABLABLA = 2 Here, the sort option -k3n causes the file to be sorted starting with the third field, in numeric order; the options to uniq are: -f2 Skip the first two fields before checking for uniqueness -D Print ...


1

You can do it : rdr pass quick on $ext_inf inet proto tcp from any to any port 1394 -> $target port 1394



Only top voted, non community-wiki answers of a minimum length are eligible