Tag Info

Hot answers tagged

11

I'd use awk, but not store the whole content of L.txt in memory and do unnecessary hash look ups ;-). list=L.txt file=F.txt LIST="$list" awk ' function nextline() { if ((getline n < list) <=0) exit } BEGIN{ list = ENVIRON["LIST"] nextline() } NR == n { print nextline() }' < "$file"


9

I'd use awk: awk 'NR==FNR {a[$1]; next}; FNR in a' L.txt F.txt Update: I've done performance measures; it seems this version scales even better with very large data sets (as is the case with the stated requirements), since the comparison is very fast and overcompensates the effort necessary to build up the hash table.


9

grep -n | sort | sed | cut ( export LC_ALL=C grep -n '' | sort -t: -nmk1,1 ./L - | sed /:/d\;n | cut -sd: -f2- ) <./F That should work pretty quickly (some timed tests are included below) with input of any size. Some notes on how: export LC_ALL=C Because the point of the following operation is to get the entire file of ./F stacked ...


8

With C omitting meaningful error messages: #include <stdio.h> #include <stdlib.h> int main (int argc, char *argv[]) { FILE *L; FILE *F; unsigned int to_print; unsigned int current = 0; char *line = NULL; size_t len = 0; if ((L = fopen(argv[1], "r")) == NULL) { return 1; } else if ((F = fopen(argv[2], ...


4

If your files are not too large to fit in memory, you could use perl to slurp the file: perl -0777pe 's/.*?PAT[^\n]*\n?//s' file Just change PAT to whatever pattern you're after. For example, given these two input files and the pattern 5: $ cat file 1 2 3 4 5 11 12 13 14 15 $ cat file1 foo bar $ perl -0777pe 's/.*?5[^\n]*\n?//s' file 11 12 13 14 15 $ ...


3

Just for completeness: we can merge the excellent awk script in the answer by St├ęphane Chazelas, and the perl script in the answer by kos but without keeping the entire list in memory, in the hope that perl might be faster than awk. (I've changed the order of args to match the original question). #!/usr/bin/env perl use strict; die "Usage: $0 l f\n" if ...


3

GNU grep; cat: { grep -m1 'pattern' && cat || ! cat ./infile } <./infile POSIX sed; cat: { sed -ne'/PATTERN/q;H;1h;$!d;x;p'; cat; } <infile GNU sed; cat: { sed -une'/PATTERN/q;H;1h;$!d;x;p'; cat; } <infile (just add -u) sharing is nice All of the above commands work because the file-descriptor from which they read() is ...


3

I wrote a simple Perl script to do that: Usage: script.pl inputfile_f inputfile_f #!/usr/bin/env perl $number_arguments = $#ARGV + 1; if ($number_arguments != 2) { die "Usage: script.pl inputfile_f inputfile_l\n"; } open($f, '<', $ARGV[0]) or die "$ARGV[0]: Not found\n"; open($l, '<', $ARGV[1]) or die "$ARGV[1]: Not found\n"; ...


3

If your system has access to NodeJS you could install the following Node package, strip-ansi-cli. $ npm install -g strip-ansi-cli You can then run your command like so: $ command-that-produces-colored-output | strip-ansi > outfile


3

You can use a regular expression: &!cat|dog|fish


3

There is no need to mix many instruments. Task can be done by sed only sed '/^INFO\|^DEBUG\|^TRACE\|^ERROR/{ /Logger2/{ :1 N /\nINFO\|\nDEBUG\|\nTRACE\|\nERROR/!s/\n// $!t1 D } }' log.entry


3

wget includes features to support this directly: wget -r -A "*.ai,*.cdr" 'address-of-page-with-hyperlinks' -r enables recursive mode so it will download more than simply the given URL, and -A limits the files it will download and keep in the end.


2

Which Unix? It will be quite difficult to make it so that file creation fails when an "invalid" name is passed to it, but there are some platform-specific ways of doing this. First of all, there is inotify and its relatives, which can at least observe the filesystem and notify you immediately when a file is created. I'm not sure you can actually stop the ...


2

You're on the right track with sed: all you need to do is to transform your list of line numbers to be followed by a p and a newline, and use that as a sed script. For example, if you have a space-separated list: lines="2 3 5 7 11 13" <sourcefile.tsv sed -n "$(echo "$lines" | sed 's/$/p/; s/ /p\n/')" >extractedrecords.tsv Awk is another ...


2

Assuming linenumbers.txt has one number per line awk 'NR == FNR{a[$0]; next};FNR in a' linenumbers.txt sourcefile.csv > extractedrecords.tsv Might do the job. Or, with bash join -t':' -o2.1,2.2 <(sort linenumbers.txt) <(awk '{print NR":"$0}' \ sourcefile.csv | sort -k1,1 -t':') | sort -k1,1n -t':' | cut -f2- -d':' All the extra jumping ...


2

Based on one answer at http://stackoverflow.com/questions/9605232/merge-two-lines-into-one this seems to fit the bill #!/usr/local/bin/bash PATTERN1='TRACE *'; PATTERN2='DEBUG *'; PATTERN3='INFO *'; PATTERN4='ERROR *'; LINEOUT="" while read line; do case $line in $PATTERN1) echo $LINEOUT LINEOUT="$line" ...


2

perl filter for multiline log records (record begin mark) Use the following perl script as a working prototype. Usage script_path regular_expression log_files e.g. script_path "line \d" log_file_1 log_file_2 #!/usr/bin/perl $pattern = qr/(?^s)$ARGV[0]/; shift; # process filtering expression # (?^s) - treats matched string as single line my $line = ''; # ...


2

You can use the usual awk de-duplicating technique, on the first field only (fields are separated by spaces): awk '!count[$1]++'


2

Using GNU sed, you can do this: :x;/PATTERN/{s/.*//;:z;N;bz};N;bx For example is we use 7 as the pattern we want to match and input data generated by seq, this will print the numbers 8 to 20 (including 17): seq 20 | sed ':x;/7/{s/.*//;:z;N;bz};N;bx' And this will print 1 to 6: seq 6 | sed ':x;/7/{s/.*//;:z;N;bz};N;bx' As noted in the comments, this ...


1

Pipe the input (cat file in this case) into a basic single-pass of awk first. Then, if pattern is not found (ie. nothing was printed), the process continues on with cat file cat file | { awk -v pat='^a.c$' ' { if( m ) print; else{ if( $0 ~ pat ) m=1 } } END{ exit !m }' || cat file } input: 1 2 abc 4 aXc 6 output ...


1

This perl solution is faster than the other awk or perl solutions by 20% or so, but oviously not as fast as the solution in C. perl -e ' open L, shift or die $!; open F, shift or die $!; exit if ! ($n = <L>); while (1) { $_ = <F>; next if $. != $n; print; exit if ! ($n = <L>); } ' -- L F


1

As long as you're interested in the last column, you can do it with sort and uniq: $ sort -k3n test.txt | uniq -f2 -D 1+1 = 2 2x1 = 2 BLABLABLA = 2 Here, the sort option -k3n causes the file to be sorted starting with the third field, in numeric order; the options to uniq are: -f2 Skip the first two fields before checking for uniqueness -D Print ...


1

Old post, but for reference, it wouldn't work for a few reasons: The priority should be 16 and not 1 The filter handle should be 800::800 and not 800:800 You must supply the parent qdisc that the filter is attached to This should work: tc filter del dev peth1 parent 1: handle 800::800 prio 16 protocol ip u32


1

Grep the first longest line grep -Em1 "^.{$(wc -L <file.txt)}\$" file.txt The command is unusually hard to read without practise because it mixes shell- and regexp syntax. For explanation, I will use simplified pseudocode first. The lines starting with ## do not run in the shell. This simplified code uses the file name F, and leaves out quoting and ...


1

You can do it : rdr pass quick on $ext_inf inet proto tcp from any to any port 1394 -> $target port 1394


1

Just Do It. dummyvalue=`sleep 10` If you want it as part of a recipe, you can easily run any command synchronously: :0w * ? sleep 10 { } (I added the w flag for good measure.) Much larger values are possible, though if you exceed the default value of TIMEOUT, Procmail will abort the sleep. Though you can bump the value of TIMEOUT to a larger value ...



Only top voted, non community-wiki answers of a minimum length are eligible