Hot answers tagged

18

They're not expressions, they're filenames for files produced as follows: printenv | sort > printenv.sorted set | sort > set.sorted That's not clear from the documentation so your confusion is understandable! Note that you may need to help diff and grep by forcing them to treat their inputs as text (with -a); environment variables can contain ...


9

The right tool for this job is column. You can specify column separator with -o (on OS X it's -s) , e.g.: column -t -o ' ' file gives TLRUIDA CBdms Status DP 6/1/1 DC 6/1/5 0 Y TLRUIDA CBdms Status DP 6/2/1 DC 6/2/5 0 Y TLRUIDA CBdms Status DP 6/3/1 DC 6/3/5 0 Y TLRUIDA CBdms Status DP 6/4/1 ...


8

To try and avoid storing the whole file in memory, you could do: awk -F , ' !count[$1]++ {save[$1] = $0; next} count[$1] == 2 { print save[$1] delete save[$1] } {print}'


7

Typical job for awk: awk 'NR == 1 {first = $1}; $1 - first <= 10' < file Or to do it only for the last line: awk 'NR == 1 {first = last = $0; next} {print last; last = $0} END {if (NR && last - first <= 10) print last}' < file


6

Use a JSON aware tool. Perl has the JSON library: #!/usr/bin/perl use warnings; use strict; use JSON; my $json = '[{"product":"Apple","id":"2134"},{"product":"Mango","id":"4567"}]'; print 'Enter product: '; chomp( my $product = <> ); print 'Your product id is: ', (grep $_->{product} eq 'Apple', @{ from_json($json) })[0]{id}, "\n";


5

Using gsub: awk '{gsub(/\"|\;/,"")}1' file chr1 134901 139379 - ENSG00000237683.5 chr1 860260 879955 + ENSG00000187634.6 chr1 861264 866445 - ENSG00000268179.1 chr1 879584 894689 - ENSG00000188976.6 chr1 895967 901095 + ENSG00000187961.9 If you want to operate only on the fifth field and preserve any quotes or semicolons ...


5

You can use perl: perl -lne ' $n = $_ if $. == 1; print unless eof; print if $_ - $n <= 10 and eof; ' <file For the first line $. == 1, we set its value to variable $n For next lines, print if it's not the last line Print the last line if the different between its first column and $n less than or equal 10. Here we used $_ - $n, forcing the ...


5

SP="ABC" if ( $1 ~ "^" SP "[0-9]{3}") You can concatenate strings but not /xxx/s which are in effect more like regular expression matching operators, and with parsing rules that can get confusing (and vary between implementations) $1 ~ /ABC/ /BCD/ could be seen as the concatenation of $1 matched against the concatenation of /ABC/ (1 or 0 depending on ...


5

EDIT: Extensible to any number of output rows, in a simple one-liner for loop: for ((i=1;i<=2;i++)); do cut -d: -f "$i" input | paste -sd: ; done | column -t -s: Original answer: You can do this as a one-liner using bash process substitution: paste -sd: <(cut -d: -f1 input) <(cut -d: -f2 input) | column -t -s: The -s option to paste makes ...


5

If you have the rs (reshape) utility available, you can do the following: rs -Tzc: < input.txt This gives the output format exactly as specified in the question, even down to the dynamic column widths. -T Transposes the input data -z sizes the columns appropriately from the max in each column -c: uses colon as the input field separator This works ...


5

If the input is processed line by line, then processing needs to go like this: if the current line is foo.bar, store it, forgetting any previous foo.bar line that wasn't enabled for output; if the current line is relevant=yes, this enables the latest foo.bar for output. This kind of reasoning is a job for awk. (It can also be done in sed if you like ...


5

I'm going to assume that what you've posted is a sample, because it isn't valid XML. If this assumption isn't valid, my answer doesn't hold... but if that is the case, you really need to hit the person who gave you the XML with a rolled up copy of the XML spec, and demand they 'fix it'. But really - awk and regular expressions are not the right tool for the ...


4

awk ' { for(i=1;i<=NF;i++){ sub(/[0-9]{4}$/,":&",$i) sub(/:[0-9]{2}/,"&:",$i) } } 1 ' <<<\ '034023 052030 034023 022130 044023 012030 034223 022030 034123 152030 024023 152030' produces: 03:40:23 05:20:30 03:40:23 02:21:30 04:40:23 01:20:30 03:42:23 02:20:30 03:41:23 ...


4

No. It is not possible to assign non-scalar variables like this on the command line. But it is not too hard to make it. If you can accept a fixed array name: awk -F= ' FNR==NR { a[$1]=$2; next} { print a[$1] } ' <(echo $'index=value\nindex two=value two') <(echo index two) If you have a file containing the awk syntax for array definitions, you ...


4

You can use a variable to track if the previous line is present or not: $ awk ' FNR % 3 == 1 {f = $0; next} # The first line keep in f, skip to next line FNR % 3 && f {print f;print} # Previous line present, print it and current line ' <file this line 1 no un1x this lines 22 0 but not 1 THIS is not Or with sed: sed -ne 'N;/\n/p;N;d' ...


4

This will print lines (with filename and line number) with repeated words: for f in *.txt; do perl -ne 'print "$ARGV: $.: $_" if /\b(\w+)\W+\1/' "$f" done For multi-line matching there's this, but you lose the line numbers because it's slurping in the file by paragraphs (that's the effect of the -00 option). The \W+ between the two words means any ...


4

Perl to the rescue! #!/usr/bin/perl use warnings; use strict; my $group_size = 3; my @first = split ' ', <>; my @groups; my $start_index = 0; while ($start_index < @first) { my $step = 1; while ( $step < $group_size && $start_index + $step < @first && $first[$start_index] == $first[ ...


4

Just pipe the output to column -t: sed 's/$/ IN A 192.168.155.128/' db.malware-host-only | column -t > tmp mv tmp db.malware-host-only You can use the -o parameter to extend the spaces between columns: column -t -o ' ' You can't use sed -i with that, so output to a temp file, then rename it to the original name.


4

Try awk awk ' NR==FNR{ A[NR]=$1 limit=NR next } /^avocado/{ i=i%limit+1 $1=A[i] } { print } ' newservers.lst servers.txt Sed is possible too: sed '/^\s*\S\+\s*$/ { #match 1st file only x #exchange line with holdspace H ...


4

Try grep: grep -iv dog inputfile -i to ignore case and -v to invert the matches. If you want to use sed you can do: sed '/[dD][oO][gG]/d' inputfile In sed, there is also the I flag, which should make the match case insensitive, but as far as I remember this does not work in all flavors of sed. For me, this works: sed '/dog/Id' inputfile but it ...


3

If your data is formatted exactly as shown (i.e. no other " or ; in other columns that need to be preserved), then you can simply use tr to remove these characters: tr -d '";' < input.txt > output.txt


3

Using sed to remove all instances of '";': sed -i 's/[";]//g' file To only remove from 5th column sed is probably not the best option.


3

I think you want something like: awk 'on{print;on=0} $1=="asad"{on=1}' test The way you have it, the 1 at the end of the script is causing every line to print (except when next is called, which skips it)


3

Just add x="F0" to the beginning so the target file is always defined, even if the first line doesn't contain the pattern: awk 'BEGIN { x="F0" ; } /START/{x="F"++i;}{print > x}' The above breaks down to this pseudo code: ### -> BEGIN { x="F0" ; } i=0 # implicit x="F0" # explicit loop through file ### -> /START/{x="F"++i;} if ( line contains ...


3

Just use sprintf: BEGIN { myarray["01"] myarray["02"] myarray["04"] myarray["05"] # ... etc, up to "12" for (i = 1; i <= 12; i++) { k = sprintf("%02d", i) if (! (k in myarray)) { print k " is missing from myarray" } } } gave you: 03 is missing from myarray 06 is missing from myarray 07 is missing from myarray 08 is ...


3

awk 'NR == 2 || NR == 3 {l = l RS $0; next} NR == 5 {$0 = $0 l} {print}'


3

One way: awk 'NR==2 || NR==3{a[i++]=$0;next}1;NR==5{for(j=0;j<2;j++){print a[j];}}' file


3

Try: $ awk ' FNR == 2 { l2 = $0; next } # Save 2nd line FNR == 3 { l3 = $0; next } # Save 3rd line FNR == 5 { # Print 5th line, follow 2nd, 3rd print print l2 print l3 next } 1 # Print other lines ' <file Note that if you will lose the 2nd and 3rd line if the file had less than five ...


3

Perl solution: perl -F, -ane ' $h{ $F[0] } .= $_ }{ $h{$_} =~ tr/\n// >= 2 and print $h{$_} for keys %h ' < input-file -n reads the input line by line -a splits each line on -F, i.e. comma, into the @F array. lines are stored in the %h hash keyed by the first field ($F[0]). They are concatenated ...


3

You could use grep with -A. Something like: $ grep -A 13 '^\[2\]' inputfile.txt The -A specifies the number of lines you want to include after the match. But I think it would be better to use sed in this case: $ sed -n '/^\[2\]/,/^$/p' inputfile.txt This will print everything between [2] and an empty line. The same using awk: $ awk -v RS='' -v ...



Only top voted, non community-wiki answers of a minimum length are eligible