New answers tagged

0

You almost have it. Step1 On GNU and Linux and perhaps other systems, the date command can be used to print out an arbitrary format for a time specification given a user-friendly time expression. For instance, I can get a string representing the time from 15 minutes in the past using this: date --date='15 minutes ago' You've pretty much done this with ...


0

I'd use: awk -v limit="$(date -d '15 minutes ago' +'[%FT%T')" ' $0 >= limit' < log-file That ignores the potential problems you may have two hours per year when the GMT offsets goes from -04:00 to -05:00 and back if daylight saving applies in your timezone. date -d is GNU specific, but you're using it already.


0

Regular expressions aren't a good choice for matching timestamps, as they don't really 'understand' numeric values. So instead, I'd suggest parsing the timestamp: #!/usr/bin/env perl use strict; use warnings; use Time::Piece; my $now = time(); my $last = $now - 15 * 60; while ( <> ) { my ( $timecode ) = m/\[([^\.]+)/; print $timecode; ...


3

Let's decompose both commands num_rec=`cat test.csv |wc -l|tr -d " " | nawk '{printf("%0.6d\n", $1) }'` is a compounded command : A | B | C | D A: cat test.csv just cats the file test.csv (and is also a UUOC, "Useless Use Of Cat") B: wc -l will count the number of lines from its input (stdin), which is here test.csv's content. So it will count the ...


0

They're commands written by a novice who either hasn't learnt anything since the early 90s or is following a HOWTO from that era. Would be better written as: num_rec=$(wc -l <test.csv) num_rec=$(printf "%0.6d" $(( num_rec + 2 )) ) This uses command substitution to get the line count from test.cv using wc -l (by redirection, so that wc doesn't print ...


0

This looks for integers in any column after the first and adds .00: $ awk '{for (i=2;i<=NF;i++) if (!($i ~ /[.]/))$i=$i ".00"} 1' input 2017 47.0000 0.783333 0.2500 1663 1920.0000 32.00 8.0000 How it works for (i=2;i<=NF;i++) This starts a loop over the fields, starting with the second field, i=2, and continuing on to the last field, i=NF. if ( ...


1

awk '$3 !~ /\./ { sub($3, $3 ".00") } { print }' columns This suggestion is based on the sample input, obviously. It works as follows: if column three (the code in the OP uses column nine, but the sample input isn't that long) does not contain a literal dot, replace column three with column three plus the literal string .00 (in other words: append the ...


3

Limit the * to non-spaces: sed -i 's/-agentpath[^ ]*//' files


2

awk can certainly do float comparisons if called from your shell script. num1=0.502E-01 num2=0.01 awk -v a="$num1" -v b="$num2" 'BEGIN{print(a>b)}' 1 awk -v a="$num1" -v b="$num2" 'BEGIN{print(b>a)}' 0


2

Try this: awk '/^[*][*][*] /{ if ($0 in seen) fname=$0; else seen[$0];} fname{print>fname}' file How it works awk implicitly reads a file line by line. For each line read, we do the following: /^[*][*][*] /{ if ($0 in seen) fname=$0; else seen[$0];} For any line that begins with three stars and a space, we check to see if we have seen that line ...


3

You could split the 2nd field on : and if you get more than 2 pieces (that is, the number of elements in array z) keep only the 1st one: awk '{n=split($2, z, ":");if (n > 2) $2=z[1]};1' infile If you wanted to use sub you could do something like: awk '{sub(/:.*:.*/,"",$2)};1' infile that is, attempt to replace two colons (or more).


1

Here's a fairly simple and straight-forward shell script that uses jsonpipe to do what you want. It doesn't use any fancy sh/bash features, and does only the bare minimum sanity checking of filenames. NOTE: jq is far more capable than jsonpipe, but jsonpipe is simpler and easier to use when you don't particularly care (or want to know) about the structure ...


0

What about echo test/{1999,2000,2001}/test or even echo test/{1999..2001}/test


1

To pass strings to an awk script, pass them through environment variables. export VAR VAR1 awk ' 1 $0 == ENVIRON["VAR"] {print ENVIRON["VAR1"]} ' ~/Scripts/tmp/file.txt I reorganized your script to make the logic simpler. I also replaced the regexp matching by a string comparison: with regexp matching, $VAR would be treated as a regular ...


0

If you have GNU awk (gawk), which is usually the case on non-embedded Linux and Cygwin, then you can use the strftime function. if (match($0, /^(.*timestamp=)([0-9]+)(.*)$/, parts)) { time = strftime("%F %T", parts[2]); $0 = parts[1] time parts[3]; } On an embedded system with BusyBox, you may have a more restricted version of awk but a date ...


0

Discarding colours - you may use: # | piped at the end awk -F'SET timestamp=' 'NF > 1{ system("date -d @" $2) }' Stripping the colours out in-line with the reset makes it more challenging; I suspect that you can control / adjust that by removing the end included escape sequences that relate to the timestamp portion.


1

Here is one bash solution as well : declare -a a=(1999 2000 2001) url='/test/test' for i in "${a[@]}" ; do echo "$url" | sed "s:/:/$i/:2"; done


1

You specify only one "URL" the following Python program support a list of them: urls = [ '/test/test', ] years = [1999, 2000, 2001] for url in urls: for year in years: spliturl = url.split('/') spliturl.insert(2, str(year)) print('/'.join(spliturl)) The trick is to insert the year at the second position, the first ...


1

I like joseph's answer but needed it to strip // comments also so I modified it slightly & tested on redhat # no comments alias alias nocom="sed -E '/^[[:blank:]]*(\/\/|#)/d;s/#.*//' | strings" # example cat SomeFile | nocom | less I bet there's a better way to remove blank lines than using strings but it was the quick & dirty solution I used. ...


1

Instead of printing matching lines with decorations, decorate matching lines, and print everything: awk '/User@Host:/ { $0 = "\033[32m" $0 "\033[39m" }; /Query_time:/ { $0 = "\033[29m" $0 "\033[39m" }; 1' This awk program consists of three patterns and associated actions: lines matching User@Host: are processed with $0 = "\033[32m" $0 "\033[39m", which ...


1

A perl solution: $ perl -lpe '$_ .= ".mtt.corp" if !/\.mtt\.corp$/' file linuxA.mtt.corp linux3V.mtt.corp linux4B.mtt.corp linux2A.mtt.corp linux5v.mtt.corp The -p makes perl print each line of the input file after applying the script given by -e and the -l removes trailing newlines from each input line and adds one to each print call. The $_ variable is ...


4

It's quite simple with sed: sed -e '/\.mtt\.corp$/!s/$/.mtt.corp/' <file That does substitute end of line with .mtt.corp on each line which does not end with that string.


-1

If you want to use awk, you have to pass the bash variable to awk with the -v option : awk -v v=$VAR -v v1=$VAR1 '/v/ { print; print v1; next }1' ~/Scripts/tmp/file.txt But check Rob's answer for more correct and detailed solution.


0

One possible AWK solution: awk '/'$VAR'/ { print; print "'$VAR1'"; next }1' break the ' quotations around all your external variable. Another possible solution like the other post suggest with passing variables is to pass VAR1 in as an awk variable but I think you still need to do the single quote block escape for the pattern using VAR: awk -v ...


0

Here's another way with sed: sed -n '/<Car>/{x;/.\{2\}/{x;$!{n;p};q};s/.*/&./;x}' infile This is using the hold space to count. Each time it encounters a line matching <Car> it exchanges buffers and checks if there are exactly N-1 occurrences of a character in the hold buffer. If the check is successful it exchanges again, and if not on ...


1

If you don't mind having the output sorted, you can keep track of the current person and the current fruit, and sum until either changes: #!/usr/bin/awk -f NF { if (who != $1) { if (count > 0) { printf "%16s %2d\n", fruit, count } who = $1 printf "%s:\n", who fruit = "" count = 0 } ...


1

How about awk ' BEGIN {FS="\n"; RS="\n\n+"} {for (i=1;i<=NF;i++) a[i] = a[i] == ""? $i : a[i]"\t"$i; next} END {for (i in a) print a[i]} ' file.ex Testing: awk ' > BEGIN {FS="\n"; RS="\n\n+"} > {for (i=1;i<=NF;i++) a[i] = a[i] == ""? $i : a[i]"\t"$i; next} > END {for (i in a) print a[i]} > ' file.ex efifc1a nhdw4s jfhg ...


1

If you accept having temporary files, you could do it as a two step process with awk and paste: n=$(awk '{ print $0 > NR; close(NR) } END { print NR }' RS= file.ex) paste $(seq $n) Or as a one-liner: paste $(seq $(awk '{ print $0 > NR; close(NR) } END { print NR }' RS= file.ex)) Output in both cases: efifc1a nhdw4s jfhg hygg4a wesf3a gsfar ...


3

Using GNU awk, which has arrays of arrays (may require gawk version 4) gawk ' NF { n[$1][$2] += $3 } END { for (name in n) { print name ":" for (fruit in n[name]) printf "%16s %2d\n", fruit, n[name][fruit] } } '


0

BEGIN { RS="=====*\n" } /Queue Manager/ { manager = $0; next; } /[a-z]/ { print RT manager RT $0; } The first rule sets the record delimiter to four or more equal signs. The second rule keeps track of the "header", i.e. the record containing the string "Queue Manager". The third rule prints the header and the current record if the record contains ...


2

stime=12 etime=13 date=2014-11-24 awk -v a="$stime" -v b="$etime" -v d="$date" -F "[: ]+" ' { if ( $1 == d && $2+0 >= a && $2+0 < b ) print $1,$2,$3 }' logfile.txt This produces the output: 2014-11-24 12 58 2014-11-24 12 58 Notes: FS="[: ];" causes the field separator to be set to a colon or space followed by a ...


2

You could try like this sed '/Queue/{N;$d;N;$d;N;/==$/d}' infile This just pulls in the next three lines when line matches Queue. If the pattern space ends with a separator1 it deletes it (or if either2 the 1st line or the 2nd line pulled in is the last one in the input). If other lines may end with consecutive = signs you should replace the ==$ in the ...


3

Alternative way with sed, e.g. for N=2 sed '1N;$!N;/.*\n.*\n.*pattern.*/P;D' infile On 1st line this will read the next N-1 lines in the pattern space then start a N;P;D cycle - read another line in and if the last line in the pattern space matches, print the first line in the pattern space then delete it, starting a new cycle. The downside is that ...


8

A buffer of lines needs to be used. Give a try to this: awk -v N=4 -v pattern="example.*pattern" '{i=(1+(i%N));if (buffer[i]&& $0 ~ pattern) print buffer[i]; buffer[i]=$0;}' file Set N value to the Nth line before the pattern to print. Set patternvalue to the regex to search. buffer is an array of N elements. It is used to store the lines. Each ...


2

tac file | awk 'c&&!--c;/pattern/{c=N}' | tac But this has the same omission as the 'forwards' use case when there are multiple matches within N lines of each other. And it won't work so well when the input is piped from a running process, but it's the simplest way when the input file is complete and not growing.


7

That code doesn't work for previous lines. To get lines before the matched pattern, you need to somehow save the lines already processed. Since awk only has associative arrays, I can't think of an equally simple way of doing what you want in awk, so here's a perl solution: perl -ne 'push @lines,$_; print $lines[0] if /PAT/; shift(@lines) if $.>LIM;' file ...


1

Save the sum to a file called sum, sorted awk -F, '{a[$1]+=$3;}END{for (i in a)print i", "a[i];}' filename | sort > sum cat sum product_a, 515 product_b, 348 product_c, 495 product_d, 27 Join the two files, first column of first file with first column of second (think "keys"); pipe it to awk and print reordered columns, using , as field separator ...


1

Answer for original question With just minor changes to your code: $ awk 'NR==1{$4="app"} ($1 in arr){$4 = $3 - prev3;} {prev3 = $3; arr[$1]; print}' infile site year count app 040130013 1997 34 040130013 1998 55 21 040130019 2006 79 040130019 2007 135 56 040130019 2008 151 16 040130019 2009 172 21 040130019 2010 179 7 The key change is that prev3 = $3; ...


0

You can iterate column by column working on awk index print field: for i in $(seq $(awk '{print NF}' linux.txt | sort -nu | tail -n 1)); do awk -v field=$i '{print $field}' linux.txt | grep -v "^$"; done > LinuxNewOrder.txt $ cat LinuxNeworder.txt linux-test01 LinuxCV LinuxDF linuxER LinuxWE LinuxXC LinuxPL LinuxSD LinuxAQ Some more explanation: awk ...


2

using combination of awk and grep : awk -v RS=" " '{print}' Linux.txt | grep . > LinuxNewOrder.txt more simpler one, using xargs : xargs -n 1 < Linux.txt > LinuxNewOrder.txt


3

You don't need the all.txt file, sed, or a for loop. find and awk can do it all: find /path/to/folder -name data1.txt -exec awk -F': ' '/^Name/ {print $2}' + Using a field separator of : (colon followed by a space character) is the key to getting the output right (no leading space, supports one or more words in the second field) If you want the filename ...


0

$ echo ',,a,,,b,,,,' | sed 's/\(\([^,]\+,\)*\),/\1-,/g; s/\(^\|,\)$/\1-/' -,-,a,-,-,b,-,-,-,- The second expression (the s/\(^\|,\)$/\1-/) is necessary only because sed seems to stop matching when it has reached the end of the pattern space, even though it could still match a $ if the only expression were s/\(\([^,]\+,\)*\)\(,\|$\)/\1-\3/g. Of course, sed ...


-2

This might help your problem solved. (Refer: Awk command I want to compare two rows in two files and update the second file if the row matches) Source.txt has following two lines: OldString NewString Before command execution Target.txt has following lines: OldString ==> NewString This is Target File containing OldString now. OldString is to be ...


1

One way to do it in sed would be to use a loop: $ echo ",,a,,,b,,,," | sed ':a; s/\(^\|,\)\(,\|$\)/\1-\2/; ta' -,-,a,-,-,b,-,-,-,- If your version supports extended regular expressions you can simplify it to a more readable sed -E ':a; s/(^|,)(,|$)/\1-\2/; ta' An alternate way to do it in perl, splitting into comma-separated fields and then mapping ...


1

echo ",,a,,,b,,,," | awk -v 'FS=,' -v 'OFS=,' '{for (i=1; i<=NF; i++) sub(/^$/,"-",$i); print};'


3

(echo ,,a,,,b,,,,; echo a,,b) | perl -pe 's/(^|,)\K(?=(,|$))/-/g' -,-,a,-,-,b,-,-,-,- a,-,b I guess I should learn sed one of these years.


0

Try this awk, it doesn't print the field(rwerq5555525525) from tmp2 file that doesn't match. Perhaps some of the experts can help with that detail. FNR==NR { a[$1]=$2; next; } { s1=split($1,B,"") lineY=""; for(j=1;j<=s1;j++) { if (B[j] ~ /[[:alnum:]]/) { lineY=lineY B[j] } else { ...


0

I did the following and the code worked perfectly: #!/bin/bash PAT=PATH TO THE PARENT FOLDER for i in $(cat $PAT/all.txt); do echo -n "data1" sed -n '/Name:/p' $PAT/$i/data1.txt | awk '{ print $2}'


4

If I understood your question correctly, you could use match first, then RSTART and RLENGTH to extract the matching string and then examine it for the presence of a digit AND a character, thus: awk --re-interval '{match($1, /[[:lower:][:digit:]]{6,10}/); x=substr($1, RSTART, RLENGTH)}; x ~ /[[:lower:]]/ && x ~ /[[:digit:]]/' test_strings Given ...


3

You could try like this: awk 'NR==1{for (i=1;i<=NF;i++) a[i]=$i; next} {for (i=1;i<=NF;i++) {print $i > a[i]".txt"}}' infile On header line this saves the value of each field in an array then for the rest of the lines it prints each field to the corresponding filename.



Top 50 recent answers are included