Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

New answers tagged

0

Tried with below awk and it worked like a charm; the commands are: j=`awk -F "," '{print NF}' filename| sort -nr| sed -n '1p'` for ((i=5;i<=$j;i++)); do awk -v i="$i" -F "," 'BEGIN{sum=0}{sum=sum+$i}END {print "Field"i"=" sum}' filename; done| perl -pne "s/\n/, /g" output: Field5=42, Field6=107, Field7=32, Field8=101, Field9=15, Field10=1, Field11=0, ...


0

I Have also used awk but with different method command awk -F "," 'NR==FNR{a[$1];next}($1 in a){print $0}' file1.txt file2.txt| awk -F "," '{if (!seen[$1]++)print }' output awk -F "," 'NR==FNR{a[$1];next}($1 in a){print $0}' file1.txt file2.txt| awk -F "," '{if (!seen[$1]++)print }' foo,1 boo,1


0

In addition to matsib.dev's excellent answer: Are you sure about the -F flag? It turns off regexes and makes grep search for fixed strings instead. So .* will only hit lines containing a dot followed by an asterisk. Another thing to check is the contents of your file 1. If it has dos-like line-endings (i.e. the lines end with CRLF instead of a single LF), ...


3

A variation of the famous seen idiom. awk -F, 'FNR==NR{a[$1]=1;next} a[$1]++==1' file1 file2


1

Not actually awk, but it works. And I suppose it allows for easy extension. #!/usr/bin/env bash while IFS= read -r line; do grep -m1 "$line" "$2" done < "$1" $ ./script.sh file1 file2 Took the reading line-by-line from https://stackoverflow.com/questions/10929453/read-a-file-line-by-line-assigning-the-value-to-a-variable. Then it's just 'find ...


3

How about $ awk -F, 'NR==FNR {a[$1]; next} $1 in a {print; delete a[$1]}' file1 file2 foo,1 boo,1


0

The is no harm on being just a little tidy. First, I think your are probably confusing regexes with Globbing; and doesn't matter which one is, you don't need to repeat the same line 2 or more times (may be you were trying to show that you have many lines to be interpreted as regexes, but you were lazy to make each line unique... but, just to be sure). So, ...


0

To print the lnb'th line after first blank (empty) line... Line number to print : lnb=2 With sed : sed -ne '/^\s*$/{:a;n;0~'"$lnb"'!ba;p;q}' my_file With perl : perl -ne '/^\s+$/ && $k++;$k!=0 && $k++ && $k=='"$lnb"'+2 && (print,last)' my_file


0

You can use: awk -F"," '{for (i=5;i<=NF;i++){a[i]=a[i]+$i}} END { for (i in a){ print "Field" i "=" a[i]}}' file In your case it will print Field5=42 Field6=107 Field7=32 Field8=101 Field9=15 Field10=1 Field11=0 Field12=0 Field13=0


0

One more awk solution: awk -F\; 'BEGIN {plus=0;minus=0} {if ($2>0) plus+=$2 else minus+=$2} END {minus=-minus; print plus minus}' input_filename


0

Here's a simpler example that illustrates the issue: $ cat input.txt line one line two line three $ for word in $(cat input.txt) ; do echo $word ; done line one line two line three The $(cat input.txt) splits its input on whitespace. (Incidentally, in bash you can replace that with $(<input.txt)). You can use the read built-in command instead: $ while ...


0

$ set -f; IFS=" " $ set $(perl -F\; -lane '$A[$F[1]<0] += $F[1]}{print "@A"' "$file") $ plus=$1 minus=$2


1

plus: awk -F';' '$2~/^[^\-]/{s+=$2} END{print s}' data minus: awk -F';' '$2~/^[\-]/{s+=$2} END{print s}' data


0

For anyone finding this, the retrieveseqs.pl is great, but slower than a similar program I have used for years by John Nash. I cannot find it online, but we have it on our github (https://github.com/NCGAS/Useful-Scripts/blob/master/subset_fasta.pl) [xxxxx@h1 test2]$ time ../retrieveseqs.pl cdna.cds.predupclean cds.list >tmp [3926 (3925/3925 found] real ...


0

Tried with below method command awk '$4 !~ /^NC|^NZ/{print $0}' filename output awk '$4 !~ /^NC|^NZ/{print $0}' o.txt NC_013132.1 7260299 7261429 WP_012793281.1 NC_013132.1 7573559 7574311 WP_012793549.1 NZ_CP022095.2 2884046 2885668 WP_003877393.1


0

Done by below method command:awk '{a[$1]++}END{for(x in a)print a[x]"="x}' filename command: @praveen:~$ awk '{a[$1]++}END{for(x in a)print a[x]"="x}' p.txt 3=1 2=2 1=3 praveen@praveen:~$


1

Use the command awk '{a[$1]++} END { for ( n in a ) { i++; print n "=" a[n]} print i }' file E.g. #!/bin/bash echo '1 2 1 4 5 4' | awk '{a[$1]++} END { for ( n in a ) { i++; print n "=" a[n]} print i }' Output will be 1=2 2=1 4=2 5=1 4


2

In GNU awk, length(my_arry) will display the number of elements your associated my_arry has taken. in other implementations that doesn't support array length, you can do: awk '{ … } END{ for (elements in my_arry) count++; print count }'


1

You need pattern matching operator ~ (or !~ for negation), treating the right hand operand as an (extended) regular expression on the left hand one as a string, so awk -F'\t' '$4 !~ "^(NC|NZ)"' infile Or shorter: awk -F'\t' '$4 !~ "^N[CZ]"' infile and even shorter if you don't have space within a column (since default awk's FS is Tab/space): awk '$4 !~ "...


3

You can simply do that as below. When you use awk with == you are performing a literal string match for equality, in which you cannot do regular expression matches like ^ or $. You can do simply do pattern matching with ~ and turn on the negation match with !. For multiple patterns, use the alternation (pat1|pat2) style supported in ERE awk 'BEGIN { OFS=FS=...


0

One possibility for the regexp you need is: grep -E "jdk([0-9]+).([0-9]+).([0-9_]+)"


1

find /usr/jdk64/* -prune -type d -name 'jdk*' | grep -P 'jdk\d+\.\d+\.\d+_\d+$'


0

you may do this: $ your_df_cmd_here | sed -e '/\s/!N;s/\n\s*/ /'


0

You can do this using GNU sed as shown: $ sed -nEe ' /\[/!{H;$!d;} x;1!s/^\[(prod|dev)_env]\n//p ' hosts.yml


0

while IFS= read -r line; do case $line in *.txt) some_txt_action ;; *.gz) some_gz_action ;; *.Z) some_Z_action ;; *) echo 'Did not match a pattern' esac done <filelist Or, for a bit more precise matching of filename suffixes, while ...


2

awk '$2==0 && $3~/^[3456]$/{print $7}' data update Restrict to certain area: awk '/o 0 1 0 1 1 5.732820000 0.000 = PERIOD\(0\)/ {b=1} b && $2==0 && $3~/^[3456]$/{print $7} b && $3==6 { b=0 }' data


0

awk '/-----(Begin Key|End Key)-----/{sub(/^[^\-]+|[^\-]+$/, "")}1' data /-----(Begin Key|End Key)-----/ find begin or end key line {sub(/^[^\-]+|[^\-]+$/, "")} clear leading and trailing nonsense. 1 always print


1

How about sed -e '/Begin Key/ s/^[^-]*//' -e '/End Key/ s/[^-]*$//' Ex. $ sed -e '/Begin Key/ s/^[^-]*//' -e '/End Key/ s/[^-]*$//' file -----Begin Key----- keep this1 keep this2 keep this3 -----End Key-----


1

Assuming that every line has the same number of fields (columns): awk ' NR==1 { for (i=1; i<=NF; i++) header[i]=$i; next; } { for (i=1; i<=NF; i++) total[i] += $i; } END { for (i=1; i<=NF; i++) print header[i] "=" total[i]+0 } ' Sample input: red green blue 1 2 4 8 16 32 64 128 256 Corresponding output: red=73 ...


0

using awk: awk -F"[\":]" '{ print $5 }' infile


0

Tried with below mentioned sed command and it worked fine too command: sed 's/.*:"//1' filename|sed 's/".*//g' output sed 's/.*:"//1' filename |sed 's/".*//g' 30c962de-b448-40ac-ade8-da6a8f49ce88


2

awk 'BEGIN{RS="[";FS="]\n"} $1 ~/(prod|dev)_env/ {print $2}' ex1


3

An sed solution: sed -nEe '/\[(prod|dev)_env]/!d;N;:loop' -e 's/.*\n//;${p;d;};N;P;/\n\[/D;bloop' hosts.yml /\[(prod|dev)_env]/!d drops all lines until [prod_env] or [dev_env] ist found N;:loop adds the next line and starts a loop inside the loop we remove the first of the two lines with s/.*\n//, because it is either the [...env] line or we already ...


2

You could do (on a mac): tr "\n" "\t" < hosts.yml | sed $'s%\t\\[%\\\n\[%g' | grep 'prod\|dev' | sed $'s%.*\]\\\t%%g' | tr "\t" "\n" tr will get everything on one line The first sed will break line before each section grep will cut out the lines (sections) you want Second sed will remove the [] part. Last tr will break line so you get one hostname ...


0

If this is actually a JSON document, then using jq would pull the string out regardless of the formatting of the data: jq -r '.document[2].part.id' file.json This assumes that the id key is part of a part object, which is in turn part of a particular element in a document array at the top level of the JSON data structure. Or, you could just extract all id ...


0

Another dirty trick using AWK on provided input. data='"id":"30c962de-b448-40ac-ade8-da6a8f49ce88","title": echo $data | awk -F , '{ print $1}' | awk -F id: '{print $1}' | awk '{print $1}' "30c962de-b448-40ac-ade8-da6a8f49ce88"


0

Quick and dirty like this? $ grep -o -P '(?<=")[0-9a-f]{8}-([0-9a-f]{4}-){3}[0-9a-f]{12}(?=")' input 30c962de-b448-40ac-ade8-da6a8f49ce88 Looks for any random "30c962de-b448-40ac-ade8-da6a8f49ce88" values with fixed length and dash - positions, enclosed in quotation marks, and prints it out. If you want to insist on the "id": part too, just add it to ...


6

awk 'BEGIN { c1 = 5 } { print c1, $0 } FNR%200 == 0 { ++c1 }' file This would initiate a variable, c1, to 5 at the start of the awk program. All input lines are then printed prepended by the current value of this variable. For any line whose line number (FNR) is a multiple of 200, this variable would be incremented by 1 after printing the line. The ...


0

Using csplit: $ csplit -s -f page- file '/^Page/' '{1}' $ ls file page-00 page-01 page-02 $ cat page-00 Page 2 line 1 line 2 line 3 line 4 line 5 $ cat page-01 Page 2 line 1 line 2 line 3 line 4 line 5 $ cat page-02 Page 3 line 1 line 2 line 3 line 4 line 5 The csplit utility splits the given file according to its last arguments. Here, /^...


0

command: awk '$0 ~/Page 1/{f=1}$0 ~/Page 2/{f=0}f' o.txt > firstfile awk '$0 ~/Page 2/{f=1}$0 ~/Page 3/{f=0}f' o.txt > secondfile awk '/Page 3/,/,/{print $0}' o.txt >thirdfile output een@praveen:~$ awk '$0 ~/Page 1/{f=1}$0 ~/Page 2/{f=0}f' o.txt > firstfile Page 1 line 1 line 2 line 3 line 4 line 5 praveen@praveen:~$ awk '$0 ~/Page 2/{f=...


0

Command: awk '$3!=$4 && $4!=$5&& $5 !=$3 {print $0}' filename output awk '$3!=$4 && $4!=$5&& $5 !=$3 {print $0}' filename description ID valueA valueB valueC yyy y 3 100 23


0

$ awk -F'|' ' BEGIN { getline; split($0,a); OFS=FS; print "EMP_ID", "EMP_NAME", "Values", "HeaderName" } { for (i=3;i<=NF;i++) print $1, $2, $i, a[i]; } ' sample EMP_ID|EMP_NAME|Values|HeaderName 123456|XXXXXXXXX|30|AGE 123456|XXXXXXXXX|10000000|SALARAy 123456|XXXXXXXXX|1234|DepId 123456|XXXXXXXXX|XYZWE|DepName If you ...


0

You can print data from a range of columns with a construct like this : echo {0..9} | awk '{ for(i=3;i<8;i++) {printf $i"|"} }' outputs : 2|3|4|5|6|


0

date0='20190501' date1='20190622' numDays=$(( ( $(date -d "$date1" +'%s' ) - $(date -d "$date0" +'%s' ) ) / (60*60*24) )) for day in $( seq 0 $((numDays-1)) ); do d=$(date -d "$date0 + ${day}days" +"%Y-%m-%d") echo $d $( find . -maxdepth 1 -type f -newermt "$d" ! -newermt "$d + 1day" | grep '.' -c ) done Get number of days by difference between date0 ...


1

Get the headers and save it in an array: NR==1 { for (i=1;i<=NF;i++) h[i]=$i } then you can simply add them using h[i] awk -F\| ' BEGIN { print "EMP_ID|Values|HeaderName" } NR==1 { for(i=2;i<=NF;i++) h[i]=$i } NR>1 { for(i=2;i<=NF;i++) print $1"|"$i"|"h[i] } ' records.txt


0

Try this code: TIME_STAMP=(`find . -maxdepth 1 -type f -printf '%TY-%Tm-%Td\n' | sort | sed -e 1b -e '$!d'`) LIST () { date1=$1 date2=`date -d "$date1 + 1 day" +"%Y-%m-%d"` find . -maxdepth 1 -type f -newermt $date1 ! -newermt $date2 | echo "$date1 `wc -l` " [ $date1 == $2 ] && exit 0; LIST $date2 $2 } LIST ${...


-1

Using awk awk -F':' '{print $2}'


1

Using GNU seq: seq -f 1_number_density_per_atom_%g.xvg 653 > test.dat


0

Answering the clarified question in the comments about how to change the date format in a filename, use a variant of drewbenn's gnu date approach. This command finds all files in the local folder, and pipes only the file name to xargs, substituting the date modified filename as the second argument to mv which performs the rename. The additional, slightly ...


0

Use awk, read key:value pair as record, read key, value as 1st and 2nd field. awk -v RS=' *, *' -v FS=' *: *' '$1=="KEY2"{print $2}' <<<$str -v RS=' *, *' set record separator to , and it's surrounding space -v FS=' *: *' set field separator to : and it's surrounding space '$1=="KEY2"{print $2}' print value if key found. change "KEY2" to your ...


Top 50 recent answers are included