Tag Info

Hot answers tagged

13

You can make smart use of the NF variable in awk awk '{print $NF}' File1 From man awk NF The number of fields in the current input record. So NF will give you the amount of fields and $NF will then expand to $3 for example, which you can use in a print statement.


11

It happens because a computer only has a limited precision when dealing with numbers. And the available precision uses a binary format to represent the number. This makes numbers that appear to be trivial to write in our decimal system only representable as an approximation (see the Wikipedia entry on this): e.g. 0.1 (as in 1/10) is really stored as ...


9

In an if statement, you have an else. If if doesn't match, the else branch is executed. In a conditional statement, both actions are executed, regardless of the condition is true or false. A simple fix: $ awk '$2!=1 {print $1,"0";next} {print $0}' file PRO 1 GLN 0 ILE 0 THR 0 And you can make it more concise: $ awk '$2 != 1 {print $1,"0";next}1' file ...


7

One way to do it using paste since your files have the same number of lines. paste file1 file2 | awk '{s += ($1-$2)^2}; END{print (s+0)/NR}' 0.0228667


7

Using gawk and assuming that the year always ends the record: awk -F"[0-9]{4}$" '{print $1}' movies


7

The second block in awk '$2!=1 {print $1,"0"} {print $0}' file isn't conditional. It is acted upon for every line, and thus prints every line. Instead, write: awk '$2!=1 {print $1,"0"} $2==1 {print $0}' file Or write: awk '$2!=1 {print $1,"0"; next} {print $0}' file This will cause the unconditional block to be skipped, if the conditional block is ...


6

Aside from functions, awk code looks like condition {actions} If the {actions} block is missing, the implicit action if the condition returns true is {print}. If the condition is missing, and this directly relates to your question, it is implicitly true for every line of the input. That's why you get that line printed so many times. You need to specify ...


6

bash: while read -r line; do if [[ $line =~ (.*)[[:blank:]]+[0-9]{4}$ ]]; then echo "${BASH_REMATCH[1]}" fi done < data sed: sed 's/[[:blank:]]\+[0-9]\{4\}$//' < data


6

I'm not completely sure what your if is trying to do there. NR is the number of records; use NF for the number of fields, if that's what you're aiming for. You can't put {} blocks in the middle of things like that. I think what you're aiming for is to compare the value of a field in this line with a field in the previous line, printing out the sum when we ...


6

You seem to be hoping that (NF","$2 -1) will be treated as a function that will return the number of comma-delimited elements in field $2 - it won't. NF is always the number of fields in the record. Instead, you can use awk's split function split($2,a,",") which splits field $2 into an array a and returns the number of elements. You can also tidy up the ...


6

You should use -v option of awk: awk -F',' -v accNum="$1" '$1 == accNum {print $3,$2}' With $1 in double quote, the shell will handle all its special character ( if has ) for you.


6

You have: $NF=a[FNR] as the final condition (the one that determines whether to print). Assignments return the value assigned, in this case a[FNR]. The first line of the data file from the linked question is: A 0 a[FNR] is initialised to $2. That means the value of a[FNR] is 0, which is a false value to awk. That means the assignment is false, which ...


6

You must use regex ^ to denote start of string: $ awk '$2 ~ /^[[:upper:]]/' file ID A56 DS AGE 56


6

Sed can't do arithmetic┬╣. Use awk instead. awk ' $4 == "calc" {sub(/calc( |\t)/, sprintf("%-6.2f", $3 - $2))} 1' The 1 at the end means to print everything (after any preceding transformation). Instead of the text substitution with sub, you could assign to $4, but doing so replaces inter-column space (which can be any sequence of spaces and tabs) ...


5

If your system has the GNU version of sed, you can use the GNU extension r command: r filename As a GNU extension, this command accepts two addresses. Queue the contents of filename to be read and inserted into the output stream at the end of the current cycle, or when the next input line is read. Note that if filename cannot be read, it ...


5

There's a part you can easily improve, but it isn't the slowest part. find /home/mydir/ -type f | sort | \ awk "/xml_20140207_000016.zip/,/xml_20140207_235938.zip/" This is somewhat wasteful because it first lists all files, then sorts the file names and extracts the interesting ones. The find command has to run to completion before the sorting can ...


5

This is really quite simple. As long as the last field, the year, does not contain any whitespace (this is not clear from your question but I am assuming it is the case), all you need to do is remove the last field. For example: $ cat movies Casablanca 1942 Eternal Sunshine of the Spotless Mind 2004 He Died with a Felafel in His Hand ...


5

$ awk -F '[\t,]' '{print $1, NF-1}' some_file where -F sets the field separator, i.e. either tab or comma $1 references the first field NF is a built-in variable that contains the number of fields in a record the awk statement is executed for each record (i.e. for each line)


5

An ugly way of doing this (i.e. causing a function call in shell based on output from awk) could look like this: awk -F '\t' ' FNR < 2 {next} FNR == NR { for (i=2; i <= NF; i++) { if (($i == 1) || ($i == 4)) printf "retrieve %s\n", $i if (($i == 2) || ($i == 2)) printf "retrieve2 ...


5

You could use awk as @Gnouc suggested, or GNU grep grep -P '^[^\s]+\s+[A-Z]' file Perl perl -lane 'print if $F[1]=~/^[A-Z]/' file GNU sed sed -rn '/^[^\s]+\s+[A-Z]/p' file shell (assumes a recent version of ksh93, zsh or bash) while read -r a b; do [[ $b =~ ^[A-Z] ]] && printf "%s %s\n" "$a" "$b"; done < file


5

awk '$1 ~ /^E|^F/ {if ($1 == "End") print $1" "$2; if ($1 == "Fin") print $1" "$3}' or awk '/^End/{print $1" "$2}/^Fin/{print $1" "$3}' (thanks to Jidder) Should work.


4

awk 'FNR==NR { file1[NR]=$1; next; }; { diff=$1-file1[FNR]; sum+=diff^2;}; END { print sum/FNR; }' file1 file2


4

Use gsub: a="YYYY-MM-DD" b=a gsub("-", "", b) print(b) will output: YYYYMMDD gsub replaces the first argument with the second in the third, in-place, so we copy the value of a into b first. We replace the - characters with nothing.


4

Some quick ideas; If all the files are in a single directory, you can get rid of the find Your file name convention sorts itself by date, so you don't need the sort bit either With this two pieces out of the way, and if the date range is known, you can use a simple filename glob instead of awk. For example (assuming your shell is bash): All files of a ...


4

Try this: $ awk 'FNR==NR{a[$1]=$0;next} {print a[$1]}' file2 file1


4

I think it can be a bug in your script, not awk itself. The situation which awk behave like this is when double newline is set to RS variable: awk '{print $1}' RS='\n|\n\n' file You should check if your script had changed the value of RS.


4

> awk -v OFS="'^'" -F"'\\\\^'" '{if(length($2)>20) $2=substr($2,1,20); print;}' file 'XYZ843141'^'ASDFSAFXYVFSHGDSDg s'^'BAAAR'^'YYY'^'..... and so on, further columns 'YYZ814384'^'ASfdEtRiuognfnseaFRE'^'foooobaaar'^'ZZZ'^'..... and so on, further columns


4

The use of getline throws away the first line. Try instead: awk '{n=0;for(i=NF;i>0;i--)n+=$i; print "sum: " n;}' By its nature, awk will iterate over input lines implicitly. Consequently, the code n=0;for(i=NF;i>0;i--)n+=$i; print "sum: " n; will be run for every line of input. There is no need here to explicitly loop over the lines.


4

As a way around this, you could use a program that is specifically designed to handle arithmetic operations like bc: $ awk '{printf "%s + ",$1}' file | sed 's/\+ $/\n/' | bc 0 If, as seems to be the case, you have a fixed number of decimal places, you could simply remove them to work with integers and then add them again at the end: $ awk ...


3

With awk this would be awk 'NR >= 1200 && NR <= 1300' with sed: sed -n '1200,1300 p' FILE with head and tail: head -n 1300 FILE | tail -n 100 so many options, so many answers on stackexchange :)



Only top voted, non community-wiki answers of a minimum length are eligible