5

When awk receives "0" as input, it behaves differently in some cases. Code below:

var=$1
echo ""; echo -n 'o/p of $1=$1 ==>'; echo $var | awk '$1=$1'
echo "";echo -n 'o/p of {$1=$1;print} ==>';echo $var | awk '{$1=$1;print}'
echo "";echo -n 'o/p of $1==$1 ==>';echo $var | awk '$1==$1'
echo "";echo -n 'o/p of {$1==$1;print} ==>';echo $var | awk '{$1==$1;print}'

The output with "0" (number zero):

[root@host ~]# sh /tmp/te.sh 0

o/p of $1=$1 ==>
o/p of {$1=$1;print} ==>0

o/p of $1==$1 ==>0

o/p of {$1==$1;print} ==>0
[root@GORJALA ~]#

The output with "1" (number one):

[root@host ~]# sh /tmp/te.sh 1

o/p of $1=$1 ==>1

o/p of {$1=$1;print} ==>1

o/p of $1==$1 ==>1

o/p of {$1==$1;print} ==>1
[root@host ~]#

Why is there a difference when I use var=0; echo $var | awk '$1=$1' and var=1; echo $var | awk '$1=$1'? All numbers are working fine other than 0.

Versions:

  • GNU bash, version 4.2.46
  • GNU Awk 4.0.2
  • coreutils-8.22-24.el7.x86_64
  • 2
    You example is using assignments (=) and comparisons (==) in the context of conditions (CONDITION { action }) and in the context of actions (condition { ACTION }) and showing comparisons of the behaviors of all combinations of that, even combinations that make no sense (e.g. {$1==$1;print}) but then your question is specifically just Why there is a difference when I use var=0; echo $var | awk '$1=$1' and var=1; echo $var | awk '$1=$1' so why include all that other stuff and get people explaining things that are irrelevant to your question? – Ed Morton Apr 4 at 13:56
16

From the The GNU Awk User’s Guide:

An assignment is an expression, so it has a value—the same value that is assigned. Thus, ‘z = 1’ is an expression with the value one.

So

  • echo 0 | awk '$1=$1' the pattern evaluates to 0 (FALSE)

  • echo 1 | awk '$1=$1' the pattern evaluates to 1 (TRUE) and the default action print is executed

| improve this answer | |
7

I don't think it is a matter of the numerical value: the standard conversions take care of that (here, at least).

The OP shows four different awk codes, all variations on: pattern { action }

(a) $1 = $1

That reassigns $1 to itself. It is not a boolean test, it is a no-op (effectively), and it returns the value of $1. If $1 is a 0, the pattern is false and the default print action is skipped completely. If $1 is non-zero, the input is printed.

(b) { $1 = $1; print; }

That reassigns $1 to itself, also a no-op. In the absence of a pattern, the action is performed and the input is always printed.

(c) $1 == $1

That is a boolean expression that is always true. 0 is 0 and 1 is 1 (and aardvark is aardvark). In the absence of an action, the input is always printed.

(d) { $1 == $1; print; }

There is no pattern. The comparison evaluates to a true boolean which is discarded. The input is always printed.

| improve this answer | |
  • 1
    Nit-pick: $1 = $1 isn't a no-op. It'll cause awk to reconstruct $0, discarding leading/trailing blanks if the default FS is used, and replacing all FSs with OFSs. – Ed Morton Apr 4 at 12:59
  • 1
    @EdMorton Absolutely correct, and useful tool in the box sometimes. I said "effectively" to try to keep it simple, as in the posted example it made no difference. – Paul_Pedant Apr 4 at 13:07
  • You should not rely on $1==$1 being true all the time, there are conditions with nan that should not be true, even if the numerical value on both sides is equal (and equal to -nan). – Isaac Apr 8 at 7:50
5

The existing answers fail to explain why

echo 0 | awk '$0="0"'
echo 0 | awk '$0=substr($0,1)'
echo 0 | awk '$0=$0""'

will all print 0, but

echo 0 | awk '$0'
echo 000 | awk '$0'

won't print anything, though in all the cases, the pattern expression evaluates to 0.

How come 0 is true in one case and false in the other?

That's because the "field variables" (the result of the $ operator) are treated as a special case, and (if possible) are automatically converted to numeric strings, which, if numerically equal to 0, will be considered false when used in a boolean context:

A string value shall be considered a numeric string if it comes from one of the following:

  1. Field variables

  2. Input from the getline() function

  3. FILENAME

  4. ARGV array elements

  5. ENVIRON array elements

  6. Array elements created by the split() function

  7. A command line variable assignment

  8. Variable assignment from another numeric string variable

and [if it looks like a number, read the whole description here]

Please also read the RATIONALE for the reasons why the concept of numeric strings and this special-casing was needed, especially the bit about a comparison like echo 0 000 | awk '$1==$2' being true, but not echo 0 | awk '$1=="000"'.


As another quirk, notice that, at least in some implementations, $0 (the current input record) loses its magical "numeric string" property if an assignment to a subfield causes it to be recomputed:

$ echo 0 | gawk '{$1=0} $0'
0

This does not seem to be covered by the standard, though it matches the behaviour of nawk/bwk the standard awk is based on (but not that of mawk).

Also, awk implementations are allowed to recognize NAN, INF and INFINITY in the input as the corresponding floating point numbers, though support for this is spotty and inconsistent. You may still be bitten by eg.

echo But his daughter named Nan | awk '$NF'

not printing anything in FreeBSD's awk (bwk, original-awk).

| improve this answer | |
  • 1
    No, in all the cases, the pattern expression evaluates to 0 ... no it doesn't. In the first three cases they evaluate to 1. An awk '0' will never print. An awk '1' will always print. Similarly, an awk ' "" ' will never print (null) and an awk ' "0" ' will always print (length("0") > 0). – Isaac Apr 3 at 19:15
  • The quote you need from the spec is: When an expression is used in a Boolean context, if it has a numeric value, a value of zero shall be treated as false and any other value shall be treated as true. Otherwise, a string value of the null string shall be treated as false and any other value shall be treated as true. 0 is false, but "0" is true. – Isaac Apr 3 at 19:15
  • Hmm, erasing your comments? In awk '$1="0"' the value comes from a section of program code ("0"), not a field variable. And "0" is an string, that string is being assigned to $1 and the final value of the assignment is still "0". In the Boolean context it is being evaluated, the boolean value of "0" is 1 and that is why the input is getting printed. – Isaac Apr 3 at 19:42
  • 1
    No. from your comment: in FreeBSD's awk, doesn't print anything (tested on FreeBSD 12) but busybox (the command you wrote) does print. So, is there is a bug in FreeBSD awk, then? – Isaac Apr 4 at 5:35
  • 1
    @mosvy About your: can only have two types: string or numeric, and it cannot have both at the same time. Yes, In Awk the value of a variable is both a number and an string and it is converted as needed. That is the reason for having CONVFMT for example. But more specific to your point: What is the output of: echo 000 | awk '{print ($1=="000"), ($1==0)}' ? Is it 1 1, so, both tests are true, the same variable is both text and a number. Converted as needed. – Isaac Apr 5 at 6:55
4

Because $0 is the whole record (complete line), $1, $2, are fields (usually separated by whitespace) in it.

| improve this answer | |
2

simple cases

Lets simplify first.

What the code awk '$1=$1' does could be understood by printing the value of $1=$1. As well as what the code awk '$1==$1' does could be understood by printing its value. Both fall into the awk definition of:

pattern   { action }

If the action part is missing, the default action of print is executed. Thus, an awk '1' will print all input lines. An awk '0' will print none.

So, the value of $1=$1 and $1==$1 would be printed by this:

$ printf '%s\n' 0 1 | awk '{print $1=$1, $1==$1}'
0 1
1 1

Therefore, for an input of 0 a pattern of $1=$1 will not print the input line. For 1 (and any other integer value) it will.

The $1==$1 is simple: it is (almost[a]) always true.

The other options you present awk '{...,print}' will always print because there is no pattern, and the default is to execute the code inside the brace, and (unless there is a command to exit awk) the last action: print will always be executed.

Of the four different options you present, only when the input is 0 and the code is awk '$1=$1' the input will not be printed. Exactly what you've got.

more complex

What seems to be in discussion is what this should do:

echo 0 | awk '$0="0"'      # true as "0" is a non-null string.

Has an string "0" assigned to the whole input ($0), the output of such assignment is also an string ("0"). An string having anything other than null means true. So, yes, that would print the input, but not because it is a number 0 but because it is an string "0".

When to convert

Variable values in awk have a dual type: number and string.

The value of a variable could be given explicitly in the written code, like

awk '{a=1234; b=1e-3; c="string"; d="1234"}'

Assignment to numeric values generate a numeric variable.
Assignment to an string inside quotes "..." generate an string variable.

So, c and d are strings and a and b are numbers (which could come from two conversions: integer (strtod) and float (strtof) ).

The problem starts when a variable receives "user input", as when a field is read by the first time. What a echo 000 | awk '{print $1}' should print: 000 or 0 ? is it a numeric value 0 or an string 000 that happens to look like a number ?.

That is where conversion starts, conversion is required both to obtain a number from an string and to obtain an string (that could be compared) from a number. In general, only "user input" needs conversion, it is assumed that the code as written contains the correct type (either a=123 or a="123"). And conversions could be forced by adding zero (var+0) or by concatenating an (maybe empty) string (var"").

trouble cases

[a]

  1. A string is always equal to itself, no matter what.
  2. A numeric value is always equal to itself except if it is nan (sometimes).

    Even if $1 is a nan (+inf -inf, or 0*inf, or some others) most awk implementations (nawk, mawk, original-awk and bsd awk) will claim that $1==$1 is true. That is against the IEEE754 spec, which requires that a NaN should not be equal to anything. So, that is an awk (most awk) bug. Except busybox awk that would not claim that $1==$1 is true if $1 is-nan, I can not confirm that that is by design as I have not reviewed their source code.

    echo '-nan' | awk '$1==$1'
    

    So, it is true that the code above would take $1==$1 as true but that might (shall) not be true in the future.

  3. Conversion is needed.

    If what is being compared are two strings or two numbers, no conversion is needed.
    When the type is mixed, conversion must be performed.
    What is generally implemented is that if an string looks like a number ("123") (called strnum in GNU awk) and comes from external input (form code values, no default conversion is performed) then it is converted to a number and a==b is performed numerically. Otherwise, the comparison is performed as strings.

So:

echo 0 | awk '$0="0"'

Is always an string ("0") and the result is true.

But:

echo 0   | awk '$0'
echo 000 | awk '$0'

are both "external input" and look like a number, so both are converted to numbers, and as the value of 0 or 000 is the numerical 0, the result of the pattern is false and both will not print.

Except, again that if the input value is a numeric NaN (yes, numeric) and the flaw in awk to not follow IEEE754 is corrected, then, this, which prints in many awk implementations:

echo '-nan' | awk '$0'

might stop printing.

Note that this happens in FreeBSD:

$ echo 'test -nan' | 
    original-awk '{print $2,($2==1),($2==0),$2+0,$2*0,($2==$2)}$2'
-nan 1 1 -nan -nan 1

A -nan is equal to 1 and equal to 0 and doesn't print test.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.