From my understanding, $1 is the first field. But strangely enough, awk '$1=$1' omits extra spaces.

$ echo "$string"
foo    foo bar               bar

$ echo "$string" | awk '$1=$1'
foo foo bar bar

Why is this happening?


When we assign a value to a field variable ie. value of $1 is assigned to field $1, awk actually rebuilds its $0 by concatenating them with default field delimiter(or OFS) space.

we can get the same case in the following scenarios as well...

echo -e "foo foo\tbar\t\tbar" | awk '$1=$1'
foo foo bar bar

echo -e "foo foo\tbar\t\tbar" | awk -v OFS=',' '$1=$1'

echo -e "foo foo\tbar\t\tbar" | awk '$3=1'
foo foo 1 bar

For GNU AWK this behavior is documented here:

$1 = $1 # force record to be reconstituted

  • 10
    Don't rely on awk '$1=$1' printing the current record after recompiling it, try echo -e "0\tbar\t\tbar" | awk '$1=$1'. Always do awk '{$1=$1}1' instead and in general only use an action in a conditional context if you need the result of that action to be evaluated as a condition. The only other thing worth mentioning is that assigning to a field will also remove all leading and/or trailing spaces from the record when you use the default FS.
    – Ed Morton
    Feb 20 '20 at 15:22
  • 1
    What does the last 1 signify in awk '{$1=$1}1'? @EdMorton
    – annahri
    Feb 21 '20 at 2:29
  • 4
    @annahri it’s a pattern, which always evaluates successfully, and executes the default action (which prints the current record). Adding 1 is a common AWK trick to print the current record, but it does make the program harder to understand for people unfamiliar with the trick in question. Feb 21 '20 at 10:19
echo "$string" | awk '$1=$1'

causes AWK to evaluate $1=$1, which assigns the field to itself, and has the side-effect of re-evaluating $0; then AWK considers the value of the expression, and because it’s non-zero and non-empty, it executes the default action, which is to print $0.

The extra spaces are removed when AWK re-evaluates $0: it does so by concatenating all the fields using OFS as a separator, and that’s a single space by default. When AWK parses a record, $0 contains the whole record, as-is, and $1 to $NF contain the fields, without the separators; when any field is assigned to, $0 is reconstructed from the field values.

Whether AWK outputs anything in this example is dependent on the input:

echo "0      0" | awk '$1=$1'

won’t output anything.

  • Hi, can you explain why it doesn't print anything? BTW as suggested elsewhere echo "0 0" | awk '{$1=$1} print' will print the expected output.
    – Hastur
    Feb 21 '20 at 19:56
  • @Hastur echo "0 0" | awk '{$1=$1} print' doesn’t print the expected output, try it ;-). echo "0 0" | awk '$1=$1' doesn’t print anything because $1=$1 then evaluates to 0, so the corresponding action isn’t taken. awk '{$1=$1}1' works instead by applying $1=$1 as an action, not a pattern, and then taking the pattern 1, which always succeeds, and running the corresponding action (the default action, print). Feb 21 '20 at 22:04
  • zZZzz ;-). I missed the {} (or the ;). Both echo "0 0" | awk '{$1=$1; print}' and echo "0 0" | awk '{$1=$1} {print}' work. So it depends only from the fact that $1 is 0... then false. Thanks.
    – Hastur
    Feb 21 '20 at 22:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.