3

I saw this line while reading a blog on IFS that is :

for i in $(<test.txt)

And thought that $(<test.txt) prints the file contents to STDOUT. I maybe wrong in this, but out of curiosity I tried to do it on shell. So picked up a random file named array having random data and

First did a cat array that gave me this :

amit@C0deDaedalus:~/test$ 
amit@C0deDaedalus:~/test$ cat array
1)      Ottawa  Canada          345644
2)      Kabul   Afghanistan     667345
3)      Paris   France          214423
4)      Moscow  Russia          128793
5)      Delhi   India           142894

And then did $(<array) that gave me this :

amit@C0deDaedalus:~/test$ $(<array)
1)      Ottawa  Ca: command not found

I only know that < is used for input redirection but not getting exactly what is being interpreted by shell as a command here.

Can anyone explain the concept behind this weird output in shell ?

Update :-

On running set -x it gives this :

amit@C0deDaedalus:~/test$ $(<array)
+ '1)' Ottawa Canada 345644 '2)' Kabul Afghanistan 667345 '3)' Paris France 214423 '4)' Moscow Russia 128793 '5)' Delhi India 142894
+ '[' -x /usr/lib/command-not-found ']'
+ /usr/lib/command-not-found -- '1)'
1): command not found
+ return 127
amit@C0deDaedalus:~/test$ 
  • It should have directly thrown that error, but why after printing this much part 1) Ottawa Ca ? – C0deDaedalus Apr 28 '18 at 10:02
  • 2
    I cannot reproduce that with 4.4.19. I get the error only for 1). My bash does not truncate long not found names either. Execute set -x and run $(<array) again. Maybe the debug output is helpful. – Hauke Laging Apr 28 '18 at 10:06
  • I am running these commands on 4.4.0-121-generic. – C0deDaedalus Apr 28 '18 at 11:13
17

The $(command) syntax executes command in a subshell environment and replaces itself with the standard output of command. And, as Bash Manual says, $(< file) is just a faster equivalent of $(cat file) (that's not a POSIX feature, though).

So when you run $(<array), Bash performs that substitution, then it uses the first field as the command's name and the rest of the fields as command's arguments:

$ $(<array)
1): command not found

I don't have any 1) command/function, so it prints an error message.

But in your specific scenario, you are getting a different error message probably because you modified the IFS variable:

$ IFS=n; $(<array)
1)      Ottawa  Ca: command not found

Edit 1

My guess is that your IFS was somehow modified, so that's why your shell tried to execute 1) Ottawa Ca instead of 1). After all, you were reading an IFS-related article. I wouldn't be surprised if your IFS ended up with a weird value.

The IFS variable controls what is known as word splitting or field splitting. It basically defines how the data will be parsed by the shell in an expansion context (or by other commands like read).

Bash manual explains this topic:

3.5.7 Word Splitting

The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.

The shell treats each character of $IFS as a delimiter, and splits the results of the other expansions into words using these characters as field terminators. If IFS is unset, or its value is exactly <space><tab><newline>, the default, then sequences of <space>, <tab>, and <newline> at the beginning and end of the results of the previous expansions are ignored, and any sequence of IFS characters not at the beginning or end serves to delimit words. If IFS has a value other than the default, then sequences of the whitespace characters space, tab, and newline are ignored at the beginning and end of the word, as long as the whitespace character is in the value of IFS (an IFS whitespace character). Any character in IFS that is not IFS whitespace, along with any adjacent IFS whitespace characters, delimits a field. A sequence of IFS whitespace characters is also treated as a delimiter. If the value of IFS is null, no word splitting occurs.

Explicit null arguments ("" or '') are retained and passed to commands as empty strings. Unquoted implicit null arguments, resulting from the expansion of parameters that have no values, are removed. If a parameter with no value is expanded within double quotes, a null argument results and is retained and passed to a command as an empty string. When a quoted null argument appears as part of a word whose expansion is non-null, the null argument is removed. That is, the word -d'' becomes -d after word splitting and null argument removal.

Note that if no expansion occurs, no splitting is performed.

Here are some examples about IFS and command substitution usage:

Example 1:

$ IFS=$' \t\n'; var='hello     world'; printf '[%s]\n' ${var}
[hello]
[world]

$ IFS=$' \t\n'; var='hello     world'; printf '[%s]\n' "${var}"
[hello     world]

In both cases, IFS is <space><tab><newline> (the default value), var is hello world and there's a printf statement. But note that in the first case word splitting is performed, while in the second case it is not (because double-quotes inhibit that behavior). Word splitting occurs in non-quoted expansions.

Example 2:

$ IFS='x'; var='fooxbar'; printf '[%s]\n' ${var}
[foo]
[bar]

$ IFS='2'; (exit 123); printf '[%s]\n' ${?}
[1]
[3]

Neither ${var} nor ${?} contain any whitespace character, so one may think that word splitting wouldn't be an issue in such cases. But that's not true because IFS can be abused. IFS can hold virtually any value and it's easy to abuse.

Example 3:

$ $(echo uname)
Linux

$ $(xxd -p -r <<< 64617465202d75)
Sat Apr 28 12:46:49 UTC 2018

$ var='echo foo; echo bar'; eval "$(echo "${var}")"
foo
bar

This has nothing to do with word splitting, but note how we can use some dirty tricks to inject code.

Related questions:

  • Could you please explain the IFS variable modify part. – C0deDaedalus Apr 28 '18 at 11:02
  • @C0deDaedalus Just edited the answer. – nxnev Apr 28 '18 at 12:49
  • You are very true. It was a messed up $IFS issue. Anyways thanks for your answer. – C0deDaedalus Apr 28 '18 at 13:43
  • 3
    @C0deDaedalus xxd is a tool that converts text to hexadecimal and vice versa. 64617465202d75 is the hex representation of date -u. So $(xxd -p -r <<< 64617465202d75) is just an obscure and obfuscated way to execute date -u. Imagine doing that with something like rm -rf / instead. Such non-intelligible command would be pretty dangerous for an inexperienced user lead by his/her curiosity. – nxnev Apr 28 '18 at 14:01
  • 2
    I want to upvote this several times. – glenn jackman Apr 28 '18 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.