4

I was trying to assign below command(which choose the random line from file) to a variable, but not working.

givinv@87-109:~$ head -$((${RANDOM} % `wc -l < file` + 1)) file | tail -1
cower
givinv@87-109:~$

Below the the error I am getting while trying to assign it to a variable.

givinv@87-109:~$ VARIA=`head -$((${RANDOM} % `wc -l < file` + 1)) file | tail -1`
bash: command substitution: line 1: unexpected EOF while looking for matching `)'
bash: command substitution: line 2: syntax error: unexpected end of file
bash: command substitution: line 1: syntax error near unexpected token `)'
bash: command substitution: line 1: ` + 1)) file | tail -1'
-l: command not found
givinv@87-109:~$

I even tried same with for loop and not working::

givinv@87-109:~$ for i in `head -$((${RANDOM} % `wc -l < file` + 1)) file | tail -1`;do echo $i ;done
bash: syntax error near unexpected token `<'
givinv@87-109:~$ 
  • 4
    You can use the command rl, if you don't wanna reinvent the wheel. – Ipor Sircer Nov 28 '16 at 10:40
  • 4
    @IporSircer, or GNU shuf – Stéphane Chazelas Nov 28 '16 at 11:22
  • 3
    Note that $RANDOM's maximum value in bash is 32767, so you would not be able to reach lines greater than that. Also note that it would not give you a uniform random distribution unless the number of lines in the file is a divisor of 32768 (is a power of 2). – Stéphane Chazelas Nov 28 '16 at 11:26
15

It's not working because you are attempting to nest unescaped backticks:

VARIA=`head -$((${RANDOM} % `wc -l < file` + 1)) file | tail -1`

That actually attempts to run head -$((${RANDOM} % as a single command first, and that gives you the 2 first errors:

$ VARIA=`head -$((${RANDOM} % `
bash: command substitution: line 1: unexpected EOF while looking for matching `)'
bash: command substitution: line 2: syntax error: unexpected end of file

Then, it tries to run

wc -l < file` + 1)) file | tail -1`

Which means it tries to evaluate + 1)) file | tail -1 (which is between the backticks), and that gives you the next errors:

$ wc -l < file` + 1)) file | tail -1`
bash: command substitution: line 1: syntax error near unexpected token `)'
bash: command substitution: line 1: ` + 1)) file | tail -1'

You can get around this by escaping the backticks:

VARIA=`head -$((${RANDOM} % \`wc -l < file\` + 1)) file | tail -1`

However, as a general rule, it is usually better not to use backticks at all. You should almost always use $() instead. It is more robust and can be nested indefinitely with a simpler syntax:

VARIA=$(head -$((${RANDOM} % $(wc -l < file) + 1)) file | tail -1)
2

just use this command

VARIA=$(head -n "$((${RANDOM} % $(wc -l < test) + 1))" test | tail -n 1)

to assign the result of a command to a variable we use $(...) (the ancient `...` form is harder to nest).

2

As another option for reading a random line from a file (and assigning it to a variable), consider a simplified reservoir sampling method, converted from perl to awk, with Peter.O's seeding improvement:

VARIA=$(awk -v seed=$RANDOM 'BEGIN { srand(seed) } { if (rand() * FNR < 1) { line=$0 } } END { print line }' /usr/share/dict/words)

Here's the awk script, wrapped nicely:

awk -v seed=$RANDOM '
BEGIN { 
  srand(seed) 
}
{ 
  if (rand() * FNR < 1) { 
    line=$0
  } 
}
END { 
  print line 
}' /usr/share/dict/words

because of the way awk's srand() works, you'll get the same value if you run this script within the same second, unless you seed it with something else random. Here I'm selecting words from /usr/share/dict/words, just as a source of text.

This method does not care how many lines are in the file (my local copy has 479,828 lines), so it should be pretty flexible.

  • After looking at this for six to eight minutes, I believe that I understand the math — although I’m not confident that I understand it well enough to explain it.  Do you understand it well enough to explain it, and, if so, would you please do so (or at least link to such an explanation)? – G-Man May 8 at 5:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.