I'm going to give a short version of the kind of issue I'm facing. The real application I'm trying to write has many parts, so I've tried to isolate the part where I'm stuck with a small example. My situation, which involves file ID numbers, is similar to a hypothetical list of Student ID Numbers parsed from a directory or something.
What I Start With
In order to try and take advantage of bash brace expansion, something like a student directory has been parsed to give the following:
$ cat student_id_potential_expansions
0000{11,24}
0001{32,38,81}
0002{02,80,81,89,97}
Desired behavior
I want to use this input to create my own (Bourne Again) shell script which looks something like the following.
#!/bin/bash
get_transcripts_for_id.sh -id 000011
get_transcripts_for_id.sh -id 000024
get_transcripts_for_id.sh -id 000132
get_transcripts_for_id.sh -id 000133
get_transcripts_for_id.sh -id 000181
get_transcripts_for_id.sh -id 000202
...
and so on
I'm not too worried about the command before; I figured I'd just use an
awk '{print "get_transcripts_for_id.sh -id" $0}' student_id_list > my_shell_script.sh
or a
sed 's/^\(.*\)$/get_transcripts_for_id.sh -id \1/' student_id_list > my_shell_script.sh
Right now, I'm just trying to get the list of id numbers (the student_id_list from above) from the list of potential expansions.
I think I know the basic problem, I just don't know how to fix it. I can get the pieces of the list I want as follows:
$ echo 0000{11,24} | tr ' ' '\n'
000011
000024
$ echo 0001{32,38,81} | tr ' ' '\n'
000132
000138
000181
$ echo 0002{02,80,81,89,97} | tr ' ' '\n'
000202
000280
000281
000289
000297
However, my first thought of a command didn't expand things
# doesn't work, as you can see
$ while read -r line; do echo $line; done < student_id_potential_expansions
0000{11,24}
0001{32,38,81}
0002{02,80,81,89,97}
I DO NOT want to just copy/paste every line. I've been given hundreds to thousands of the potentially-expandable lines - and these were parsed by someone else from who-knows-where, and I don't have access to the person who did the original parsing from records. Also, each of my lines is more like
0000009{882,739,861,014,952,611,862,935,976,916,080,697,323,843,840,487,517,407,256,756,374,682,162,930,758,157,770,505,867,233,198,131,917,848,613,247,961,261,616,392,876,747,873,148,844,849,280,626,817,819,174,771,172,284,217,200,018,624,418,292,642,529,755,855,647,317,881,962,975,237,635,805,298,835,053}
My Understanding
From the bash manual, section 3.5.1
Brace expansion is a mechanism by which arbitrary strings may be generated.
...
A correctly-formed brace expansion must contain unquoted opening and closing braces, and at least one unquoted comma or a valid sequence expression. Any incorrectly formed brace expansion is left unchanged.
As I understand it, when I read each line, I'm basically giving a string to be fed to whatever is inside the while loop, i.e. with my example above, I first give $line as "0000{11,24}" (note the quotes). Carrying this on further, It's as if my first trip through the loop results in the following command
echo "0000{11,24}"
Once again, notice the quotes, even though they were not around $line in my while loop. To see if this seemed consistent, I ran the following commands and got the following results:
$ echo "0000{11,24}"
0000{11,24}
$ echo "0001{32,38,81}"
0001{32,38,81}
$ echo "0002{02,80,81,89,97}"
0002{02,80,81,89,97}
So, as far as I can tell, all of the braces ({ and }) and commas are quoted - something that the bash manual says makes for an invalid expansion statement. The main question is, then, how to I un-quotify or un-stringify each $line that I read in?
My Attempts/Research
At the risk of making this post too long, here are some attempts I've made.
Looking through this SO post, I'm pretty sure my issue is not a word separation issue.
This answer and this comment, specifically the mention about
In
bash,{}expansion happens before$expansion, so I don't think there's any other way to do this other than usingevalor some other trick to cause two passes through the expression.
made me think of trying
$ while read -r line; do eval "$line"; done < student_id_potential_expansions
bash: 000011: command not found
bash: 000132: command not found
bash: 000202: command not found
and
$ while read -r line; do echo $(eval "$line"); done < student_id_potential_expansions
bash: 000011: command not found
bash: 000132: command not found
bash: 000202: command not found
where it seems only the first expansion gets carried out before the shell tries to run (evaluate) the command.
I also tried the commands that follow, looking at the comment here.
$ while read -r line; do expansions=$line; echo $expansions; done < trying_inner_echo.txt
$(echo 0000{11,24})
$(echo 0001{32,38,81})
$(echo 0002{02,80,81,89,97})
$ while read -r line; do echo $line; done < trying_inner_echo.txt
$(echo 0000{11,24})
$(echo 0001{32,38,81})
$(echo 0002{02,80,81,89,97})
$ while read -r line; do threads=${line}; exec $threads; done < trying_inner_echo.efl
bash: exec: $(echo: not found
bash: exec: $(echo: not found
bash: exec: $(echo: not found
$ while read -r line; do threads=${line}; exec "${threads}"; done < trying_inner_echo.txt
bash: exec: $(echo 0000{11,24}): not found
bash: exec: $(echo 0001{32,38,81}): not found
bash: exec: $(echo 0002{02,80,81,89,97}): not found
Some other attempts:
$ while read -r line; do echo "$line"; done < student_id_potential_expansions
$ while read -r line; do echo "$(echo $line)"; done < student_id_potential_expansions
System Info
$ uname -a
CYGWIN_NT-10.0 C-D-ENG-E-INT3 2.11.2(0.329/5/3) 2018-11-08 14:34 x86_64 Cygwin
$ bash --version
GNU bash, version 4.4.12(3)-release (x86_64-unknown-cygwin)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ systeminfo | sed -n 's/^OS\ *//p'
Name: Microsoft Windows 10 Enterprise
Version: 10.0.17134 N/A Build 17134
Manufacturer: Microsoft Corporation
Configuration: Member Workstation
Build Type: Multiprocessor Free
What I Used (thanks @EchoMike444)
The answer by @EchoMike444 is absolutely right and got me where I needed to go, hence it's the accepted answer. Since I used something slightly different than what appears in the answer, I'm going to put it here (rather than in a comment to the answer). From the comment:
Thanks for getting me there. When I used
while read -r line; do
eval "echo $line" ## RIGHT!
done < student_id_potential_expansions | tr ' ' '\n'
I got exactly the output I was looking for. I needed the help to get there from echo $(eval "$line") # wrong!.
