0

I'm going to give a short version of the kind of issue I'm facing. The real application I'm trying to write has many parts, so I've tried to isolate the part where I'm stuck with a small example. My situation, which involves file ID numbers, is similar to a hypothetical list of Student ID Numbers parsed from a directory or something.


What I Start With

In order to try and take advantage of bash brace expansion, something like a student directory has been parsed to give the following:

$ cat student_id_potential_expansions
0000{11,24}
0001{32,38,81}
0002{02,80,81,89,97}

Desired behavior

I want to use this input to create my own (Bourne Again) shell script which looks something like the following.

#!/bin/bash

get_transcripts_for_id.sh -id 000011
get_transcripts_for_id.sh -id 000024
get_transcripts_for_id.sh -id 000132
get_transcripts_for_id.sh -id 000133
get_transcripts_for_id.sh -id 000181
get_transcripts_for_id.sh -id 000202
...

and so on

I'm not too worried about the command before; I figured I'd just use an

awk '{print "get_transcripts_for_id.sh -id" $0}' student_id_list > my_shell_script.sh

or a

sed 's/^\(.*\)$/get_transcripts_for_id.sh -id \1/' student_id_list > my_shell_script.sh

Right now, I'm just trying to get the list of id numbers (the student_id_list from above) from the list of potential expansions.

I think I know the basic problem, I just don't know how to fix it. I can get the pieces of the list I want as follows:

$ echo 0000{11,24} | tr ' ' '\n'
000011
000024

$ echo 0001{32,38,81} | tr ' ' '\n'
000132
000138
000181

$ echo 0002{02,80,81,89,97} | tr ' ' '\n'
000202
000280
000281
000289
000297

However, my first thought of a command didn't expand things

# doesn't work, as you can see
$ while read -r line; do echo $line; done < student_id_potential_expansions
0000{11,24}
0001{32,38,81}
0002{02,80,81,89,97}

I DO NOT want to just copy/paste every line. I've been given hundreds to thousands of the potentially-expandable lines - and these were parsed by someone else from who-knows-where, and I don't have access to the person who did the original parsing from records. Also, each of my lines is more like

0000009{882,739,861,014,952,611,862,935,976,916,080,697,323,843,840,487,517,407,256,756,374,682,162,930,758,157,770,505,867,233,198,131,917,848,613,247,961,261,616,392,876,747,873,148,844,849,280,626,817,819,174,771,172,284,217,200,018,624,418,292,642,529,755,855,647,317,881,962,975,237,635,805,298,835,053}

My Understanding

From the bash manual, section 3.5.1

Brace expansion is a mechanism by which arbitrary strings may be generated.
...
A correctly-formed brace expansion must contain unquoted opening and closing braces, and at least one unquoted comma or a valid sequence expression. Any incorrectly formed brace expansion is left unchanged.

As I understand it, when I read each line, I'm basically giving a string to be fed to whatever is inside the while loop, i.e. with my example above, I first give $line as "0000{11,24}" (note the quotes). Carrying this on further, It's as if my first trip through the loop results in the following command

echo "0000{11,24}"

Once again, notice the quotes, even though they were not around $line in my while loop. To see if this seemed consistent, I ran the following commands and got the following results:

$ echo "0000{11,24}"
0000{11,24}

$ echo "0001{32,38,81}"
0001{32,38,81}

$ echo "0002{02,80,81,89,97}"
0002{02,80,81,89,97}

So, as far as I can tell, all of the braces ({ and }) and commas are quoted - something that the bash manual says makes for an invalid expansion statement. The main question is, then, how to I un-quotify or un-stringify each $line that I read in?


My Attempts/Research

At the risk of making this post too long, here are some attempts I've made.

Looking through this SO post, I'm pretty sure my issue is not a word separation issue.

This answer and this comment, specifically the mention about

In bash, {} expansion happens before $ expansion, so I don't think there's any other way to do this other than using eval or some other trick to cause two passes through the expression.

made me think of trying

$ while read -r line; do eval "$line"; done < student_id_potential_expansions
bash: 000011: command not found
bash: 000132: command not found
bash: 000202: command not found

and

$ while read -r line; do echo $(eval "$line"); done < student_id_potential_expansions
bash: 000011: command not found

bash: 000132: command not found

bash: 000202: command not found

where it seems only the first expansion gets carried out before the shell tries to run (evaluate) the command.

I also tried the commands that follow, looking at the comment here.

$ while read -r line; do expansions=$line; echo $expansions; done < trying_inner_echo.txt
$(echo 0000{11,24})
$(echo 0001{32,38,81})
$(echo 0002{02,80,81,89,97})

$ while read -r line; do echo $line; done < trying_inner_echo.txt
$(echo 0000{11,24})
$(echo 0001{32,38,81})
$(echo 0002{02,80,81,89,97})

$ while read -r line; do threads=${line}; exec $threads; done < trying_inner_echo.efl
bash: exec: $(echo: not found
bash: exec: $(echo: not found
bash: exec: $(echo: not found

$ while read -r line; do threads=${line}; exec "${threads}"; done < trying_inner_echo.txt
bash: exec: $(echo 0000{11,24}): not found
bash: exec: $(echo 0001{32,38,81}): not found
bash: exec: $(echo 0002{02,80,81,89,97}): not found

Some other attempts:

$ while read -r line; do echo "$line"; done < student_id_potential_expansions
$ while read -r line; do echo "$(echo $line)"; done < student_id_potential_expansions

System Info

$ uname -a
CYGWIN_NT-10.0 C-D-ENG-E-INT3 2.11.2(0.329/5/3) 2018-11-08 14:34 x86_64 Cygwin

$ bash --version
GNU bash, version 4.4.12(3)-release (x86_64-unknown-cygwin)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

$ systeminfo | sed -n 's/^OS\ *//p'
Name:                   Microsoft Windows 10 Enterprise
Version:                10.0.17134 N/A Build 17134
Manufacturer:           Microsoft Corporation
Configuration:          Member Workstation
Build Type:             Multiprocessor Free

What I Used (thanks @EchoMike444)

The answer by @EchoMike444 is absolutely right and got me where I needed to go, hence it's the accepted answer. Since I used something slightly different than what appears in the answer, I'm going to put it here (rather than in a comment to the answer). From the comment:

Thanks for getting me there. When I used

while read -r line; do 
  eval "echo $line"  ## RIGHT!
done < student_id_potential_expansions | tr ' ' '\n' 

I got exactly the output I was looking for. I needed the help to get there from echo $(eval "$line") # wrong!.

1 Answer 1

1

The trick is to use eval

A very small example :

( echo '000{90,91}' ; echo '002{10,11}' ; echo '110{50,51}' ) | \
       while read L ; do eval "echo $L" | tr ' ' '\n'  ; done
1
  • Thank you. It seems that what I needed to do was toswitch the eval and echo in my attempt with echo $(eval "$line"). Thanks for getting me there. When I used while read -r line; do eval "echo $line"; done < student_id_potential_expansions | tr ' ' '\n', I got exactly the output I was looking for. I also really like the way you did the example, with the parentheses around the echo statements. That allows a much shorter way of showing what I showed with a big file. Commented Nov 8, 2019 at 3:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .