-1

I am trying to do url testing by fetching the values from env.properties file.

For example, my env.properties will look like this

a.host.name=wanx.com
b.host.name=xyu.com
c.host.name=${b.host.name}
d.host.name=${c.host.name}

url1=https://${d.host.name}/test
url2=https://${a.host.name}/test2

So far what I have done - 1. since there is a "." in the file I could not do the direct substitution. So I have replaced the dots with underscore using awk

awk -F= -vOFS="=" 'gsub(/\./,"_",$1)+1' endpoint_test.txt

Now my file looks like below -

a_host_name=wanx.com
b_host_name=xyu.com
c_host_name=${b.host.name}
d_host_name=${c.host.name}

url1=https://${d.host.name}/test
url2=https://${a.host.name}/test2
  1. I am trying to replace the values present inside ${b.host.name} and ${c.host.name} , I have tried most of the awk commands found from google. Below are the commands which I have tried

    awk -F= -vOFS="${#*}" 'gsub(/\./,"_",$1)+1' endpoint_test2.txt

    awk -F\" '{OFS="\""; for (i = 2; i < NF; i += 2) gsub(/[$,]/,"",$i); gsub(/"/,""); print}' endpoint_test2.txt

But it doesn't work. I want to change the dots to underscore present inside ${value}. So if I move that contents to a shell it's very easy for me to substitute.

Edit #1 -

Ultimately i need a output file like below -

a_host_name=wanx.com
b_host_name=xyu.com
c_host_name=${b.host.name}
d_host_name=${c.host.name}

url1=https://${d_host_name}/test
url2=https://${a_host_name}/test2

So if I convert this file to shell i will be doing this

a_host_name=wanx.com
b_host_name=xyu.com
c_host_name=${b_host_name}
d_host_name=${c_host_name}

url1=https://${d_host_name}/test
url2=https://${a_host_name}/test2

echo $url1
echo $url2

Output of this file -

https://xyu.com/test
https://wanx.com/test2

Edit #2 - I tried direct bash, But its saying as bad substitution since i need to replace the value inside ${}.

Edit #3 -

The OS i am trying is in AIX, And the file may not contain the same variable like "a.host.name" as a variable. It may also contains variable like "a.name.host" for example the file may looks like this -

a.b.host=qwel.wanx.net
b.host.name=ioy.xyu.net
c.xcv.host=poiu.deolite.net
d.host.name=${b.host.name}
e.host.name=${c.host.name}

abcv.stub.url=https://${d.host.name}/test
xcm.stub.url=https://${a.b.host}/test2

With this command i am able to achieve my edge case which i have mentioned in edit #3 -

perl -pe 's/^[\w.]+(?==)|\$\{[\w.]+\}/ $_ = $&; tr|.|_|; $_ /ge' file

Now my output looks like this -

a_b_host=qwel.wanx.net
b_host_name=ioy.xyu.net
c_xcv_host=poiu.deolite.net
d_host_name=${b_host_name}
e_host_name=${c_host_name}

abcv_stub_url=https://${d_host_name}/test
xcm_stub_url=https://${a_b_host}/test2

Finally i need the url alone in a separate file, Like below

https://ioy.xyu.net/test
https://qwel.wanx.net/test2
  • So you basically want to replace . by _ in the variable names but not in the host names? – nohillside Aug 3 '18 at 13:38
  • Yes @nohillside . That is what i want . – Harish P C Aug 3 '18 at 13:39
  • @ilkkachu - it works only when the variable is always same - for eg - "host.name". It wont work if the variable is 'a.name.host' or some different_variable = value – Harish P C Aug 5 '18 at 6:57
  • @ilkkachu - But the variable always ends with '.host' - Your solution is close. I tried this sed - sed -e 's/\.*\.host/__host/g' filename - its just replacing the variable with ''. I know its wrong . Now can you try modifying your answer and give ? – Harish P C Aug 5 '18 at 7:09
  • @HarishPC, actually, I rather like the Perl solution below. I wouldn't want to try the general solution in sed ... – ilkkachu Aug 5 '18 at 7:25
2

You could do something like:

perl -pe 's/^[\w.]+(?==)|\$\{[\w.]+\}/$& =~ y|.|_|r/ge' < file

That is replace . with _ on sequences of word characters or . that are at the beginning of the line and followed by = or inside ${...}.

The r flag to the y/// operator (so the result of the substitution is returned instead of being applied to the variable) needs perl 5.14 or above. With older versions, you can always do:

perl -pe 's/^[\w.]+(?==)|\$\{[\w.]+\}/$_ = $&; y|.|_|; $_/ge' < file

Now, for your ultimate task, it's just as easy to do the whole thing in perl here rather than interpret the code in a shell which would be quite dangerous:

 perl -lne '
    s/\$\{([\w.]+)\}/$v{$1}/g;
    if (/^([\w.]+)=(.*)/) {
      $v{$1} = $v = $2;
      print $v if $1 =~ /_url$/
    }' < file > separate-file
  • 1
    I am getting this error " Bareword found where operator expected at -e line 1, near "y|.|_|r" syntax error at -e line 1, near "y|.|_|r " Execution of -e aborted due to compilation errors. - Did i miss anything ? – Harish P C Aug 5 '18 at 6:38
  • @HarishPC, old version of Perl? That indeed doesn't work with Perl 5.10.1. It seems to be the /r in tr. Wikipedia says that was added in 5.14, in 2011... perl -pe 's/^[\w.]+(?==)|\$\{[\w.]+\}/ $_ = $&; tr|.|_|; $_ /ge' seems to work as a workaround. – ilkkachu Aug 5 '18 at 7:21
  • Its worked now. Thanks a lot for providing this answer. Can you please help me with the edit 3 in my question. I will mark your answer. Thanks again – Harish P C Aug 5 '18 at 8:46
  • @HarishPC, see edit. – Stéphane Chazelas Aug 5 '18 at 19:40
  • In what sense is perl limited so it is requiring a < file instead of a simpler file ? – Isaac Aug 6 '18 at 4:15
0

If what you call "output of this file" is your final desired result, try

    awk -F= '
                    {gsub (/[${}]/, "")
                    }
    !/^url/         {T[$1]=($2 in T)?T[$2]:$2
                     next
                    }
                    {sub (/^url[^=]*=/, "")
                     for (t in T)   if ($0 ~ t) sub (t, T[t])
                    }
    1
    ' file
https://xyu.com/test
https://wanx.com/test2
0

Try this,

sed -e 's/\./_/g' -e '/com/ s/_/\./3g' test789
a_host_name=wanx.com
b_host_name=xyu.com
c_host_name=${b_host_name}
d_host_name=${c_host_name}

url1=https://${d_host_name}/test
url2=https://${a_host_name}/test2
  • first sed will replace all . to _
  • second sed will replace 3rd_ to . if the line has com
  • @HarishPC try my updated answer. – msp9011 Aug 3 '18 at 15:24
  • Thanks for the updated answer - This Sed ignoring first two lines variables. I am getting out like this a.host.name=qwel.wanx.net b.host.name=ioy.xyu.net c.host.name=poiu.deolite.net d_host_name=${b_host_name} e_host_name=${c_host_name} abcv_stub_url=https://${d_host_name}/test xcm_stub_url=https://${a_host_name}/test2 – Harish P C Aug 5 '18 at 6:22
  • I tried this sed as well - sed -e 's/\./_/g' -e '/#net/ s/_/\./3g' endpoint_test3.txt , But it replacing both Value and variable. I am getting out like this - a_host_name=qwel_wanx_net b_host_name=ioy_xyu_net c_host_name=poiu_deolite_net d_host_name=${b_host_name} e_host_name=${c_host_name} abcv_stub_url=https://${d_host_name}/test xcm_stub_url=https://${a_host_name}/test2 – Harish P C Aug 5 '18 at 6:26
0

You can try like this :

cat get_env_properties.sh 
sed -i'.bak' '
  :A
    s/\(.*\)\(\.\)\([^=]*=.*\)/\1_\3/
    tA
  :B
    s/\([^{]*{.*\)\(\.\)\([^}]*}.*\)/\1_\3/
    tB
' "$1"

. "$1"

echo "url1 = $url1"
echo "url2 = $url2"

mv "$1.bak" "$1"

You call it this way

./get_env_properties.sh env.properties

Your sed must be ok with -i

  • -E is not supported . Do you have any alternate solution ? – Harish P C Aug 5 '18 at 6:33
  • See my update answer, I remove the -E option not supported. – ctac_ Aug 5 '18 at 8:24
0

An awk solution is:

awk -F= -vOFS='=' '{                         gsub(/\./,"_",$1);
                        if($2~/\$\{[^}]*\}/){gsub(/\./,"_",$2)}
                   } 1' endpoint_test.txt

It will replace all dots with _ on first field and the same on field $2 that have a ${…}.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.