1

I'm working on a bash script that reads a csv file with fields and field values and creates a json string out of the input.

For simplicity I have a bash script that takes in two arguments simulating above script.

#!/bin/bash

fieldValue1=$1
fieldValue2=$2

jsonString='{"field1":"'$fieldValue1'", "field2":"'$fieldValue2'"}'
echo $jsonString

When I call above script with two arguments, I correctly get below output:

 ./test.sh "abc" "def"
{"field1":"abc", "field2":"def"}

What I am trying to achieve now, is to be able to assign and print a variable in the first argument, then reuse the variable from the first argument in the second argument.

For example when I call the script with below sample arguments (the syntax is flexible):

 ./test.sh "VAR=abc;echo $VAR" "$VAR"

then the output should look like this:

{"field1":"abc", "field2":"abc"}

The practical use of this is that for example for the real script inside the large input file, I only have to maintain a date once:

startDate1,field2,startDate2
var startDate=1/1/2020;echo $startDate,someValue,$startDate
2/1/2020,someOtherValue,$startDate

Does anybody have an idea on how to best achieve above output?

closed as unclear what you're asking by Kusalananda, RalfFriedl, G-Man, Thomas, roaima Sep 7 '18 at 19:42

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • You realise that none of your variables in your script is double quoted? That's poor programming style. Try fieldValue1="$1" and jsonString="{\"field1\":\"$fieldValue1\", \"field2\":\"$fieldValue2\"}' or jsonString='{"field1":"'"$fieldValue1"'", "field2":"'"$fieldValue2"'"}' if you must, and finally echo "$jsonString". – roaima Sep 7 '18 at 19:40
2

Why not var=abc; ./test.sh "$var" "$var" ?

Or, call your script with one argument, and the script handles the missing value:

#!/bin/bash
fieldValue1=$1
fieldValue2=${2:-$1}
...

then: ./test.sh foo outputs {"field1"="foo", "field2"="foo"}


Note that's not JSON: use a colon not equal: {"field1":"foo", "field2":"foo"}

Also you're leaving the variables unquoted when you assign the jsonString variable. Do this

jsonString='{"field1":"'"$fieldValue1"'", "field2":"'"$fieldValue2"'"}'
# ......................^............^...............^............^
# or
jsonString="{'field1':'$fieldValue1', 'field2':'$fieldValue2'}"
# or
printf -v jsonString '{"field1":"%s","field2":"%s"}' "$fieldValue1" "$fieldValue2"

Also, you should protect against the values containing quote characters:

printf -v jsonString '{"field1":"%s","field2":"%s"}' \
    "${fieldValue1//\"/\\\"}" \
    "${fieldValue2//\"/\\\"}"

Then

$ ./test.sh 'he said "foo"'
{"field1":"he said \"foo\"","field2":"he said \"foo\""}

Based on updated requirements. Assigning the variable must be a separate step. You're also missing the $(...) command substitution syntax

First an updated script:

#!/bin/bash
pairs=()
for ((i=1; i<=$#; i++)); do
    value=${!i}
    pairs+=( "$(printf '"%s":"%s"' "field$i" "${value//\"/\\\"}")" )
done
IFS=,
echo "{${pairs[*]}}"

Then invoke it like this

startDate=1/1/2020
bash test.sh "$startDate" someOtherValue "$startDate" "$(date -u -d "$startDate 1 hour" "+%FT%TZ")" "anotherValue"

and get this output:

{"field1":"1/1/2020","field2":"someOtherValue","field3":"1/1/2020","field4":"2020-01-01T01:00:00Z","field5":"anotherValue"}
  • I fixed the json string by using : instead of = I also added an explanation or the "real" use case to this post, which is having an large input file with values and declaring and using a variable only once in the input file instead of having to maintain a value in many places inside the input file... – Bernie Lenz Sep 6 '18 at 17:15
0

It seems as if you are trying to reinvent csvjson from CSVkit:

$ cat file.csv
A,B,C
1,2,3
4,5,6

$ csvjson file.csv
[{"A": 1.0, "B": 2.0, "C": 3.0}, {"A": 4.0, "B": 5.0, "C": 6.0}]

$ csvjson -i 4 file.csv
[
    {
        "A": 1.0,
        "B": 2.0,
        "C": 3.0
    },
    {
        "A": 4.0,
        "B": 5.0,
        "C": 6.0
    }
]
  • Similar but with variables: e.g. A,B,C\nVAR=1/1/20;echo $VAR,$VAR,3 – Bernie Lenz Sep 6 '18 at 17:18
  • @BernieLenz I'm not sure I understand. Could you give an example CSV input file that you are working on and what the JSON should look like after having processed it? – Kusalananda Sep 6 '18 at 17:52
  • Updated the post with an example – Bernie Lenz Sep 6 '18 at 18:06
  • 1
    @BernieLenz You made it very unclear. You have shell code in your CSV file? – Kusalananda Sep 6 '18 at 18:12
  • Correct. That's the reason I have my own shell script instead of using any tools. – Bernie Lenz Sep 6 '18 at 18:25
0

If you enclose your first argument within $(), you'll get the output you want.

./test.sh "$(VAR=abc; echo $VAR)" "def"

In this case, bash will use a subshell to execute setting VAR=abc, followed by echo $VAR, and then STDOUT from that subshell is being used as test.sh argument $1.


EDIT

To get both variables assigned from a single setting of the variable, you'll need to get both from the subshell.

./test.sh $(VAR='abc'; echo $VAR $VAR)

By removing the quotation marks around $(), we know catch the first word in the STDOUT of the subshell as $1, and the second word as $2.

However, since $VAR isn't being set in the parent shell, you can't use $VAR directly.

You could however, use the subshell to set $VAR.

VAR=$(VAR=abc; echo $VAR) ./test.sh $VAR $VAR

or, for readability

VAR=$(FOO=abc; echo $FOO) ./test.sh $VAR $VAR
  • 1
    Does not answer the question: he wants to use the variable for the 2nd parameter as well. And since you're assigning it in a subshell, the variable is not available there. – glenn jackman Sep 6 '18 at 17:05
  • This seems to be on the right track, but when I try to reference VAR in the second argument it turns out to be empty: ./variableAssignmentTest.sh "$(VAR=abc; echo $VAR)" "$VAR" – Bernie Lenz Sep 6 '18 at 17:10
  • 1
    Yeah. That's because VAR isn't being set in the current shell. It's being set in the sub shell, and then we're catching it as STDOUT. – Tim Kennedy Sep 6 '18 at 17:55
  • If you assign $VAR in the parent shell, then you could use it to pass arguments to test.sh. – Tim Kennedy Sep 6 '18 at 18:01
  • Is there a way to export the variable from the sub to the parent? – Bernie Lenz Sep 6 '18 at 18:07

Not the answer you're looking for? Browse other questions tagged or ask your own question.