2

Let's say I have a file with the following content:

var1='a random text'
var2='another random text'
var3='a third random text'

I know that if I use the command eval like the following I'll store all those variables directly on my shell:

$ eval $(cat file)

Doing that, my shell will create $var1, $var2 and $var3 with their respective contents. Knowing that, I could generate a JSON manually like the following:

$ JSON="{ \"var1\" : \"$var1\", \"var2\" : \"$var2\", \"var3\" : \"$var3\"}"

And that would result in a valid JSON:

$ echo $JSON
{ "var1" : "a random text", "var2" : "another random text", "var3" : "a third random text"}

The problem here is that I'm hardcoding the keys var1, var2 and var3... In my case, the file could be bigger and with more variables stored in it (not just var1, var2 and var3). I was thinking if there is an easy way of achieving that using the command line, just like eval does for storing file variables on the shell, but instead of storing the variables, generating a JSON output. Is it possible? Can I directly convert a file structured like that to JSON using the command line?


My alternative solution here would be developing a code (not using shell) that goes char by char inside this file and then I separate everything dynamically in a loop. But I'm making this question because I want to avoid overcomplicating the solution.

2
  • 2
    Should rather be . ./file or eval "$(cat file)". eval $(cat file) is wrong as leaving that $(...) unquoted invokes split+glob. Jun 11, 2021 at 6:56
  • Which shell is it? Would be easier if you had an array. With just some random variable names it will be difficult. You would need something like this to find out the variable names that your file set.
    – pLumo
    Jun 11, 2021 at 7:23

2 Answers 2

3

Using a combination of jo (from here) and jq (from here), without creating shell variables or letting the shell interpret the file at all:

jo <file |
jq --arg sq "'" '.[] |= ( ltrimstr($sq) | rtrimstr($sq) )'

This first uses jo to create the JSON document

{
   "var1": "'a random text'",
   "var2": "'another random text'",
   "var3": "'a third random text'"
}

(but on a single line). It does this by interpreting the variable assignments in your file as key-value pairs.

The jq tool is then used to delete the single quotes from the start and end of each value.

The final result is

{
  "var1": "a random text",
  "var2": "another random text",
  "var3": "a third random text"
}

This will not cope with newlines being embedded in the values. Other special characters will however be automatically JSON-encoded by jo.

4
  • Is jo this: github.com/jpmens/jo? Jun 11, 2021 at 13:24
  • 1
    @glennjackman should be (eg: manpages.debian.org/jo/jo.1#AVAILABILITY )
    – A.B
    Jun 11, 2021 at 15:25
  • @glennjackman That's the one. Updated answer with proper links. Thanks!
    – Kusalananda
    Jun 11, 2021 at 16:27
  • It should be noted that it assumes the input lines are in var='value' format only (with and only with single quotes around the value), which if we assume the file is also valid shell syntax precludes values containing single quotes. Jun 11, 2021 at 19:08
1

With zsh, and assuming that file is in a syntax compatible with that of zsh, that the file doesn't make use of shell expansion features (like var1=~/foo, var2=$var1, var3=$(uname)...) and that the values are text encoded in UTF-8, you could do:

tokens=( ${(Q)${(zZ[nC])"$(<file)"}} )

To tokenise the contents of that file as per the shell syntax (using the z parameter expansion flag, tweaked (with Z[flags]) with n to consider unquoted newlines as whitespace and C to strip shell comments) and remove one layer of quoting with the Q parameter expansion flag.

Then you can pass those tokens to something that can encode json (take care of control characters including newline, backslash, " characters, etc):

perl -CA -MJSON -le '
  for (@ARGV) {
    if (/(.*?)=(.*)/s) {
      $h{$1} = $2;
    }
  }
  print encode_json \%h' -- $tokens

For instance, one a file such as:

var1='a random text' # comment
var2='another'\'' random text'
var3='a third random text'

name=$'St\u00e9phane Chazelas'
empty=
at=@
more=broken\
down"with 1 \\ backslash"
numstring=1.1

It gives:

{"numstring":"1.1","name":"Stéphane Chazelas","empty":"","more":"brokendownwith 1 \\ backslash","var1":"a random text","at":"@","var2":"another' random text","var3":"a third random text"}

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .