I am appending data to a file and then use
echo "$(tail -n10 newdata.txt)" > newdata.txt;
to keep only 10 lines in the newdata.txt file.
I want to read this file and store all of its content in a variable so that when I perform
{"Value":" "} | jq .
to create a JSON file. I will be able to write all 10 values to only one keyword value.
I tried
value=cat newdata.txt
but that is giving me parsing error:
Invalid string: control characters from U+0000 through U+001F must be escaped.
This is an XY Problem. To assign the content of file newdata.txt to variable lines, use:
lines="$(cat newdata.txt)"
But why would you want to assign the content of newdata.txt to a variable?
Your desired outcome is a JSON object with a field value that has the content of newdata.txt as its value. The content of newdata.txt is plain text, not JSON, so echoing {"value", $(cat newdata.txt)} or equivalently {"value", $lines} into jq . will not work, because jq expects to read valid JSON.
The actual question then is: how do I turn lines of plain text into JSON?
jq can parse lines of arbitrary input into a stream of JSON strings using option -R, --raw-input. To parse all input lines at once, use option -s, --slurp. The input is then available for regular jq stream processing, so:
jq -sR '{"value", .}' newdata.txt
will give the desired JSON object.
On a separate note, echo "$(tail -n10 newdata.txt)" > newdata.txt is bound to fail intermittently. The >newdata.txt has a fair chance of truncating newdata.txt to 0 bytes before tail gets a chance to read its last 10 lines. You could use sponge to solve this, but see below.
Also, why the echo? tail -n 10 already echoes to stdout, so there is no need to catch its output in a string, just to have echo echo it to stdout again.
Combining all this, a one-line solution would be:
tail -n 10 olddata.txt | tee newdata.txt | jq -sR '{"value", .}' &&
mv newdata.txt olddata.txt
bash has all the facilities of other programming languages, like variables and strings, but it is more productive to think of it as a stream processor that reads from stdin and writes to stdout.
Try:
value=$(perl -e 'print qx/cat newdata.txt/;')
This uses the "-e" lets you define Perl code to be executed by the compiler.
The $() makes it so you can capture the output of a command and assign it to a shell variable.
I figured it out. I am posting answer for whoever needs it.
I am appending data to a file called newdata.txt like this: echo "$(tail -n10 newdata.txt)" > newdata.txt;
Then I simply copied file to a variable called value: value="$(cat newdata.txt)"
After doing echo "{\"Values\":[$value]}" | jq . I am getting the JSON object.
The only way this works is there is a comma at the end of each line except for last line.
I am not sure if this is foolproof and right way to do this. But it worked for my problem.
jquse-case a little more? Provide examples of your expected input and expected output, if you would. As it reads now, it seems to sound like a fairly simple case of command substitution. Yourvalue=cat newdata.txtshould just bevalue=$(cat newdata.txt)(one way to do it). Or potentially even justvalue=$(tail -n10 newdata.txt). Note -- Since there are linebreaks in the variable, remember to quote it if echo'ing -- E.g.echo "$value".foo=$(php getdata.php), the value of foo is now 11 and then echoing it to the JSON file like ```echo "{\"bar\":\"$foo\"}" | jq . > sample.json This gives me JSON file like: { "bar": "11" } But I want to write multiple values. Like instead of one value 11, now when I do foo=$(php getdata.php) is 11,12,13,4,12 Then I want to create a JSON like { "bar": "11,12,13,4,12" }