1

I am appending data to a file and then use

echo "$(tail -n10 newdata.txt)" > newdata.txt;

to keep only 10 lines in the newdata.txt file.

I want to read this file and store all of its content in a variable so that when I perform

{"Value":" "} | jq .

to create a JSON file. I will be able to write all 10 values to only one keyword value.

I tried

value=cat newdata.txt

but that is giving me parsing error:

Invalid string: control characters from U+0000 through U+001F must be escaped.
3
  • 1
    Can you elaborate on the jq use-case a little more? Provide examples of your expected input and expected output, if you would. As it reads now, it seems to sound like a fairly simple case of command substitution. Your value=cat newdata.txt should just be value=$(cat newdata.txt) (one way to do it). Or potentially even just value=$(tail -n10 newdata.txt). Note -- Since there are linebreaks in the variable, remember to quote it if echo'ing -- E.g. echo "$value". Jun 16, 2021 at 21:11
  • I am trying to write a JSON file. For this I am reading values from a board using shell-scripting like foo=$(php getdata.php), the value of foo is now 11 and then echoing it to the JSON file like ```echo "{\"bar\":\"$foo\"}" | jq . > sample.json This gives me JSON file like: { "bar": "11" } But I want to write multiple values. Like instead of one value 11, now when I do foo=$(php getdata.php) is 11,12,13,4,12 Then I want to create a JSON like { "bar": "11,12,13,4,12" }
    – Sneha
    Jun 17, 2021 at 13:35
  • Best to edit your question with that info so it can be more readable with line breaks and formatting. Thanks! Jun 17, 2021 at 15:03

3 Answers 3

2

This is an XY Problem. To assign the content of file newdata.txt to variable lines, use:

lines="$(cat newdata.txt)"

But why would you want to assign the content of newdata.txt to a variable?

Your desired outcome is a JSON object with a field value that has the content of newdata.txt as its value. The content of newdata.txt is plain text, not JSON, so echoing {"value", $(cat newdata.txt)} or equivalently {"value", $lines} into jq . will not work, because jq expects to read valid JSON.

The actual question then is: how do I turn lines of plain text into JSON?

jq can parse lines of arbitrary input into a stream of JSON strings using option -R, --raw-input. To parse all input lines at once, use option -s, --slurp. The input is then available for regular jq stream processing, so:

jq -sR '{"value", .}' newdata.txt

will give the desired JSON object.

On a separate note, echo "$(tail -n10 newdata.txt)" > newdata.txt is bound to fail intermittently. The >newdata.txt has a fair chance of truncating newdata.txt to 0 bytes before tail gets a chance to read its last 10 lines. You could use sponge to solve this, but see below.

Also, why the echo? tail -n 10 already echoes to stdout, so there is no need to catch its output in a string, just to have echo echo it to stdout again.

Combining all this, a one-line solution would be:

tail -n 10 olddata.txt | tee newdata.txt | jq -sR '{"value", .}' && 
mv newdata.txt olddata.txt

bash has all the facilities of other programming languages, like variables and strings, but it is more productive to think of it as a stream processor that reads from stdin and writes to stdout.

0

Try:

value=$(perl -e 'print qx/cat newdata.txt/;')

This uses the "-e" lets you define Perl code to be executed by the compiler.

The $() makes it so you can capture the output of a command and assign it to a shell variable.

0

I figured it out. I am posting answer for whoever needs it. I am appending data to a file called newdata.txt like this: echo "$(tail -n10 newdata.txt)" > newdata.txt;

Then I simply copied file to a variable called value: value="$(cat newdata.txt)"

After doing echo "{\"Values\":[$value]}" | jq . I am getting the JSON object.

The only way this works is there is a comma at the end of each line except for last line.

I am not sure if this is foolproof and right way to do this. But it worked for my problem.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .