0

I am trying to create a while loop so that it takes content from one file and creates some content on another file. But what i noticed is that it is only creating the last line of the file instead of all the lines in the file. What am i missing here? Or is my approach with echo wrong?

My file called "test" contains a list of strings for ex.

unix_idx
web_pn_iis
wis_healthpartners

I am using the following command to try and create a while loop.

while read -r line;
do
echo "  {
    \"name\": \"$line\",  
    \"datatype\": \"event\", 
    \"searchableDays\": 180, 
    \"maxDataSizeMB\": 0, 
    \"totalEventCount\": \"0\", 
    \"totalRawSizeMB\": \"0\" 
    }," > myfile.json;
done < test;

but once the command is run myfile.json only contains the last line read from the test file. i.e wis_healthpartners

  {
"name": "wis_healthpartners",  
"datatype": "event", 
"searchableDays": 180, 
"maxDataSizeMB": 0, 
"totalEventCount": "0", 
"totalRawSizeMB": "0" 
},

So i think the echo is over writing the lines as the while loop runs and it is left with only the last line. How do i tweak this so that it contains all the lines together? My desired output is as below.

{
  "name": "unix_idx",
  "datatype": "event",
  "searchableDays": 180,
  "maxDataSizeMB": 0,
  "totalEventCount": "0",
  "totalRawSizeMB": "0"
},
{
  "name": "web_pn_iis",
  "datatype": "event",
  "searchableDays": 180,
  "maxDataSizeMB": 0,
  "totalEventCount": "0",
  "totalRawSizeMB": "0"
},
{
  "name": "wis_healthpartners",
  "datatype": "event",
  "searchableDays": 180,
  "maxDataSizeMB": 0,
  "totalEventCount": "0",
  "totalRawSizeMB": "0"

},

1

2 Answers 2

2

In Bash, the > operator intentionally overwrites any existing data in the file, while the >> operator will append.

If you need to make sure the file is empty before you start, you can use printf "" > myfile.json to clear it out before your loop runs, then use >> to continue writing to the end.

2
  • worked like a charm. thanks for that. i will keep that in mind going forward. May 9 at 15:18
  • 3
    Or you could simply place the > myfile.json outside the loop.
    – DonHolgo
    May 9 at 15:39
2

If the entries in the test file were properly quoted JSON strings

$ cat test
"unix_idx"
"web_pn_iis"
"wis_healthpartners"

then you could use jq's --slurpfile to pull them into an array, then iterate over that to create an array of objects from a template JSON file:

{
  "name": "",
  "datatype": "event",
  "searchableDays": 180,
  "maxDataSizeMB": 0,
  "totalEventCount": "0",
  "totalRawSizeMB": "0"
}

ex.

$ jq --slurpfile names test '
    . | [foreach $names[] as $name (.; .name |= $name)]' template.json
[
  {
    "name": "unix_idx",
    "datatype": "event",
    "searchableDays": 180,
    "maxDataSizeMB": 0,
    "totalEventCount": "0",
    "totalRawSizeMB": "0"
  },
  {
    "name": "web_pn_iis",
    "datatype": "event",
    "searchableDays": 180,
    "maxDataSizeMB": 0,
    "totalEventCount": "0",
    "totalRawSizeMB": "0"
  },
  {
    "name": "wis_healthpartners",
    "datatype": "event",
    "searchableDays": 180,
    "maxDataSizeMB": 0,
    "totalEventCount": "0",
    "totalRawSizeMB": "0"
  }
]

If you must work with unquoted elements, then you can do the same with --rawfile although it's a bit more work:

jq --rawfile names test '
    . | [foreach ($names | split("\n")[0:-1] | .[]) as $name (.; .name |= $name)]
' template.json

The slice [0:-1] is necessary because split("\n") interprets the test file's trailing newline as indicative of an additional empty element.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.