1

I have a folder with multiple .json files. There are certain files with empty arrays, Sample file :

{
"WarehouseActivity": []
}

The file has no other data apart from the one shown above.

I need to identify these files and move them into a error folder. Any suggestions on how to go about this would be great.

Thanks, Kavin

  • 1
    is key WarehouseActivity common/same for all the files? – RomanPerekhrest May 4 '18 at 12:14
  • Did you try with jq? – dirkt May 4 '18 at 12:16
  • { "x": [[],[]] } would have an array made of two empty arrays. Should that also be flagged? – Stéphane Chazelas May 4 '18 at 12:50
  • @RomanPerekhrest yes its same across all the files and no, i haven't tried the jq option...am gonna try the code posted by you and revert back to you. – Kavin Palaniswamy May 4 '18 at 15:41
  • @StéphaneChazelas well, there hasn't been any such case yet. Just one empty array. – Kavin Palaniswamy May 4 '18 at 15:42
2

Jq would be the right tool for processing/analyzing JSON data:

for f in *.json; do
    if jq -e 'keys_unsorted as $keys
              | ($keys | length == 1) 
                and .[($keys[0])] == []' "$f" > /dev/null; then
        mv "$f" error_dir/
    fi
done
  • this works great! It identifies the empty files and moves them to the error folder. To start with i wasn't even aware of the jq command, i had to install the jq package in my machine to get this to work. Thanks a lot. Really appreciate the help. – Kavin Palaniswamy May 4 '18 at 16:22
  • @KavinPalaniswamy, you're welcome – RomanPerekhrest May 4 '18 at 17:16
  • Hey! There is a scenario when there is no such file , am getting a error message "jq: error: Could not open file *.json: No such file or directory" . For now am just suppressing it. Any way around it ? – Kavin Palaniswamy May 10 '18 at 2:31
  • @KavinPalaniswamy, it seems that you are not applying the above approach in exact manner. for f in *.json; will iterate through existed .json files (unless you don't have folders which end with .json) – RomanPerekhrest May 10 '18 at 5:13
  • Sorry for the delayed response. Well i think i didn't explain myself well. When the folder is completely empty and there are no json files i get the error :jq: error: Could not open file Test_*.json: No such file or directory . I have implemented the exact same code given by you. As a work around i am just check if any file exist and then am running your code. This does't throw any error. Any better way to implement this sir? – Kavin Palaniswamy May 18 '18 at 20:50
2
mkdir -p error_folder &&
for json in ./*.json; do
    if jq -e '..|select(type == "array" and length == 0)' <"$json" >/dev/null
    then
        mv "$json" error_folder/
    fi
done

This is more or less the same approach as Roman took in his answer, but uses a different jq expression.

The expression ..|select(type == "array" and length == 0) will recurse the full JSON structure and select all bits of it that are zero-length arrays (anywhere, at any depth).

If the select() is successful, then jq will exit with a zero exit status (success), which means that the JSON document contains an empty array somewhere (or the file is totally empty). This triggers a moving of the document to error_folder in the script.


From comments below it is clear that the user is only interested in the WarehouseActivity array.

My code with a modified jq expression:

mkdir -p error_folder &&
for json in ./*.json; do
    if jq -e '..|select(type == "object" and has("WarehouseActivity") and (.WarehouseActivity|length) == 0)' <"$json" >/dev/null
    then
        mv "$json" error_folder/
    fi
done
  • 1
    Note that jq (at least my version) also returns true here for empty files and files made entirely of blanks (which the OP may also want to consider as errors anyway). – Stéphane Chazelas May 4 '18 at 14:52
  • Hi @Kusalananda , i gave this code a try but it is not working as expected. It ends up moving all the json files in that folder including the empty json file. – Kavin Palaniswamy May 4 '18 at 16:20
  • @KavinPalaniswamy Then all files contain at least one empty array somewhere, or you have an error in your script. I've tested the code in my answer. – Kusalananda May 4 '18 at 16:24
  • @Kusalananda Yes, you are correct the json files do have empty arrays. I was looking to identify the json files which have only the below : { "WarehouseActivity": [] } Don't really want to identify all the empty arrays in the json files. But great solution. I will use this when i need to identify all the empty arrays . Thanks a lot. really appreciate your help. – Kavin Palaniswamy May 4 '18 at 16:30
  • @KavinPalaniswamy Updated my answer. – Kusalananda May 4 '18 at 16:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.