9

I have a folder with multiple .json files. There are certain files with empty arrays, Sample file :

{
"WarehouseActivity": []
}

The file has no other data apart from the one shown above.

I need to identify these files and move them into a error folder. Any suggestions on how to go about this would be great.

Thanks, Kavin

6
  • 1
    is key WarehouseActivity common/same for all the files? May 4, 2018 at 12:14
  • Did you try with jq?
    – dirkt
    May 4, 2018 at 12:16
  • { "x": [[],[]] } would have an array made of two empty arrays. Should that also be flagged? May 4, 2018 at 12:50
  • @RomanPerekhrest yes its same across all the files and no, i haven't tried the jq option...am gonna try the code posted by you and revert back to you. May 4, 2018 at 15:41
  • @StéphaneChazelas well, there hasn't been any such case yet. Just one empty array. May 4, 2018 at 15:42

2 Answers 2

10
mkdir -p error_folder &&
for json in ./*.json; do
    if jq -e '.. | select(type == "array" and length == 0)' "$json" >/dev/null
    then
        mv "$json" error_folder/
    fi
done

This is more or less the same approach as Roman took in his answer, but uses a different jq expression.

The expression ..|select(type == "array" and length == 0) will recurse the full JSON structure and select all bits of it that are zero-length arrays (anywhere, at any depth).

If the select() is successful, then jq will exit with a zero exit status (success), which means that the JSON document contains an empty array somewhere (or the file is totally empty). This triggers a moving of the document to error_folder in the script.


From comments below it is clear that the user is only interested in the WarehouseActivity array.

My code with a modified jq expression:

mkdir -p error_folder &&
for json in ./*.json; do
    if jq -e '.. | .WarehouseActivity? | select(type == "array" and length == 0)' "$json" >/dev/null
    then
        mv "$json" error_folder/
    fi
done
5
  • 1
    Note that jq (at least my version) also returns true here for empty files and files made entirely of blanks (which the OP may also want to consider as errors anyway). May 4, 2018 at 14:52
  • Hi @Kusalananda , i gave this code a try but it is not working as expected. It ends up moving all the json files in that folder including the empty json file. May 4, 2018 at 16:20
  • @KavinPalaniswamy Then all files contain at least one empty array somewhere, or you have an error in your script. I've tested the code in my answer.
    – Kusalananda
    May 4, 2018 at 16:24
  • @Kusalananda Yes, you are correct the json files do have empty arrays. I was looking to identify the json files which have only the below : { "WarehouseActivity": [] } Don't really want to identify all the empty arrays in the json files. But great solution. I will use this when i need to identify all the empty arrays . Thanks a lot. really appreciate your help. May 4, 2018 at 16:30
  • @KavinPalaniswamy Updated my answer.
    – Kusalananda
    May 4, 2018 at 16:39
5

Jq would be the right tool for processing/analyzing JSON data:

for f in *.json; do
    if jq -e 'keys_unsorted as $keys
              | ($keys | length == 1) 
                and .[($keys[0])] == []' "$f" > /dev/null; then
        mv "$f" error_dir/
    fi
done
5
  • this works great! It identifies the empty files and moves them to the error folder. To start with i wasn't even aware of the jq command, i had to install the jq package in my machine to get this to work. Thanks a lot. Really appreciate the help. May 4, 2018 at 16:22
  • @KavinPalaniswamy, you're welcome May 4, 2018 at 17:16
  • Hey! There is a scenario when there is no such file , am getting a error message "jq: error: Could not open file *.json: No such file or directory" . For now am just suppressing it. Any way around it ? May 10, 2018 at 2:31
  • @KavinPalaniswamy, it seems that you are not applying the above approach in exact manner. for f in *.json; will iterate through existed .json files (unless you don't have folders which end with .json) May 10, 2018 at 5:13
  • Sorry for the delayed response. Well i think i didn't explain myself well. When the folder is completely empty and there are no json files i get the error :jq: error: Could not open file Test_*.json: No such file or directory . I have implemented the exact same code given by you. As a work around i am just check if any file exist and then am running your code. This does't throw any error. Any better way to implement this sir? May 18, 2018 at 20:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .