I am working on an ETL process for a customer. Another vendor has provided the raw data as a set of approximately 100 password protected ZIP files.
I want to validate that the password given is correct for this set of files.
The script I am currently working with uses a loop and 7zip:
#!/bin/bash set -x for filename in ../TheData/*Data*of*.zip; do echo "Checking $filename" 7z t -ple_super_secret_assword $filename done
The output piped to a file is like this:
Checking ../TheData/Project1999Data_1of7.zip + 7z t '-pseeeecret' ../TheData/Project1999Data_1of7.zip 7-Zip  16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Xeon(R) CPU E5-2470 0 @ 2.30GHz (206D7),ASM) Scanning the drive for archives: 1 file, 632866983 bytes (604 MiB) Testing archive: ../TheData/Project1999Data_1of7.zip ERRORS: Headers Error
When run without pipes there is a progress meter. I think the headers error is spurious since testing the archive works and spot checking some full extracts also seems OK.
Simply extracting all the data is not an option at this time, waiting on the change-request process for storage allocation.
Is there a faster way to simply make sure that a password works on a set of zip files?
Is there some way to capture the return codes from this and echo out a simple pass/fail? 7zip has several return codes documented
Is it possible to change this script to run the checks in parallel? 7z appears to use only 80% of one core, suggesting it is I/O bound.
There does not appear to be a change in performance with 7zip using either the t (test) or l (list) function.