I have a list of datasets containing satellite data arranged in monthly folders as follows:

01  02  03  04  05  06  07  08  09  10  11  12

These folder are further divided into daily data folder for example for first month 01, daily files are arranged in folder as:

01  02  03  04  05  06  07  08  09  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31

Which eventually has data sets:

OMI-Aura_L2-PROFOZ_2016m0101t0059-o60970_v003-2016m0107t153711.he5.met  OMI-Aura_L2-PROFOZ_2016m0101t1410-o60978_v003-2016m0107t153715.he5
OMI-Aura_L2-PROFOZ_2016m0101t0237-o60971_v003-2016m0107t153714.he5      OMI-Aura_L2-PROFOZ_2016m0101t1410-o60978_v003-2016m0107t153715.he5.met
OMI-Aura_L2-PROFOZ_2016m0101t0237-o60971_v003-2016m0107t153714.he5.met  OMI-Aura_L2-PROFOZ_2016m0101t1549-o60979_v003-2016m0107t153713.he5
OMI-Aura_L2-PROFOZ_2016m0101t0416-o60972_v003-2016m0107t153715.he5      OMI-Aura_L2-PROFOZ_2016m0101t1549-o60979_v003-2016m0107t153713.he5.met
OMI-Aura_L2-PROFOZ_2016m0101t0416-o60972_v003-2016m0107t153715.he5.met  OMI-Aura_L2-PROFOZ_2016m0101t1727-o60980_v003-2016m0107t153718.he5
OMI-Aura_L2-PROFOZ_2016m0101t0555-o60973_v003-2016m0107t153709.he5      OMI-Aura_L2-PROFOZ_2016m0101t1727-o60980_v003-2016m0107t153718.he5.met
OMI-Aura_L2-PROFOZ_2016m0101t0555-o60973_v003-2016m0107t153709.he5.met  OMI-Aura_L2-PROFOZ_2016m0101t1906-o60981_v003-2016m0107t153716.he5
OMI-Aura_L2-PROFOZ_2016m0101t0734-o60974_v003-2016m0107t153717.he5      OMI-Aura_L2-PROFOZ_2016m0101t1906-o60981_v003-2016m0107t153716.he5.met
OMI-Aura_L2-PROFOZ_2016m0101t0734-o60974_v003-2016m0107t153717.he5.met  OMI-Aura_L2-PROFOZ_2016m0101t2045-o60982_v003-2016m0107t153719.he5
OMI-Aura_L2-PROFOZ_2016m0101t0913-o60975_v003-2016m0107t153711.he5      OMI-Aura_L2-PROFOZ_2016m0101t2045-o60982_v003-2016m0107t153719.he5.met
OMI-Aura_L2-PROFOZ_2016m0101t0913-o60975_v003-2016m0107t153711.he5.met  OMI-Aura_L2-PROFOZ_2016m0101t2224-o60983_v003-2016m0107t153717.he5

I want to extract only files with .he5 extension and convert it to netcdf file using following code where file name is preserved.

ncks inputfile.he5 inputfile.nc

I am trying to process every file so I wrote a shell script as follow

shopt -s globstar
   for f in ./**; do
       echo "$f" |grep -v .met | grep ".he5"
       echo "$f" |grep -v .met |ncks $(grep".he5") $(echo $(grep  -o 'OMI-Aura_L2-PROFOZ_[0-9]\{4\}m[0-9]\{4\}t[0-9]\{4\}-o[0-9]\{5\}_v003-[0-9]\{4\}m[0-9]\{4\}t[0-9]\{6\}').nc)


It is able to extract files names but, I am not getting the ouput. How can I convert all files eventually?

2 Answers 2


Your command is overly complicated, you don't need any grep or alike:

shopt -s globstar
for f in ./**/*.he5; do
    ncks "$f" "${f%.*}.nc"
  • ${f%.*} remove all characters beginning with the last .

Try using find and its execdir option which it allows find to switch the directory to the one that a file found then execute your command within that file's parent directory as following:

find /path/to/source/directory -type f -name '*.he5' -execdir sh -c '
    for file; do
        ncks "${file}" ${file%.he5}.nc"
' sh_ncks {} +

The ${parameter%word} syntax is the Parameter Expansion, that strips the shortest suffix from its parameter.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .