9

I have 2000+ files in a folder, but there are few files missing from the folder.

Name of the files are like

GLDAS_NOAH025SUBP_3H.A2003001.0000.001.2015210044609.pss.grb GLDAS_NOAH025SUBP_3H.A2003001.0600.001.2015210044609.pss.grb GLDAS_NOAH025SUBP_3H.A2003001.1200.001.2015210044609.pss.grb GLDAS_NOAH025SUBP_3H.A2003001.1800.001.2015210044609.pss.grb GLDAS_NOAH025SUBP_3H.A2003002.0000.001.2015210044609.pss.grb GLDAS_NOAH025SUBP_3H.A2003002.0600.001.2015210044609.pss.grb GLDAS_NOAH025SUBP_3H.A2003002.1200.001.2015210044609.pss.grb GLDAS_NOAH025SUBP_3H.A2003002.1800.001.2015210044609.pss.grb GLDAS_NOAH025SUBP_3H.A2003003.0000.001.2015210044609.pss.grb GLDAS_NOAH025SUBP_3H.A2003003.0600.001.2015210044609.pss.grb GLDAS_NOAH025SUBP_3H.A2003003.1200.001.2015210044609.pss.grb GLDAS_NOAH025SUBP_3H.A2003003.1800.001.2015210044609.pss.grb

001 indicates day, while 0000 is the hour.

How to find out which file is missing in the folder? I got few answer in google but could not figure out how to implement those.

8
  • You mean, every day from 001 to xyz there must be 4 files: 0000, 0600, 1200 and 1800? And if not that name should be printed? Do I understand this correct?
    – chaos
    Jul 29, 2015 at 12:54
  • In *nix, we call them 'directories', not folders. Folders is a Windows term.
    – Rob
    Jul 29, 2015 at 13:01
  • 1
    @chaos it is 001 to 365 & yes you got my point.
    – Maria
    Jul 29, 2015 at 13:33
  • 3
    @Rob: meh. Both terms are pretty well understood. Actually, Apple probably came up with the folder metaphor around the time of the first release of MacOS before MS Windows even existed. And since as we all know MacOS is UNIX, that makes folder a UNIX term :-)
    – Celada
    Jul 29, 2015 at 13:39
  • 2
    Folder and directory are very different metaphors, though. And directories in Unix behave very much like directories in real-life and unlike folders in real-life, whereas folders in Windows behave much more like real-life folders than real-life directories, so it makes sense to use the term which more closely resembles the corresponding real-life concept. I have personally seen data loss caused by a user thinking directories behaved like folders because his teachers kept calling them folders instead of directories. Jul 29, 2015 at 16:10

4 Answers 4

14

With zsh or bash4, you can use brace expansion for that:

ls -d GLDAS_NOAH025SUBP_3H.A2003{001..006}.{0000,0600,1200,1800}.001.2015210044609.pss.grb >/dev/null

Notice the brackets:

  • {001..006} means expand to 001, 002, ... 006
  • {0000,0600,1200,1800} to every one of the above add 0000, 0600, 1200 and 1800.
  • >/dev/null is to avoid the standard output of ls -> we only want standard error

Now if one file is not present, ls will show an error for that:

ls: cannot access GLDAS_NOAH025SUBP_3H.A2003004.0000.001.2015210044609.pss.grb: No such file or directory
ls: cannot access GLDAS_NOAH025SUBP_3H.A2003004.0600.001.2015210044609.pss.grb: No such file or directory
ls: cannot access GLDAS_NOAH025SUBP_3H.A2003004.1200.001.2015210044609.pss.grb: No such file or directory
ls: cannot access GLDAS_NOAH025SUBP_3H.A2003004.1800.001.2015210044609.pss.grb: No such file or directory
ls: cannot access GLDAS_NOAH025SUBP_3H.A2003005.0000.001.2015210044609.pss.grb: No such file or directory
ls: cannot access GLDAS_NOAH025SUBP_3H.A2003005.0600.001.2015210044609.pss.grb: No such file or directory
ls: cannot access GLDAS_NOAH025SUBP_3H.A2003005.1200.001.2015210044609.pss.grb: No such file or directory
ls: cannot access GLDAS_NOAH025SUBP_3H.A2003005.1800.001.2015210044609.pss.grb: No such file or directory
ls: cannot access GLDAS_NOAH025SUBP_3H.A2003006.0000.001.2015210044609.pss.grb: No such file or directory
ls: cannot access GLDAS_NOAH025SUBP_3H.A2003006.0600.001.2015210044609.pss.grb: No such file or directory
ls: cannot access GLDAS_NOAH025SUBP_3H.A2003006.1200.001.2015210044609.pss.grb: No such file or directory
ls: cannot access GLDAS_NOAH025SUBP_3H.A2003006.1800.001.2015210044609.pss.grb: No such file or directory

With ksh93, replace {001..006} with {1..6%.3d}.

5
  • 2
    Cool .... + 1 !
    – JJoao
    Jul 29, 2015 at 13:28
  • Thanks but there are 365 days, so {001..006} cannot be expanded to that extent in the command. What to do for this ?
    – Maria
    Jul 29, 2015 at 13:30
  • 3
    @Maria Why not, have you tried {001..365}, for me it works
    – chaos
    Jul 29, 2015 at 13:33
  • 1
    See also the {0000..1800..600} form. Jul 29, 2015 at 13:58
  • You don't really need the minutes in the list. The second brace expansion could just be .{00,06,12,18}00..
    – WAF
    Jul 29, 2015 at 18:37
7

A variation on @chaos solution (bash 4.0 or above or zsh 4.3.11 and above):

for a in GL.....2003{001..365}.{00..18..6}00.001.2015210044609.pss.grb 
do  
  [[ -f $a ]] || echo "$a"
done

or

for a in {001..365}.{00..18..6}
do
  [[ -f "GL.....2003${a}00.001.2015210044609.pss.grb" ]] || echo "$a"
done

to print only the missing day+hour

0
4

While chaos's answer is good to be used in interactive shells, this one can be used as a POSIX script, for example if you need to do this periodically and/or do it on another computers.

#!/bin/sh
i=0
while test "$((i+=1))" -lt 366 ; do
    for j in 00 06 12 18 ; do
        file="GLDAS_NOAH025SUBP_3H.A2003$(printf '%03d' "$i").${j}00.001.2015210044609.pss.grb"
        test -e "$file" || echo "$file"
    done
done

(seq or brace expansion aren't specified by POSIX.)

2

Build the file names in a loop and then test for non-existence of a file:

for day in `seq -f "%03g" 1 30`
  do
  for hour in 0000 0600 1200 1800
    do
    filename="GLDAS_NOAH025SUBP_3H.A2003${day}.${hour}.001.2015210044609.pss.grb"
    if [[ ! -e $filename ]]
    then
      echo "File missing: $filename"
    fi
  done
done

Note: I do not guarantee this example to be error-free. It is an example, not a known-working script.

Portability: needs ksh, bash or zsh and a system with the GNU seq command available.

1
  • The concept is the one I converged on as well, but please note that very few Julian months have 30 days.
    – WAF
    Jul 29, 2015 at 13:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .