How can I see if a consecutive number name file is missing some file? [duplicate]

Question

If I give ls -1 I get like this,

file_0001.jpeg
file_0002.jpeg
file_0003.jpeg
file_0004.jpeg
file_0005.jpeg
file_0006.jpeg
file_0007.jpeg
file_0008.jpeg
file_0009.jpeg
file_0010.jpeg
file_0011.jpeg
file_0012.jpeg
file_0013.jpeg
file_0014.jpeg
file_0015.jpeg
file_0016.jpeg
file_0017.jpeg
file_0018.jpeg
file_0019.jpeg
file_0020.jpeg
file_0021.jpeg
...
file_0999.jpeg

Is there a way using awk or other tool to see if some file is missing in this consecutive incremental way.

Answer 1

If you're using the bash shell, you can use seq or jot to create a "perfect" output reference sample, then compare the output of ls -1 to that reference:

$ diff <(ls -1) <(seq -f 'file_%04g.jpeg' 999)

This will not only show you any missing files, it will also find extraneous files.

Answer 2

While the other answers provide ways to find exactly which files are missing, the original question of seeing if some file is missing can be checked in a more typing-friendly way.

Your list is the output of ls -1, so piping it into wc should give you a number of files matching the name of the last entry.

ls -1 | wc -l

If the count doesn't match the name of the last file, there must be a missing file.

Answer 3

You could use Awk to filter out the missing ones. On GNU Awk, with support for multi-char FS, you could pipe your result to

awk -F'[_.]' ' $2 != prev+1 { print "file "  prev+1 " missing" }{ prev = $2 }'

Or using perl

perl -F'[_.]' -ane 'if ($F[1] != $prev+1) {printf "file %d missing\n",$prev+1}; $prev=$F[1]'

If there are more gaps anticipated, you could have awk print out the range of file numbers missing. Modifying the above

awk -F'[_.]' ' $2 != prev+1 { print "file(s) "  prev+1 "-" $2-1 " missing" }{ prev = $2 }' file

Answer 4

You can filter with comm.

For example, while in a directory with such files with a few missing:

$ cd "$(mktemp -d)"
$ touch file_{0001..0999}.jpeg
$ rm file_0388.jpeg file_0795.jpeg

You can filter like this:

$ ls | comm -13 - <(printf 'file_%s.jpeg\n' {0001..0999})
file_0388.jpeg
file_0795.jpeg

comm is a command to compare 2 files, showing which lines are only present in file 1, which are only in file 2, and which are in both. -1 suppresses lines only present in file 1, -2 suppresses lines only present in file 2, and -3 suppresses the lines that are present in both.

-13 is the same as -1 -3, which suppresses all lines except those that are only present in file 2, the generated filenames.

Extraneous files in the directory can be listed by using -23 instead of -13:

$ touch foo.txt bar.txt
$ ls | comm -23 - <(printf 'file_%s.jpeg\n' {0001..0999})
bar.txt
foo.txt

Not using these options, one can see everything differentiated by indentation. Lines only in file 1 don't have indentation, lines only in file 2 have 1 tab indentation, and lines present in both have 2 tabs indentation:

$ ls | comm - <(printf 'file_%s.jpeg\n' {0001..0999})
bar.txt
        file_0001.jpeg
...
        file_0386.jpeg
        file_0387.jpeg
    file_0388.jpeg
        file_0389.jpeg
        file_0390.jpeg
...
        file_0793.jpeg
        file_0794.jpeg
    file_0795.jpeg
        file_0796.jpeg
        file_0797.jpeg
...
        file_0998.jpeg
        file_0999.jpeg
foo.txt

Answer 5

Use a for loop.

 for i in file_{0000..0999}.jpeg; do if !  test -e "$i" ; then echo "$i doesn't exist"; fi; done

As a script

#! /bin/bash
for i in file_{0000..0999}.jpeg
do 
if !  test -e "$i"
then
echo "$i doesn't exist"
fi
done

Answer 6

an easy way to do that is to loop over numbers from seq command

francois@zaphod:/tmp/tata$
 ls
file_0.txt   file_1.txt   file_2.txt   file_3.txt   file_4.txt   file_60.txt  file_70.txt  file_80.txt  file_90.txt
file_10.txt  file_20.txt  file_30.txt  file_40.txt  file_50.txt  file_61.txt  file_71.txt  file_81.txt  file_91.txt
file_11.txt  file_21.txt  file_31.txt  file_41.txt  file_51.txt  file_62.txt  file_72.txt  file_82.txt  file_92.txt
file_12.txt  file_22.txt  file_32.txt  file_42.txt  file_52.txt  file_63.txt  file_73.txt  file_83.txt  file_93.txt
file_13.txt  file_23.txt  file_33.txt  file_43.txt  file_53.txt  file_64.txt  file_74.txt  file_84.txt  file_94.txt
file_14.txt  file_24.txt  file_34.txt  file_44.txt  file_55.txt  file_65.txt  file_75.txt  file_85.txt  file_95.txt
file_15.txt  file_25.txt  file_35.txt  file_45.txt  file_56.txt  file_66.txt  file_76.txt  file_86.txt  file_96.txt
file_16.txt  file_26.txt  file_36.txt  file_46.txt  file_57.txt  file_67.txt  file_77.txt  file_87.txt  file_97.txt
file_17.txt  file_27.txt  file_37.txt  file_47.txt  file_58.txt  file_68.txt  file_78.txt  file_88.txt  file_98.txt
file_18.txt  file_28.txt  file_38.txt  file_48.txt  file_59.txt  file_69.txt  file_79.txt  file_89.txt  file_99.txt
file_19.txt  file_29.txt  file_39.txt  file_49.txt  file_5.txt   file_6.txt   file_7.txt   file_8.txt   file_9.txt
francois@zaphod:/tmp/tata$
 for i in $(seq 0 99) ; do test -f file_$i.txt || echo "file_$i.txt missing" ; done                                    
file_54.txt missing
francois@zaphod:/tmp/tata$

just adapt to your 4 numbers format in your own filenames

Answer 7

(Turns out this question was a duplicate, and what I use is the same as the top answer on How to print the name of missing files in a folder?)

---

For interactive use in bash, quick and easy to type and remember:
I typically use {01..99} brace expansion to generate the expected series and look for ls errors:

ls file_0{001..999}.jpeg > /dev/null

The redirect hides the stdout listing of existing files, but stderr remains connected to the terminal. I sometimes leave out the >/dev/null and just scroll back in the terminal if there are fewer than ~100 files, because error messages show up first, while ls is still reading through its args before sorting and printing them. That also verifies that I typed it right and my pattern is matching the files I did want (especially if it includes a glob).

For piping / capturing, you could do foo=$(ls ... 2>&1 >/dev/null) to redirect stderr to the pipe, then redirect stdout to /dev/null while leaving stderr going to the shell's pipe. This would be useful for checking for empty/non-empty error output. But in a script, if you want to obtain the names of the missing files, have look at other answers instead of trying to parse it out of ls error messages which may be internationalized etc.

If you want / if necessary, you can use quotes on other parts of the filename, like 'foo bar '{01..22}.jpg. Or even foo\ bar\ {01..22}*.jpg to expand to 'foo bar '01*.jpg / 'foo bar '02*.jpg etc, so it works even if there's some extra unique name for some or all files somewhere other than the sequence numbers.

This works even if your numbering doesn't use leading zeros, e.g. {1..99} instead of {01..99}. If you want to include leading zeros inside the brace range, you can do stuff like {01..09} which works the way you'd hope, instead of factoring them out of the brace expression like I did in the example.

---

Note that modern Linux systems support very long arg lists, like 128kiB of text. This method does depend on generating a command line with every file on it. That's 100% fine for interactive use: in the rare case where the list is too huge, the shell will tell you about it.

Another answer used a for loop over the list, which would be slower (starting a separate ls for every file), but safe even in the huge, or on a more limited system with much smaller argv limits.