How to move files into subdirectories segregated by date

Question

I have a large directory of files last modified over the past several years through now. Is there an easy command or commands I can type one-time in an interactive bash shell that can create subdirectories with the name of each subdirectory being a four-digit year, and move respective files into each subdirectory when I cannot rely on any information in the file name regarding the age of the file, and finally verify everything I ran worked properly and I didn't lose any files or data?

For example given this completely fake example:

$ cd ~/Documents
$ ls -lhrt
...
-rw-r--r--  4 user   user    4.0K Jun 29  2017 oldfile.txt
-rw-r--r--  4 user   user    4.0K May 15  2018 2018file.md
-rw-r--r--  4 user   user    4.0K Apr 14  2019 04.dat
-rw-r--r--  4 user   user    4.0K Jul 21  2019 somepage.html
drw-r--r--  4 user   user    4.0K Jul 21  2019 somepage_files
-rw-r--r--  4 user   user    4.0K Mar 13  2020 march.dat
-rw-r--r--  4 user   user    4.0K Feb 12 18:03 file02.dat
-rw-r--r--  4 user   user    4.0K Oct 11 18:03 OctReport.txt

When I'm done, I want to end up with the following:

$ cd ~/Documents
$ find .
.
./2017
./2017/oldfile.txt
./2018
./2018/2018file.md
./2019
./2019/04.dat
./2019/somepage.html
./2019/somepage_files
./2019/somepage_files/...
./2020
./2020/march.dat
./2021
./2021/file02.dat
./2021/OctReport.txt

Answer 1

You can use a for loop with date to get the modification year of each file (here assuming GNU date or compatible for its -r option) to create the directories with. Each file will then be moved to their respective directory.

for file in *; do
    [ ! -L "$file" ] &&
      dir_name=$(date -r "$file" +%Y) &&
      mkdir -p "$dir_name" &&
      mv -- "$file" "$dir_name"
done

Answer 2

Using the perl rename utility (not to be confused with rename from util-linux or any other rename):

$ rename -n -e 'BEGIN {use POSIX};
                next unless -f $_;
                my $Y = strftime "%Y", localtime((stat $_)[9]);
                mkdir $Y unless -d $Y;
                s=^=$Y/=' *
rename(04.dat, 2019/04.dat)
rename(2018file.md, 2018/2018file.md)
rename(file02.dat, 2021/file02.dat)
rename(march.dat, 2020/march.dat)
rename(OctReport.txt, 2021/OctReport.txt)
rename(oldfile.txt, 2017/oldfile.txt)
rename(somepage_files, 2019/somepage_files)
rename(somepage.html, 2019/somepage.html)

The -n option makes this a dry-run. Remove it (or replace with -v for verbose output) when you're sure it's going to do what you want. e.g.

$ find . -type f -ls
   864326      9 drwxr-xr-x   2 cas      cas            10 Dec 22 18:08 .
   863863      1 -rw-r--r--   1 cas      cas             0 Oct 11 18:03 ./OctReport.txt
   863999      1 -rw-r--r--   1 cas      cas             0 Jul 21  2019 ./somepage_files
   864123      1 -rw-r--r--   1 cas      cas             0 Mar 13  2020 ./march.dat
   863997      1 -rw-r--r--   1 cas      cas             0 May 15  2018 ./2018file.md
   864122      1 -rw-r--r--   1 cas      cas             0 Apr 14  2019 ./04.dat
   863862      1 -rw-r--r--   1 cas      cas             0 Feb 12  2021 ./file02.dat
   863998      1 -rw-r--r--   1 cas      cas             0 Jul 21  2019 ./somepage.html
   863996      1 -rw-r--r--   1 cas      cas             0 Jun 29  2017 ./oldfile.txt


$ rename -v -e 'BEGIN {use POSIX}; next unless -f $_; my $Y = strftime "%Y", localtime((stat $_)[9]); mkdir $Y unless -d $Y; s=^=$Y/=' *
04.dat renamed as 2019/04.dat
2018file.md renamed as 2018/2018file.md
file02.dat renamed as 2021/file02.dat
march.dat renamed as 2020/march.dat
OctReport.txt renamed as 2021/OctReport.txt
oldfile.txt renamed as 2017/oldfile.txt
somepage_files renamed as 2019/somepage_files
somepage.html renamed as 2019/somepage.html


$ find . -type f -ls
   863996      1 -rw-r--r--   1 cas      cas             0 Jun 29  2017 ./2017/oldfile.txt
   863862      1 -rw-r--r--   1 cas      cas             0 Feb 12  2021 ./2021/file02.dat
   863863      1 -rw-r--r--   1 cas      cas             0 Oct 11 18:03 ./2021/OctReport.txt
   863999      1 -rw-r--r--   1 cas      cas             0 Jul 21  2019 ./2019/somepage_files
   864122      1 -rw-r--r--   1 cas      cas             0 Apr 14  2019 ./2019/04.dat
   863998      1 -rw-r--r--   1 cas      cas             0 Jul 21  2019 ./2019/somepage.html
   864123      1 -rw-r--r--   1 cas      cas             0 Mar 13  2020 ./2020/march.dat
   863997      1 -rw-r--r--   1 cas      cas             0 May 15  2018 ./2018/2018file.md

see perldoc -f stat and perldoc -f localtime for details on those perl functions. Details on strftime() can be found in perldoc POSIX.

BTW, rename can take filenames as args on the command line, or from stdin (even NUL-separated filenames if you use rename's -0 option, which is useful when dealing with filenames containing newlines and other annoying characters). See man rename.

Answer 3

find may be of great use here - note that printf needs GNU-find:

1. Create necessary directories

   find -maxdepth 1 -mindepth 1 -type f -printf '%TY\n' | sort -u | xargs mkdir

2. Move files accordingly:

   find -maxdepth 1 -mindepth 1 -type f -printf '%f\0%TY/%f\0' | xargs -0 -l2 mv --

Using NUL-delimited names in step 2 makes this approach very robust towards filenames with spaces, newlines, etc.

When it comes to including directories (your text says files, but your example includes directories), one would have to exclude the year-named directories in step 2:

find -maxdepth 1 -mindepth 1 ! -name '[12][0129][0-9][0-9]' -printf '%f\0%TY/%f\0' | xargs -0 -l2 mv

... assuming that you have no files older than the year 1000 or newer than 2999, hopefully.

---

Maybe you also want to update the age of the directories? Then run:

for dir in [12][0129][0-9][0-9] ; do
   touch -d ${dir}-01-01 $dir
done

That way the age of the directory is set to first of Jan of the respective year and future, date-based searches are simplified.