Move files in bash script to subdirectory extracted from file name

Question

I have a directory with a bunch of backup files like this:

projectA20210901.gz
projectA20210902.gz
projectB20210901.gz
projectB20210902.gz

I would like to move all projectA* files to ./projectA/ and all projectB* files to ./projectB/ creating the subdirectories if they don't already exist.

The project names before the date 2021... vary and need to be extracted from the file name, but the date is always there and will always begin with 202 - i.e. 2021, 2022, etc. - we don't need to worry about what happens after 2029 :)

Parameter Expansion can be used to extract the project name, which will be the subdirectory name:

$ string=projectA20211103-23.gz
$ echo ${string%202*}
projectA

How can that be put into a script to run for all files in the current directory? Folders and symlinks should be ignored.

Answer 1

Your first step is good. Try using findand pipe its result in bash shell as follows:

$ find . -maxdepth 1 -iname "project?202?????.gz" -print0 | xargs -0 -I {} bash -c '
        var="$1";
        var1="${var%%202*}";
        mkdir -p "$var1";
        echo mv "$var" "$var1"' xargs_bash {}

mv ./projectB20210901.gz ./projectB
mv ./projectA20210901.gz ./projectA
mv ./projectB20210902.gz ./projectB
mv ./projectA20210902.gz ./projectA

Comments:

• You can write the one liner above on a single line.
• Once you are satisfied that the output corresponds to the mv commands you want to issue, just delete the "echo" cmd from the one liner for the move to be executed.
• I did not aim at conciseness; I favored readability.

How it works:

• find looks in the current directory and down the tree from there.
• -maxdepth 1 tells find to limit its search to the current directory, i.e. to not go down the tree.
• -iname looks for file name according to the pattern that follows, in a case independent way
• the filename pattern uses special characters: ? to signify any one character, * to signify any number of characters
• print0 outputs results placing an \0 null character at the end of each result, instead of the standard newline character. This is to take into account that sometimes filenames contain newline characters.
• xargs -0 after the pipe recognizes that each filename in the output is separated from the next by a null character
• xargs bash -c '...' spawns a bash shell whose part is to treat each output from find as needed, based on string manipulation, sub-dir creation as needed (mkdir -p), and finally moving each file to sub-dir as requested.

So the final one-liner w/o the echo cmd, is:

 $ find . -maxdepth 1 -iname "project?202?????.gz" -print0 | xargs -0 -I {} bash -c 'var="$1"; var1="${var%%202*}"; mkdir -p "$var1"; mv "$var" "$var1"' xargs_bash {}

HTH.

Answer 2

Another approach is:

cd /mybackups
for file in *;
do
    if [ -f "$file" ]; then
        newdir=${file%202*}
        mkdir -p "$newdir"
        mv "$file" "${newdir}"
    fi
done