Use parent directory base name for making a new directories in a for loop

Question

I am attempting to use a for loop to move files (e.g. .txt) into a new directory, where the original directory name as part of the name, but add an extra "with-new-files" onto the name so I know that the directory is new and what it contains.

Since I have ~10 directories with .txt files in the parent directory, I'd ideally like to use a for loop rather than having to manually create new directories and move the files.

For example, I have something similar to the following:

1_SpecialDirectoryNameA
    - fileA1.txt
    - fileA2.txt
    - unrelated_directory_A

2_SpecialDirectoryNameB
    - fileB1.txt
    - fileB2.txt
    - unrelated_directory_B

What I want:

1_SpecialDirectoryNameA
    - unrelated_directory_A

2_SpecialDirectoryNameB
    - unrelated_directory_B

New-directories-with-files
    - 1_SpecialDirectoryNameA_with_new_files
         - fileA1.txt
         - fileA2.txt
    - 2_SpecialDirectoryNameB_with_new_files
        - fileB1.txt
        - fileB2.txt

I've tried the following code, but I'm stuck at the mkdir step, where I get these messages: mkdir: 1_SpecialDirectoryNameA: No such file or directory mkdir: 2_SpecialDirectoryNameB: No such file or directory

mkdir New-directories-with-files

for dir in */   
do
    cd $dir
    mkdir "${dir}_with_new_files"
    mv *.txt "${dir}_with_new_files"
    mv "${dir}_with_new_files" ../New-directories-with-files/
    cd ../
done

So far, the only thing I've gotten to work successfully is:

for dir in */   
do
    cd $dir
    mkdir newfolder_files
    cd ../
done

Using this code I can successfully get the .txt files moved into the new "newfolder_files" directory, but then I would have to manually change the names to 1_newfolder_files before moving all of them new directory together.

I'm just not sure how to implement adding parent directory name with the ${dir} bit.

Any suggestions would be greatly appreciated, thank you!

Answer 1

Michelle, your for loop is populating dir with the directory's names including trailing slashes. You can trim them on the fly by replacing ${dir} with ${dir%%/}; the %% indicates you want to exclude the pattern that follows from the end of the variable. (Note that you're not actually changing the variable content itself, only its substitution in that point of the code.)

There's another issue with your script: your for loop will also iterate over the new directory you created in the first line ("New-directories-with-files"). Perhaps instead of for dir in */ you could use for dir in *_*/, assuming your directories' names follow the pattern in your example. Another option would be placing the "New-directories-with-files" in the parent directory.

Answer 2

The problem as I see it is that with the way you match directory names including the trailing slash, so $dir will contain 1_SpecialDirectoryNameA/ and mkdir ${dir}_with_new_files will resolve to mkdir 1_SpecialDirectoryNameA/_with_new_files and mkdir without the -p flag will not create parent folders, so thats where the error comes from.

One way to avoid this is to use ${dir%/}_with_new_files instead this is a shell parameter extension, similar to the one used int this answer

An other problem I see is that the for dir in */ part of your script will also match your newly created directory, so it may be a good idea to create it somewhere safe (such as the parent folder)

So a possible modified version of your script would be

mkdir ../New-directories-with-files

for dir in */
do
    cd $dir
    mkdir "${dir%/}_with_new_files"
    mv *.txt "${dir%/}_with_new_files"
    mv "${dir%/}_with_new_files" ../../New-directories-with-files/
    cd ../
done

mv ../New-directories-with-files .