mv: cannot stat No such file or directory in shell script

Question

I wrote a script to move some files form one folder to another folder but I got the following error, I checked 2 folders and notice for 1 folder there are such files and another there is no such files, but why all of them shows "mv cannot stat No such files or directory"

mv: cannot stat `/home/esolve/project/capture/tcp_50x50/dest_folder/194.199.68.165_tcp.folder/data/*': No such file or directory
mv: cannot stat `/home/esolve/project/capture/tcp_50x50/dest_folder/194.42.17.124_tcp.folder/data/*': No such file or directory
mv: cannot stat `/home/esolve/project/capture/tcp_50x50/dest_folder/195.113.161.13_tcp.folder/data/*': No such file or directory
mv: cannot stat `/home/esolve/project/capture/tcp_50x50/dest_folder/203.159.127.3_tcp.folder/data/*': No such file or directory
mv: cannot stat `/home/esolve/project/capture/tcp_50x50/dest_folder/212.199.61.205_tcp.folder/data/*': No such file or directory
mv: cannot stat `/home/esolve/project/capture/tcp_50x50/dest_folder/212.51.218.235_tcp.folder/data/*': No such file or directory
mv: cannot stat `/home/esolve/project/capture/tcp_50x50/dest_folder/213.73.40.105_tcp.folder/data/*': No such file or directory
mv: cannot stat `/home/esolve/project/capture/tcp_50x50/dest_folder/41.225.7.4_tcp.folder/data/*': No such file or directory
mv: cannot stat `/home/esolve/project/capture/tcp_50x50/dest_folder/83.230.127.122_tcp.folder/data/*': No such file or directory
[esolve@kitty tcp_50x50]$ ls /home/wgong/project/capture/tcp_50x50/dest_folder/194.199.68.165_tcp.folder/
[esolve@kitty tcp_50x50]$ ls /home/wgong/project/capture/tcp_50x50/dest_folder/203.159.127.3_tcp.folder/data/
129.88.70.226   132.187.230.1    138.96.116.22   155.185.54.250   192.38.109.144  193.136.227.163  193.175.135.61  195.113.161.13  83.230.127.122
130.104.72.200  132.227.62.122   147.83.29.232   156.17.10.52     192.42.43.22    193.137.173.218  193.205.215.74  212.199.61.205
131.130.69.164  132.252.152.194  148.81.140.193  157.181.175.249  192.43.193.71   193.144.21.131   193.226.19.30   212.51.218.235
131.188.44.102  134.151.255.180  152.66.245.162  160.78.253.31    193.1.170.136   193.145.46.243   194.199.68.165  213.73.40.105
131.254.208.10  138.48.3.203     152.81.47.4     192.114.4.3      193.136.166.56  193.166.160.98   194.42.17.124   41.225.7.4

the script is:

list=`ls dest_folder`
cd dest_folder
cwd=`pwd`
for folder in $list;do
        mv ${cwd}/${folder}'/data/*' ${cwd}/${folder}
done

I ran it in /home/esolve/project/capture/tcp_50x50/.

Answer 1

mv ${cwd}/${folder}'/data/*' ${cwd}/${folder}

The quotes (') there prevent the shell from doing globbing. The * is being passed literally to the mv command, which fails since it doesn't find files called * in the directories indicated.

Change this to:

mv "${cwd}/${folder}/data"/* "${cwd}/${folder}"

(Double quotes to prevent problems if you ever have a directory name with spaces in it. * outside the quotes.)

You'll still get the errors for the empty directories though. (Same sort of reason: if the file doesn't find a match for the pattern, it passes the pattern itself as an argument to the command.)

Answer 2

There are several problems with your code:

list=`ls dest_folder`

You're storing the output of ls without the trailing newlines characters into $list. ls outputs the list of file names separated with newline characters. newline is as valid a character as any in a filename, so that output cannot be used reliably. For instance the ls output for a directory that contains a and b is the same as the one for a directory that contains one file called a<newline>b.

 cd dest_folder

You're not checking for failure of that command. In general you should check the exit status of commands, but that's especially true of cd, because the rest of the commands assume you're in that new directory, and that could have dramatic consequences when you're not.

cwd=`pwd`

POSIX shells already maintain the (one) path to the current directory in the $PWD variable, so you shouldn't need to use pwd here (especially in the general case since command substitution would remove trailing newline characters from the path). Also, mv accepts relative paths so you don't need to build up the absolute path.

for folder in $list;do

Leaving a variable unquoted is the split+glob operator in shells. That is, the content of the variable is split (on the separators mentioned in $IFS with special rules for the whitespace ones), and each element resulting of that splitting is looked for wildcard characters so they can be expanded to the list of matching files.

Here, you do want the splitting, but only on newline characters, and you don't want the globbing, so you'd need to disable it:

 IFS='
 '; set -f
 for folder in $list

   mv ${cwd}/${folder}'/data/*' ${cwd}/${folder}

Again, leaving a variable unquoted is the split+glob operator. Here, you want neither, so you need to quote those variables.

As already mentioned, wildcards are only expanded when not quoted, to you need to move that * out of the quotes. If you've disabled globbing earlier with set -f, you'd need to reinstate it with set +f prior to calling that command.

A much better way to write it would be:

cd dest_folder &&
  for folder in */;do
    mv -- "${folder}data/"* "$folder"
  done

A few notes though:

• That will exclude hidden folders and won't move hidden files from the data folders.
• We're not checking for files being overridden in the process (you may want to add the -i option to mv).
• By using */, we're looping over directories only, but that includes symbolic links to directories as well. If you'd rather not, you'd need to add a [ -L "${folder%/}" ] && continue inside the loop.
• If there's no non-hidden folder in there */ will expand to itself, so you will get and error message from mv saying it can't find a file called */data/*. Similarly, if there's no non-hidden file in any of the folders, you'll get an error message that that-folder/data/* doesn't exist.