How can I move files and view the progress (e.g. with a progress bar)?

Question

When moving large directories using mv, is there a way to view the progress (%)? The cp command on gentoo had a -g switch that showed the progress.

Answer 1

There's a new tool called progress that can find any descriptor related to a running command and show progress and speed: available here

progress -w

outputs the stats for all running cp,mv etc. operations

Answer 2

I don't like the idea to overwrite binaries from coreutil when there are simpler solutions, so here are mine:

rsync: Rsync copies files and has a -P switch for a progress bar. So if you have rsync installed, you could use a simple alias in your shells dotfile:

alias cp='rsync -aP'

The downside is, that rsync is a little bit slower than cp, but you should measure this with time and decide for your self, I can live with it :-)

Shell Script: A shell script can also create the progress bar. I found this a while ago on the net and I don't remember the source:

#!/bin/sh
cp_p()
{
   strace -q -ewrite cp -- "${1}" "${2}" 2>&1 \
      | awk '{
        count += $NF
            if (count % 10 == 0) {
               percent = count / total_size * 100
               printf "%3d%% [", percent
               for (i=0;i<=percent;i++)
                  printf "="
               printf ">"
               for (i=percent;i<100;i++)
                  printf " "
               printf "]\r"
            }
         }
         END { print "" }' total_size=$(stat -c '%s' "${1}") count=0
}

This will look like:

% cp_p /home/echox/foo.dat /home/echox/bar.dat
66% [===============================>                      ]

bar:

‘bar’ - ‘cat’ with ASCII progress bar

bar is a small shell script to display a process bar for all kind of operations (cp, tar, etc.). You can find examples on the project homepage.

Its also written for the bourne shell, so it will run nearby everywhere.

Answer 3

You can build a patched cp and mv which then both support the -g switch to show progress. There are instructions and patches at this page. However: The page instructs you to do

$ sudo cp src/cp /usr/bin/cp
$ sudo cp src/mv /usr/bin/mv

which overwrites the original cp and mv. This has two disadvantages: Firstly, if an updated coreutils package arrives at your system, they are overwritten. Secondly, if the patched version has a problem, they might break scripts relying on standard cp and mv. I would rather do something like this:

$ sudo cp src/cp /usr/local/bin/cpg
$ sudo cp src/mv /usr/local/bin/mvg

which copies the files to /usr/local/bin which is intended for user compiled programs and gives them a different name. So when you want a progress bar, you say mvg -g bigfile /mnt/backup and use mv normally.

Also you can do alias mvg="/usr/local/mvg -g" then you only need to say mvg bigfile /mnt/backup and directly get the progress bar.

Answer 4

You can use pipe viewer command pv to show progress bar:

pv /original/file > /new/file

I often use this to copy a big file over a mounted network filesystem (combine with gzip and tar). The drawback is that you can only copy one file and not directory. And you must give the new file a name, you can not just give destination directory like cp does. However copying is not pv's purpose. It is a powerful tool and do much more than just copy file. See the homepage for more examples of pv.

A much better option is to use rsync -aP. If you want to mv instead, append the flag --remove-source-files. Add this to your .bashrc if you want to use the commands frequently:

alias rscp='rsync -aP'
alias rsmv='rsync -aP --remove-source-files'

The downside here is rsync only shows progress, not a progress bar.

Answer 5

If your goal is to move/copy a directory with progress bar, but avoiding non-terminal GUI, mc (Midnight Commander) is a good choice.

Answer 6

My solution is to use rsync. It can copy directories, remove the source files after a successful copy (thus "moving" them) and display progress reports, among many other features (most notably syncing partially copied directories and working over the network.)

Here is how I would move a directory with rsync, with progress report:

rsync -aP --remove-source-files $src $dst && rm -r $src

Make sure $src does not end in a backslash, otherwise you will get a different meaning than that of GNU cp.

The rm -r at the end is needed because --remove-source-files only removes the source files, after each one is successfully copied over, not the directories. The && makes sure to run it only if rsync completes successfully. If you are paranoid, you can replace it with some kind of recursive rmdir, such as find $src -d -exec rmdir {} +

The only glitch is that the -P option shows progress for each file, not for the entire copy operation. This might be useless if you are trying to move a directory with many tiny files. The latest development version of rsync has an alternate flag --info=progress2 that shows progress for the entire operation.

Answer 7

This does not give you a progress bar, but hit CTRL-T and you will send a SIGINFO to mv, which will report which file it's working on and how far along in the process it is. Pretty handy to have this for large file transfers, possibly not as useful for a transfer with many small files.

Quick example:

mv 135/61 /Volumes/Jules/135/
# hit C-t
load: 1.08  cmd: cp 35470 uninterruptible 0.00u 0.04s
135/61/1/1-s2.0-S006738-main.pdf -> /Volumes/Jules/135/61/1/1-s2.0-S006738-main.pdf  68%

Answer 8

Invariably I end up searching for this answer after I start the 'mv' command, at which point I don't want to start it again. If you're in this situation a very simple way to monitor the progress (or to see if it's far enough along that it's not worth killing) is to run:

du -hs /directory/you're/moving/data/to

This will just print the size of the directory so you can see how much data has been copied, or you can run it a couple of times to see how fast the data is being transfered.

Answer 9

mv -v sourcedir targetdir | pv -l -s filecount > logfile

You will need pv (pipe viewer): http://www.ivarch.com/programs/pv.shtml

mv -v will print one line per moved file and pv -l will report progress based on line count.

Use the following command to get file count:

find sourcedir | wc -l

This command will show progress information based on number of files copied. This works best if there are many smallish files. If there are only a few files which are huge then you will not have much fun.

---

to report progress based on bytes copied:

tar c sourcedir | pv -s size | tar x -C targetdir

afterwards you have to rm sourcedir manually

to get total size

du -sb sourcedir

---

See also: progress information via pv for directory copy

Answer 10

To avoid coreutil patching to add progress bar to mv and cp, gcp (Goffi's CoPier) is a new alternative: http://wiki.goffi.org/wiki/Gcp/en

To use the same way as cp with options -r for recursivity and -v for verbosity.

It uses a file queue when multiple commands are invoked concurrently. It also allows to record copied file names in a log, see Source saving man page section for details.

Here is a good tutorial to get started with gcp: http://mylinuxbook.com/gcp-advanced-command-line-file-copier-inspired-by-cp/

Answer 11

First off: I never copy large files without using ionice, unless I know that I will not want to use the computer for half an hour or more.

Second: all my partitions are jouranled so intrapartition copying takes not time. If it is a long copy I do a du -sm on the files and df -m | grep copy_to_partition. Then if curious how much more time it will take I do the df again and see how much of the files was copied.

Answer 12

On my Mint I like to use gcp on this or the curl-solution (which is really nice)

gcp SOURCE DESTINATION

You might need to install it sudo apt-get install gcp

And a very nice and excellent task for the progress bar with info about

• % Total
• % Received
• % Xferd
• Average DL
• Speed Upload
• Time left
• Time spent
• Time total
• Current Speed

in a nice readable format is:

curl -o DESTINATION FILE://source

Also, you might need to install it first sudo apt-get install curl

Answer 13

I use the following script saved with rsmv name:

#!/usr/bin/env bash

rsync -aP --remove-source-files "$@" && rm -rf "${@:1:$#-1}"

It handles more then one source and filenames with whitespaces:

$ rsmv source1 source2 source\ 3 "source 4" dest

Answer 14

On mac you can open Activity Monitor and go to the Disk tab, look for the process 'mv' in the list and look at 'Bytes Written'. That shows how far through the transfer is.

Not an exact solution but it does put your mind at ease to know your transfer is working and you can see the approximate speed.