202

When moving large directories using mv, is there a way to view the progress (%)? The cp command on gentoo had a -g switch that showed the progress.

3

14 Answers 14

193

There's a new tool called progress that can find any descriptor related to a running command and show progress and speed: available here

progress -w

outputs the stats for all running cp,mv etc. operations

5
  • 11
    WOW, I did not expect this to be so awesome! I simply typed progress after installing and it presented me with progress for the mv command I was currently waiting for. Thanks!
    – jredd
    Commented Sep 30, 2015 at 0:59
  • 5
    Thanks! When moving stuff as a root it seems you have to use sudo (same user I guess) - you can also use watch to say what is happening every 2 seconds: watch progress -w
    – Wilf
    Commented May 30, 2016 at 20:13
  • 23
    Awesome program, but when moving e.g. a folder with many files it only shows the progress of the current file, not the whole folder in total. Instead of watch progress -w you can also simply use progress -m
    – mxmlnkn
    Commented Nov 13, 2016 at 15:38
  • 11
    IMHO, this is the most unix-way to solve the problem. Instead of patching coreutils or using tools which are not designated for specific purposes (like rsync for mv) or writing overkilling scripts, just use that tool written for the subject purpose with love.
    – Grief
    Commented Dec 28, 2016 at 1:54
  • The best solution, intuitive. When using rsync I always catch myself on checking man rsync..
    – Konrad
    Commented Sep 4, 2020 at 20:20
107

I don't like the idea to overwrite binaries from coreutil when there are simpler solutions, so here are mine:

rsync: Rsync copies files and has a -P switch for a progress bar. So if you have rsync installed, you could use a simple alias in your shells dotfile:

alias cp='rsync -aP'

The downside is, that rsync is a little bit slower than cp, but you should measure this with time and decide for your self, I can live with it :-)

Shell Script: A shell script can also create the progress bar. I found this a while ago on the net and I don't remember the source:

#!/bin/sh
cp_p()
{
   strace -q -ewrite cp -- "${1}" "${2}" 2>&1 \
      | awk '{
        count += $NF
            if (count % 10 == 0) {
               percent = count / total_size * 100
               printf "%3d%% [", percent
               for (i=0;i<=percent;i++)
                  printf "="
               printf ">"
               for (i=percent;i<100;i++)
                  printf " "
               printf "]\r"
            }
         }
         END { print "" }' total_size=$(stat -c '%s' "${1}") count=0
}

This will look like:

% cp_p /home/echox/foo.dat /home/echox/bar.dat
66% [===============================>                      ]

bar:

‘bar’ - ‘cat’ with ASCII progress bar

bar is a small shell script to display a process bar for all kind of operations (cp, tar, etc.). You can find examples on the project homepage.

Its also written for the bourne shell, so it will run nearby everywhere.

6
  • 4
    bar doesn't wrap mv yet.
    – Tobu
    Commented Jan 2, 2011 at 23:04
  • 8
    Credit for the shell script
    – Lamnk
    Commented Jun 26, 2011 at 16:42
  • 2
    Note that rsync doesn't replace mv (the OP is asking about) completely, especially when it comes to fast copies on copy-on-write filesystems like btrfs. Commented May 29, 2016 at 14:40
  • Does it work for mv ?
    – alper
    Commented Dec 1, 2020 at 11:20
  • I have "nmv" aliased locally to rsync -a --stats --progress --remove-source-files. There are different flags you may want to set to control local caching, intermediate compression, etc., depending on your setup.
    – djvs
    Commented Oct 30, 2022 at 4:13
59

You can build a patched cp and mv which then both support the -g switch to show progress. There are instructions and patches at this page. However: The page instructs you to do

$ sudo cp src/cp /usr/bin/cp
$ sudo cp src/mv /usr/bin/mv

which overwrites the original cp and mv. This has two disadvantages: Firstly, if an updated coreutils package arrives at your system, they are overwritten. Secondly, if the patched version has a problem, they might break scripts relying on standard cp and mv. I would rather do something like this:

$ sudo cp src/cp /usr/local/bin/cpg
$ sudo cp src/mv /usr/local/bin/mvg

which copies the files to /usr/local/bin which is intended for user compiled programs and gives them a different name. So when you want a progress bar, you say mvg -g bigfile /mnt/backup and use mv normally.

Also you can do alias mvg="/usr/local/mvg -g" then you only need to say mvg bigfile /mnt/backup and directly get the progress bar.

3
46

You can use pipe viewer command pv to show progress bar:

pv /original/file > /new/file

I often use this to copy a big file over a mounted network filesystem (combine with gzip and tar). The drawback is that you can only copy one file and not directory. And you must give the new file a name, you can not just give destination directory like cp does. However copying is not pv's purpose. It is a powerful tool and do much more than just copy file. See the homepage for more examples of pv.

A much better option is to use rsync -aP. If you want to mv instead, append the flag --remove-source-files. Add this to your .bashrc if you want to use the commands frequently:

alias rscp='rsync -aP'
alias rsmv='rsync -aP --remove-source-files'

The downside here is rsync only shows progress, not a progress bar.

1
  • 6
    tar cf - source-dir | pv | tar xf - -C out-dir will give you throughput of a whole directory. If you know the size and pass -s to pv it can give you progress as well.
    – arantius
    Commented Feb 10, 2018 at 17:34
10

If your goal is to move/copy a directory with progress bar, but avoiding non-terminal GUI, mc (Midnight Commander) is a good choice.

0
8

My solution is to use rsync. It can copy directories, remove the source files after a successful copy (thus "moving" them) and display progress reports, among many other features (most notably syncing partially copied directories and working over the network.)

Here is how I would move a directory with rsync, with progress report:

rsync -aP --remove-source-files $src $dst && rm -r $src

Make sure $src does not end in a backslash, otherwise you will get a different meaning than that of GNU cp.

The rm -r at the end is needed because --remove-source-files only removes the source files, after each one is successfully copied over, not the directories. The && makes sure to run it only if rsync completes successfully. If you are paranoid, you can replace it with some kind of recursive rmdir, such as find $src -d -exec rmdir {} +

The only glitch is that the -P option shows progress for each file, not for the entire copy operation. This might be useless if you are trying to move a directory with many tiny files. The latest development version of rsync has an alternate flag --info=progress2 that shows progress for the entire operation.

5

This does not give you a progress bar, but hit CTRL-T and you will send a SIGINFO to mv, which will report which file it's working on and how far along in the process it is. Pretty handy to have this for large file transfers, possibly not as useful for a transfer with many small files.

Quick example:

mv 135/61 /Volumes/Jules/135/
# hit C-t
load: 1.08  cmd: cp 35470 uninterruptible 0.00u 0.04s
135/61/1/1-s2.0-S006738-main.pdf -> /Volumes/Jules/135/61/1/1-s2.0-S006738-main.pdf  68%
2
  • 1
    This didn't work for me: I got INFO: unknown signal; killall -l lists signals. I think SIGINFO is only on some BSDs, not in standard POSIX. Commented Aug 26, 2016 at 21:42
  • 4
    Right. It does work in Mac OS X, but likely not in, say, Ubuntu.
    – Charles
    Commented Aug 27, 2016 at 9:12
4
mv -v sourcedir targetdir | pv -l -s filecount > logfile

You will need pv (pipe viewer): http://www.ivarch.com/programs/pv.shtml

mv -v will print one line per moved file and pv -l will report progress based on line count.

Use the following command to get file count:

find sourcedir | wc -l

This command will show progress information based on number of files copied. This works best if there are many smallish files. If there are only a few files which are huge then you will not have much fun.


For more alternatives, including methods which can report progress based on size, see here: progress information via pv for directory copy

Methods described there all copy the files instead of moving. but unless you are moving on the same filesystem a move is practically a copy followed by a delete.

3

Invariably I end up searching for this answer after I start the 'mv' command, at which point I don't want to start it again. If you're in this situation a very simple way to monitor the progress (or to see if it's far enough along that it's not worth killing) is to run:

du -hs /directory/you're/moving/data/to

This will just print the size of the directory so you can see how much data has been copied, or you can run it a couple of times to see how fast the data is being transfered.

1
  • if you're moving a lot of data the du command itself will also take quite some time to compute the size at dest
    – axolotl
    Commented Apr 15, 2022 at 22:28
2

To avoid coreutil patching to add progress bar to mv and cp, gcp (Goffi's CoPier) is a new alternative: http://wiki.goffi.org/wiki/Gcp/en

To use the same way as cp with options -r for recursivity and -v for verbosity.

It uses a file queue when multiple commands are invoked concurrently. It also allows to record copied file names in a log, see Source saving man page section for details.

Here is a good tutorial to get started with gcp: http://mylinuxbook.com/gcp-advanced-command-line-file-copier-inspired-by-cp/

2
  • Is there gmv like gcp for cp?
    – CodyChan
    Commented Sep 1, 2016 at 6:07
  • I have found no equivalent. By the way, there is no need for progress bar when running "mv" on a single file system... it is quite fast. If move is done across two different file systems, it is implemented as cp and then remove source if succeed, or remove target if failed. A simple shell script based on gcp may do it properly (with a first check to know if source and target are really on different file systems) Commented Sep 1, 2016 at 6:23
2

On my Mint I like to use gcp on this or the curl-solution (which is really nice)

gcp SOURCE DESTINATION

You might need to install it sudo apt-get install gcp

And a very nice and excellent task for the progress bar with info about

  • % Total
  • % Received
  • % Xferd
  • Average DL
  • Speed Upload
  • Time left
  • Time spent
  • Time total
  • Current Speed

in a nice readable format is:

curl -o DESTINATION FILE://source

Also, you might need to install it first sudo apt-get install curl

1
  • 1
    gcp doesn't support moving of file, i.e. source files have to be removed manually after successful copying. Commented May 29, 2016 at 14:31
2

I use the following script saved with rsmv name:

#!/usr/bin/env bash

rsync -aP --remove-source-files "$@" && rm -rf "${@:1:$#-1}"

It handles more then one source and filenames with whitespaces:

$ rsmv source1 source2 source\ 3 "source 4" dest
5
  • Doesn't --remove-source-files accomplish the removal? Why is the && rm ... there?
    – Jeff Schaller
    Commented Dec 17, 2019 at 12:59
  • This is otherwise very similar to existing answers: unix.stackexchange.com/a/184943/117549 and unix.stackexchange.com/a/15656/117549
    – Jeff Schaller
    Commented Dec 17, 2019 at 13:00
  • --remove-source-files removes only files. So if source arguments contains directories then they will not be deleted. rm ... handles deletion of such directories.
    – mixel
    Commented Dec 17, 2019 at 13:03
  • Yep, it's improved version that is closer to mv, because it handles many source arguments.
    – mixel
    Commented Dec 17, 2019 at 13:04
  • Be very very careful if you have symbolic links and are moving an entire dir transfer (i.e. rsync --remove-source-files A/* B/). If B/ is a symlink to A/ you will lose everything in A/ whereas with mv you would not. It happens to me once every 12 months or so and I keep having to use extundelete to undo the mistake. Commented Aug 24, 2022 at 21:36
1

First off: I never copy large files without using ionice, unless I know that I will not want to use the computer for half an hour or more.

Second: all my partitions are jouranled so intrapartition copying takes not time. If it is a long copy I do a du -sm on the files and df -m | grep copy_to_partition. Then if curious how much more time it will take I do the df again and see how much of the files was copied.

1
  • 2
    Would you care to share your ionice tips ? what priority would you give to mv cpu-wise and disk i/o wise ? Here are my settings but the CPU load is still high even after renicing : imgur.com/a/DLMJplz
    – ychaouche
    Commented Feb 17, 2019 at 11:55
1

On mac you can open Activity Monitor and go to the Disk tab, look for the process 'mv' in the list and look at 'Bytes Written'. That shows how far through the transfer is.

Not an exact solution but it does put your mind at ease to know your transfer is working and you can see the approximate speed.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .