248

I'm having problems when copying large files using nautilus (it gets stuck). I need to copy, using cp. I would like to know if there are any parameters that shows the % copied and also the transfer speed.

  • 39
    You could use rsync instead of cp. It has a --progress option. – frostschutz Feb 17 '13 at 14:15
  • 2
    Midnight Commander also works well, and has a pretty text-based UI with progress bars, but might not be installed. Try mc on the command prompt. Also, obviously, it's not cp. – a CVn Feb 17 '13 at 14:24
  • 3
    Using rsync with --progress just made me figure out what is happening. It gets stuck when it reaches the 100%, and then last a couple of minutes in end up. I think is the same that is happening with nautilus. Anyway that is another question already, so the best answer for this question is the rsync --progress, it shows size, %, speed and elapsed time. – yzT Feb 17 '13 at 15:16
  • 2
    one problem with rsync is that it runs slower than a pure copy. On my big server, the difference is 60MB/s and 300MB/s respectively. – pdwalker Nov 15 '14 at 17:37
  • 1
    @pdwalker even with the -W option? on all my hosts rsync is almost as fast as cp ... so close i regularly use rsync instead of cp. – Skaperen Oct 4 '15 at 10:48

16 Answers 16

189

rsync has a flag called progress2 which shows the overall percentage:

rsync --info=progress2 source dest
  • 5
    --info=progress2 still hasn't made it into a default repository and here it is 2015. Seems like a good idea. – Lonniebiz Apr 1 '15 at 8:27
  • 1
    --info=progress2 finally included on version 3.1.0 protocol version 31. Maybe you could edit your answer, yzT. – Sopalajo de Arrierez May 9 '15 at 21:14
  • 1
    I noticed from --info=progress2 that rsync is blazing fast! 200 MB/s, while Windows goesn't further than 90 MB/s! – ReneFroger Jun 28 '16 at 18:12
  • 1
    @ReneFroger although that might be the real speed (depending on device), often in Linux that fast speed is really false, and after the copy has been finished the system is still writing to the device, unlike in Windows that you can eject the device almost immediately. If you copy large files you can notice how it start at that fast speed but then it goes down till the standard one. – yzT Jun 29 '16 at 5:46
  • 7
    it worked for me with just --progress. rsync version 3.0.9 – adrianTNT Mar 8 '17 at 17:11
103

If you allow other tools than cp it's surely possible. For a single file you can use pv. It's a small tool providing nice statistics.

pv inputfile > outputfile

If you have multiple files or directories you can use tar:

tar c sourceDirectory | pv | tar x -C destinationDirectory

You can wrap it in a shell function. It's less to type and you get semantics close to the ones of cp. Here's a very simple (and not error-proof!) function:

cpstat () {
  tar c "$1" | pv | tar x -C "$2"
}

Note that some versions of tar don't support the abovementioned syntax (e.g. Solaris tar) and you have to use the following variant:

cpstat () {
  tar cf - "$1" | pv | (cd "$2";tar xf -)
}

You call it like this

cpstat sourceDirectory destinationDirectory

You can enhance it further, so that pv provides an estimation of the remaining time.

Another solution (as frostschutz mentioned in a comment) is to use rsync with the --progress option:

rsync --progress -a sourceDirectory destinationDirectory
  • 1
    cpstat is going in the pastebin =) – Matt Feb 17 '13 at 14:57
  • 7
    This will give you transfer speed, but not ETA, because pv doesn't know how much data will come through the pipe. You can specify this using -s. E.g. first do du -sh sourceDirectory to get a (rough) idea, and then pass it to pv, like this: pv -s 100m. – Jan Fabry May 11 '14 at 15:25
  • 1
    --info=progress2 gives you directory level progress statistics. – AlexLordThorsen Dec 5 '14 at 23:40
  • vbuf (debian, ubuntu) is a good bit faster than pv, probably because of its virtual ring buffer – Skaperen Oct 4 '15 at 10:56
  • @Skaperen I don't know vbuf. But as it seems it is in Debian unstable at the moment and therefore often not available on Servers which are usually running Debian stable. – Marco Oct 4 '15 at 18:35
56

rsync works the best for showing the progress during the copying progress.

ex:

rsync -avh --progress sourceDirectory destinationDirectory
27

As explained in other answers, cp doesn't provide that functionality. However, progress (formerly called cv) is a small tool which is very helpful in providing this missing feature.

This tool can be described as a Tiny Dirty Linux Only* C command that looks for coreutils basic commands (cp, mv, dd, tar, gzip/gunzip, cat, ...) currently running on your system and displays the percentage of copied data.

I've been using it for the past few months and it's solid.

  • doesn't seem to work with all filesystems (e.g. cifs) but +1, didn't know such crazy ting exits – akostadinov Mar 8 '15 at 17:23
  • 1
    Note that this project is now called "progress - Coreutils Progress Viewer (formerly known as cv)". – chappjc Oct 2 '15 at 17:34
17

Is it possible to see cp speed and percent copied?

Yes, it is possible to display the progress of the command cp on the terminal.


Method #1: (use progress)

Open a new terminal and type progress it will display the progress of cp command and others command such as:

cp, mv, dd, tar, cat, rsync, grep, fgrep, egrep, cut, sort, md5sum, sha1sum, sha224sum, sha256sum, sha384sum, sha512sum, adb, gzip, gunzip, bzip2, bunzip2, xz, unxz, lzma, unlzma, 7z, zcat, bzcat, lzcat, split, gpg

If you didn't install progress, you can install it from repository, for example in the CentOS with this command: yum -y install progress.

Then after you run cp command in the background, or after run cp normally, open another terminal, so just try this command:

# progress -m

See output:

[30904] cp /home/user/file.mp4
        2.3% (200.9 MiB / 8.4 GiB) 3.6 MiB/s remaining 0:38:58

Method #2: (use patch cp)

Or download the GNU coreutils and patch it.

cd /tmp
wget http://ftp.gnu.org/gnu/coreutils/coreutils-8.21.tar.xz
tar -xvJf coreutils-8.21.tar.xz
cd coreutils-8.21
wget https://raw.githubusercontent.com/atdt/advcpmv/master/advcpmv-0.5-8.21.patch
patch -p1 -i advcpmv-0.5-8.21.patch

Run:

export FORCE_UNSAFE_CONFIGURE=1
./configure
make

Copy the two files cp and mv loacated under coreutils-8.21/src/ to /usr/local/bin:

cp src/cp /usr/local/bin/cp
cp src/mv /usr/local/bin/mv

Or copy it using your file manager as root

Add the following lines to your ~/.bashrc:

alias cp='cp -gR'
alias mv='mv -g'

Logout and Login

The progress bar can be displayed without the –g or –progress-bar as recommended by the Author

See output:

0 files copied so far...                                              194.4 MiB /   8.4 GiB
[||--------------------------------------------------------------------------------]  2.3 %
Copying at   3.3 MiB/s (about 0h 5m 47s remaining)
/home/user/file.mp4                                                   194.4 MiB / 446.1 MiB
[||||||||||||||||||||||||||||||||||||----------------------------------------------] 43.6 %

Source:linuxhelp

14

Noticed this posting was somewhat old. But as I made a solution to the issue at hand, I thought I'd post my solution.

Scenario: We have a very large directory-structure we want to copy, so we issue the command cp -R sourcedir destdir in one terminal window.

Open another terminal window or push the process to the background (Ctrl+Z,bg<CR>), then let's make a progress indicator the ghetto way:

We first get the total size of the directory structure we want to copy, you do this with:

du -sh /path/sourcedirectory/

Convert the result to gigabytes (G), then substitute the n for the result you got in the variable sourcesize. Once you've inserted the sourcesize and destdir (destination directory), run the command below.

while true; do sourcesize=n destdir=/path/destinationdirectory/ copyprogress="$(export | du -sh $destdir | awk '{print $1}' | sed 's/[^0-9.]*//g' )" ; echo "scale=3 ; $copyprogress / $sourcesize * 100" | bc | xargs echo -n ; echo % completed ; sleep 10 ; done

This will print the progress every 10 seconds with messages like:

0.100% completed
0.200% completed
0.300% completed 
... etc

I'm aware that this solution did not address the speed of copying, but it would be possible to add that by by expanding the commands above, possibly making it into a bash script so it would be easier to read. For instance by checking how much data is copied pr. n time unit, the speed should be trivial to calculate and display.

  • This is actually perfect for my use case, so thanks for posting it! I want to know about the progress of copying a huge directory with thousands of files, not just one file, so the other solutions don't really help me. – Ibrahim Apr 10 '15 at 18:09
  • This is great, especially in cases such as the one I am experiencing right now: Remotely started a huge mv-job to move terrabytes of backups from one storage pool to another, without thinking about the implications beforehand. Now I can easily background the process and peek in on it. Awesome! – Riley Jan 27 '17 at 23:13
8

To improve on Marco's answer, today I wrote this version of cpstat, so you can copy many directories and use the last parameter as the destination path. Note that if you use a relative deep path as source, you can, thanks to tar, automatically create the same dependent directory tree.

function cpstat () {

    tar -cf - "${@: 1: $#-1}" |
    pv -s "$( du -cs -BK --apparent-size "${@: 1: $#-1}" |
              tail -n 1 |
              cut -d "$(echo -e "\t")" -f 1)" |
    ( cd "${@: $#}"; tar -xf - )

;}
6

Not in a standard version of cp.

You could background the cp and list the size while waiting for cp to finish.

I use this perl to monitor growth of a single file but it runs externally to the cp (or whatever else I'm doing)

5

I found this to use, worked great:

sudo watch lsof -p`pgrep -x cp`

from a comment by Mike on the question how to check progress of running cp

5

Even though rsync will give you some progress stats, if you want a progress bar you should use pv.

You can use cpv, a wrapper for pv with an easier syntax, more similar to that of cp that you can use to recursively copy directories

enter image description here

You can get it here

  • +1 for the hint on pv. Nice find! – Peter Apr 6 '18 at 8:30
4

Using a tool like mc (midnight commander) can be a good alternate option to other answers.

It's a simple file manager that shows you copy / move progress.

4

dd status=progress

Option added in GNU Coreutils 8.24+ (Ubuntu 16.04):

dd if=src of=dst status=progress

Stdout is a periodically updated line like:

462858752 bytes (463 MB, 441 MiB) copied, 38 s, 12,2 MB/s

See also: How do you monitor the progress of dd? | Ask Ubuntu

4

Another simple solution is to just watch the output folder as follows:

Linux/Cygwin: watch -n 10 --differences du -sk --apparent-size <WATCHED_FOLDER>

FreeBSD: gnu-watch -n 10 --differences du -sk <WATCHED_FOLDER>

This handy utility will run du and highlight the differences.

One other potential solution would be to write a simple script that loops, calling du and spitting out a rough estimate of the throughput.

3

This tool is a Linux utility command that looks for coreutils basic commands (cp, mv, dd, tar, gzip/gunzip, cat, etc.) currently running on your system and displays the percentage of copied data:

https://github.com/Xfennec/cv

  • 1
    This has already been suggested by @AmrMostafa in Nov '14. – Dubu May 12 '15 at 13:18
2

I just had a case where cp was more or less blocked due to a FUSE bug in the receiving filesystem.

If you can't just switch from cp (i.e. because --sparse=always) and want something that is actually reliable for progress, go via /proc.

Every 2.0s: cat /proc/20977/io                                                      Thu Sep 28 04:08:57 2017

rchar: 268573800
wchar: 268500992
syscr: 4112
syscw: 4097
read_bytes: 268697600
write_bytes: 0
cancelled_write_bytes: 0
0

VCP – copy files in a curses interface (Show the Progress Bar in Linux).

DESCRIPTION: vcp copies files and directories in a curses interface, with text only output available.  its options and output are similar to BSD's cp while adding some new features.

It provides information on:

  • files copied and left to copy
  • data written and total data size
  • data being written every second
  • two status bars, one showing current file status, the other total status (except with 1 file, both show current), and percentage

when output is sent to the console:

  • a status bar
  • size copied and speed

VCP man page: https://linux.die.net/man/1/vcp

Download vcp-2.2.tar.gz from:

http://distfiles.gentoo.org/distfiles/vcp-2.2.tar.gz

Dependence: curses.h (libncurses-dev).

On Ubuntu, install:

sudo apt-get install libncurses-dev

or

sudo apt-get install libncurses5-dev

INSTALL vcp:

make && make install

Manual:

man vcp

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.