1

How can I gzip and copy files to another folder keeping its directory structure with one command in Linux?

For example, I have:

/dir1
/dir1/file1.fit
/dir1/file2.fit
/dir1/file3.fit
/dir1/dir2/file1.fit
/dir1/dir2/file2.fit
/dir1/dir2/file3.fit

After I use a command (Lets we say I copy /dir1 to /another_dir), I want to get:

/another_dir/dir1
/another_dir/dir1/file1.fit.gz
/another_dir/dir1/file2.fit.gz
/another_dir/dir1/file3.fit.gz
/another_dir/dir1/dir2/file1.fit.gz
/another_dir/dir1/dir2/file2.fit.gz
/another_dir/dir1/dir2/file3.fit.gz

Here /another_dir is actually another hard drive. Since no enough space in this target drive (it is a data of 2TB!), please do not suggest me to copy the files first and then gzip all (or vice-versa). Similarly, the gz files should not remain in the source folder after the operation.

0

2 Answers 2

2

Assuming you're in the root folder where are all directories for compression (in your case /), you can use find along with xargs command, e.g.

find dir1/ -name "*.fit" -print0 | xargs -i% -r0 sh -c 'mkdir -vp "$(dirname "/another_dir/%")" && gzip -vc "%" | tee "/another_dir/%".gz > /dev/null && rm -v "%"'

Note: You can also replace | tee "/another_dir/%".gz > /dev/null with > "/another_dir/%".gz.

This will find all .fit files in dir1/ and pass them to xargs command for parsing where % is replaced with each of your file.

The xargs command will:

  • create the empty folder (mkdir) with its parents (-p) as a placeholder,
  • compress given file (%) into standard output (-c) and redirect compressed output to tee,
  • tee will save the compressed input into .gz file (since tee by default prints the input to the terminal screen, sending it to /dev/null will suppress it, but it'll still save the content into the given file).

After successful compression, remove the original (rm). You can always remove that part, in order to remove them manually after verifying your compressed files.

It is important that you're in relative folder to your dir1/, so all paths returned by find are relative to the current folder, so you don't have to convert absolute paths into relative (this still can be done by realpath, e.g. realpath --relative-to=$absolute $current, but it will just overcomplicate the above command).

On macOS, to use -r argument for xargs, you need to install GNU xargs (brew install xargs) and use gxargs command instead. Similar on other BSD systems.

Related question: gzip several files in different directories and copy to new directory.

1
  • 1
    The corrected command worked like a charm and I did not lose anything (I tried on junk files first). I really appreciate your help. You taught me many things today. Commented Dec 22, 2016 at 13:56
1

I am not aware of single command to do it, but over on ServerFault is example of simple script that implements the same. Note that it does not preserve file permissions or access/modification times, and may have issues if file names contain spaces or similar.

4
  • The entirety of your solution here is in the link, and that's generally discouraged here. After all, links die all the time and then this answer would no longer be helpful. It would be better to copy the important parts from that link into your answer Commented Dec 22, 2016 at 12:59
  • 1
    @EricRenouf this particular one is a link to ServerFault, so I'd have assumed that site would have the same longevity as this one? Commented Dec 22, 2016 at 13:03
  • 1
    @roaima good point there, though I'd still prefer to have answers actually contain the answer itself. A link to another relevant answer would probably be better as a comment Commented Dec 22, 2016 at 13:05
  • I am voting this answer up. Even though it is a link, it links to ServerFault which is part of the same network of StackExchange. It is really tiring and sad to see people discouraging other people to post answers. Person A posts something, Person B says: "this is not helpful" (or similar), but person B does not add anything to the discussion. Please let's stop that.
    – thiagowfx
    Commented Dec 22, 2016 at 13:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .