40
votes
Compress a folder with tar?
Examples for Most Common Compression Algorithms
The question title for this is simply "Compress a folder with tar?" Since this title is very general, but the question and answer are much ...
- 501
39
votes
Why can't I copy my DVD with dd?
People mention that opening the DVD with VLC (which displays the DVD menu) magically makes the data accessible to dd, but nobody has yet explained why that is and how VLC accomplishes this feat.
I ...
- 495
31
votes
Accepted
Is "dd" a reliable tool to make an SSD image?
TLDR: Use ddrescue
It supports resume/continue capabilities, has automatic logs, and tons of other options. More at the ddrescue home page.
Example syntax:
ddrescue /dev/sde yourimagename.image sde....
- 2,382
27
votes
Create a tar archive split into blocks of a maximum size
tar split archive
I found this to be the best solution for a few reasons:
It creates parts without interaction, automatically naming parts
You can use any compression you want, usual tar options
...
- 371
21
votes
Reasons for rsync NOT transferring all files?
To all other poor lost souls working from vacation in the dead of night,
--checksum makes rsync actually check if there are changes in the files, otherwise it checks timestamps and file sizes and ...
- 311
19
votes
how to tell rsync to preserve time stamp on files when source tree has a mounted point
as hr3miller already said, -a (or --archive) is equal to -rlptgoD and already includes syncing time.
However when rsync copies data to, eg. an NFS/FAT32/NTFS mount where preserving user and owner ...
- 291
19
votes
Accepted
Signal-Desktop: how to export messages?
Yes, it is possible.
Just save this in a file <yourFilename>:
sigBase="${HOME}/.config/Signal/";
key=$( /usr/bin/jq -r '."key"' ${sigBase}config.json );
db="${HOME}/.config/Signal/sql/db....
- 820
19
votes
Restoring a very old hard drive
I did something like that just a few weeks ago -- I got everything from a disk that had been sitting in storage for over 10 years! I had no intention of restoring the hard disk itself, I just wanted ...
- 1,894
19
votes
Accepted
Is it a good idea to replace files in etc with symlinks?
No!
You'll have a non-standard system. Future system updates to files in /etc will break the symlinks.
Many security and authentication programs regard their configuration files in /etc not being ...
- 5,523
18
votes
Accepted
How to take selective backup?
You could use rsync instead of cp. Mostly the same for local-to-local copies except that there's an exclude option
rsync --archive --exclude /logs /home/user/glassfish/domains/domain1 /home/user/...
- 121k
17
votes
Basic rsync command for bit-identical copies
I usually use the command from this answer. The advantage is that it also preserves extended attributes and ACLs.
rsync -avxHAX --info=progress2 / /new-disk/
The options are:
-a : all files, with ...
- 395
17
votes
How do I determine the block size for ext4 and btrfs filesystems?
You could use stat -f to get an answer of any filesystem, eg.:
% stat -f /home
File: "/home"
ID: 5013a37be3cd6a47 Namelen: 255 Type: ext2/ext3
Block size: 4096 Fundamental ...
- 3,176
17
votes
Accepted
Using RAID 1 to mirror two external backup drives
From a high level persepective, using a RAID of external disks as a backup device ...
... has the following benefits:
Logically, you only have to backup data once (the RAID layer handles the ...
- 505
16
votes
Rsync operation not permitted
The rsync man page says that -a is the same as -rlptgoD. I recommend making this replacement in your rsync command, i.e. rsync -rlptgoDvz, and then removing individual options which break under NTFS. ...
- 804
15
votes
Linux backup utility for incremental backups
I tried rsync, but it doesn't seem to be able to do what I want, or more likely, I don't know how to make it do that.
I know I could probably create a script that runs a diff and then selects the ...
- 4,835
15
votes
Accepted
How do I determine the block size for ext4 and btrfs filesystems?
You'll want to look at the data block allocation size, which is the minimum block that any file can allocate. Large files consist of multiple blocks. And there's always some "waste" at the ...
- 22.1k
15
votes
Is it a good idea to replace files in etc with symlinks?
If you have your home directory within the root filesystem, it might work. Mostly. With some pretty important caveats.
Some security-sensitive files, like /etc/sudoers, may become ineffective if ...
- 103k
14
votes
How to run a command when a directory's contents are updated?
Try entr command-line tool which can run arbitrary commands when files change. Since 2.9 release, a directory watch option (-d) was added to react to events when a new file is added to a directory.
...
- 21.5k
14
votes
Accepted
Using the /tmp folder for backups, recommended?
I would say it is not safe in general. On many systems, /tmp is cleaned on reboot by default. See /etc/default/rcS (TMPTIME defaults to 0),
# delete files in /tmp during boot older than x days.
# '0' ...
- 2,246
13
votes
Accepted
Backup my EFI boot entry for easy restore
It's easy enough to recreate a boot entry from scratch once you know how... and have the efibootmgr tool at hand, of course.
Boot0000* debian HD(13,GPT,007a058a-8e5e-45df-8d97-6575b66b5355,...
- 103k
13
votes
Accepted
How to exclude swapfile from rsync backup?
The filter file is particular about its whitespace. For example, after the leading - (dash, space) everything is a relevant character. This includes trailing space.
You can check the filter file for ...
- 121k
12
votes
Accepted
Backup and restore IMAP mail account with (open source) Linux tools
Updated Oct 2024.
Originally posted the answer when the question was 'just' 7 years old, it is now ~10 years old. At the time I was looking for a solution to migrate off of Google Workspaces as they ...
- 748
12
votes
Accepted
Serious backup options for linux disk (dmcrypt, luks, ext4, ext3, btrfs) normal and encrypted system
Linux system backup
When targeting a true full system backup, disk image backup (as asked) offer substantial advantage (detailed bellow) compared to files based backup.
With files based backup disk/...
- 14.7k
11
votes
Fastest way combine many files into one (tar czf is too slow)
Using tar with lz4 crompression like in
tar cvf - myFolder | lz4 > myFolder.tar.lz4
gives you the best of both worlds (rather good compression AND speed).
Expect a compression ratio of about 3 ...
- 781
11
votes
Linux backup utility for incremental backups
Although tar does have an incremental mode there are a couple of more comprehensive tools to do the job:
Duplicity
Duplicati
They not only support incremental backups, it's easy to configure a ...
- 111
10
votes
Accepted
Alternative of rsync --delete to move files to another directory instead of deleting
You can use --backup
-b, --backup
With this option, preexisting destination files are renamed as each file is transferred or deleted.
in combination with --backup-dir
--backup-dir=DIR
In ...
- 3,852
10
votes
How reliable is a system backup created with the dd command?
You have highlighted a real risk. If you are copying the underlying structure of a live filesystem you have no guarantee that what you have copied will be usable.
Even in the filesystem copy mounts ...
- 121k
9
votes
Accepted
Speed up / avoid calculation of rsync file lists
As far as rsync is concerned you're copying between two local file trees, so it disables most of its optimisations (including its delta algorithm for which it is famous). If you can run an rsync on ...
- 121k
9
votes
Accepted
rsync: Why doesn't --archive imply --recursive when --files-from=FILE is used?
-a, --archive
This is equivalent to -rlptgoD. It is a quick way of saying you
want recursion and want to preserve almost everything (with -H
being a notable omission). ...
- 342k
9
votes
Any work-around to pipe zip files?
If you are open to alternative compression tools, try this almost identical alternative.
drt="/var/www/html"
mysqldump -u root -p --all-databases | gzip >"$drt/db-$date.gz"
If you prefer you can ...
- 121k
Only top scored, non community-wiki answers of a minimum length are eligible