A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Hot answers tagged

6

The relevant standard for date and time representation is ISO-8601. For times, the standard representations use either a colon, or no separator, between hours, minutes and seconds. So I would suggest skipping the separator entirely: database-username_database-name---2019-10-10-014111.sql For the sake of consistency it might make sense to remove the ...


5

Log-structured file systems provide this, albeit not in the same way as VMS. You can think of them as a circular buffer, remembering previous versions as long as there’s room for them. On Linux, one such file system is NILFS2; it provides continuous snapshots, and user-configurable garbage collection. The latter means that when a volume fills up, the user ...


5

Assuming your question is about data deduplication, there are a few file systems which support that on Linux: ZFS, with online deduplication (so data is deduplicated as it is stored), but with extreme memory requirements which make the feature hard to use in practice; Btrfs, with “only” out-of-band deduplication, albeit with tightly-integrated processes ...


4

You are much better to turn off / unplug the internal disk. But... You can actually change the UUID of your backup drive after you backup. Remember to do this for every partition. For example /dev/sdb1: # Generate a random UUID: cat /proc/sys/kernel/random/uuid 7a5e6fad-4fa2-4177-b8a6-dfeeeb6289df # Then use this for the partition UUID tune2fs -U ...


4

Using tar with lz4 crompression like in tar cvf - myFolder | lz4 > myFolder.tar.lz4 gives you the best of both worlds (rather good compression AND speed). Expect a compression ratio of about 3 even if your data contains binary files. Further reading: comparison of compression algorithms How to tar with lz4


4

Yes on a Copy On Write file systems (Btrfs, ZFS). git-annex is as close as you are likely to get on ext4. Note that you can mount --bind a LVM backed volume or a Btrfs file system over a folder in another file system.


4

"Cold" backup is much safer than "live" backup. On a running PI, run following command dd if=/dev/zero of=/tmp/zero bs=8M; rm /tmp/zero It will write zeroes on all free space, which will give high compression rate. then put raspberry pi's microSD onto another system, for example, on a desktop running ubuntu and run fdisk -l I see microSD as /dev/sdb, ...


3

rsync is rather standard tool for transferring files. One advantage of rynsc is also that it can resume interrupted transfers. It use ssh to transfer the data over network: rsync [OPTION...] SRC... [USER@]HOST:DEST For example (-a archive, -z compress, -v verbose): rsync -avz /local/path user@remote:/remote/path You can also use rsync for copying to ...


3

On machine receiver.example.com: cd /destination socat -u tcp-listen:33333,reuseaddr - | gunzip | pv -trab | bsdtar xpSf - Then on machine sender.example.com: cd /source bsdtar cf - . | pigz -3 | socat -u - tcp:receiver.example.com:33333 Transfer speed will depend on how fast and reliable your WiFi connection is and how compressible your data is (...


3

You say that you don't really want the remote server's directory tree mirrored onto your local machine. In that case rsync is the wrong tool. You could tar the remote content directly, delivering the compressed tarball to your local machine: ssh user@remote.machine tar -C /path/to/directory -czf - . | cat >tarball.tgz Unlike rsync, this maintains no ...


3

As root, just cat the partition to another partition: cat /dev/sdXn > /dev/sdYi or to a file: cat /dev/sdXn > backup.img Or to a file or partition on another machine: cat /dev/sdXn | ssh user@host 'cat > backup.img' You could use dd instead of cat, but there's no good reason to do so: dd if=/dev/sdXn of=backup.img Or if you want a progress bar ...


2

I found the "solution" was defective hw,in my case the SAS cable. After cable replacement works fine


2

1> go to MediaMarkt 2> buy a 500GB USB Hard drive 3> transfer your data onto that HD 4> plug the HD to destination machine 5> transfer data on destination machine (considering an HDD) USB2 ~25MiB/s, USB3 ~80MiB/s, ethernet 100Mbps ~12MiB/s, ethernet 1Gbps ~120MiB/s


2

To all other poor lost souls working from vacation in the dead of night, --checksum makes rsync actually check if there are changes in the files, otherwise it checks timestamps and file sizes and calls it a day, this is sufficient in 99.9% of the cases and lets you burn in hell for the rest of the 0.01% until you figure this out


2

You should use the -d option to let cpio create the leading directories (path/to) if they don't exist: cpio -id < archive.cpio path/to/file Also, bsdtar (the regular tar on FreeBSD) knows how to extract cpio archives, whether compressed or not.


2

The directory structure you're seeing in /media/<redacted>/External_Backup/Backups looks like a Git repository with some extras - and, sure enough, Kup Backup System is a backup scheduler and GUI front-end to command-line tool bup, which uses some parts of Git and supplements it with some extensions of its own. That means, your backup repository does ...


2

If I'm not mistaken, these failed transfers are printed to standard error (stderr) rather than stdout, though by default your shell will be printing both of these to your screen and the errors can be lost in the wall of text. In order to capture just the errors, redirect file descriptor 2 (stderr) to a real file: rsync -options /source/* /destination/ 2>...


2

/opt and /usr/local/ are subvolumes in a btrfs filesystem (look for subvol= in /etc/fstab to confirm). Subvolumes are similar to partitions in that tar treats btrfs subvolumes as separate mount points and will skip them if the --one-file-system option is set. This is why (1) the sda2 partition appears as multiple mount points, and (2) when backing up /, /opt ...


2

How can we enforce that files use the same inode numbers for files across storage devices? You can't. On the other hand, now that you've characterised your requirement, Each file should have a record in the form: ID: 5683958754 Date: xxxx-xx-xx Tags: picture, message ordoc Action: store Origin: client x or web-page www.sample.com ... you could use tab-...


2

Yes, it is possible. Just save this in a file <yourFilename>: sigBase="${HOME}/.config/Signal/"; key=$( /usr/bin/jq -r '."key"' ${sigBase}config.json ); db="${HOME}/.config/Signal/sql/db.sqlite"; clearTextMsgs="${sigBase}clearTextMsgs.csv"; /usr/bin/sqlcipher -list -noheader "$db" "PRAGMA key = \"x'"$key"'\";select json from messages;" > "$...


2

No, the first 512 bytes of a GPT disk are actually reserved for an MBR partition table called the Protective Partition Table. This table is optional, but the space is reserved for this. The next 512 bytes contain the GPT header, but that doesn't contain the partition table, which comes next. The partition table on a GPT disk is much larger than MBR table. ...


2

sudo dd if=/dev/sda conv=sync,noerror bs=64K Don't use conv=sync,noerror bs=64K. If there actually is a read error, dd might corrupt your data and the backup image will be useless. If you expect read errors, it's much safer to stick to ddrescue. Otherwise just get rid of the conv=noerror,sync and if there is an error you'll get a proper error message. If ...


2

If you are using rsync -n, you will not copy anything to the destination. The -n option to rsync is the same as --dry-run, i.e. "perform a trial run with no changes made" (wording taken from the manual). Remove -n from the command line and try again.


2

You can run a script at shutdown by following the instructions here. Basically, you can place the script in /etc/rc6.d/, and make it executable. The one drawback with this approach is that any script in the above directory will run as the root user, which can change file permissions and cause issues later. The alternative is to leave your script in your ...


2

e2image can be used to create an image of an ext4 file system, while only copying sectors which are in use: e2image -ra /dev/sda1 /path/to/file.img file.img will be created as a sparse file, so it will only occupy the space which is really used in the file system, even though its apparent size will reflect the capacity of the file system. You can also use ...


2

The S3QL filesystem has block level deduplication. It's promoted as an S3 cloud storage solution but it also works very nicely on local storage. Here's an example from part of our backups/archive server s3qlstat /path/to/some/archives Directory entries: 12430247 Inodes: 6343756 Data blocks: 1357349 Total data size: 12.4 TB ...


2

Do quick testing; export a table in PhpMyadmin and with the following command in terminal. mysqldump -u $UserName -p database table > /tmp/table.sql Then, open both SQL files with a text-editor; difference should be recognized immediately: I recognized lots of comments which is exported from PHPMyAdmin and the way of INSERTing data. For example: ...


2

The following is simplest procedure I came up with. It will work on any GNU/Linux handheld, but instructions for an Ubuntu touch device are provided, anyway. On the handheld device Ubuntu Touch specific: Increase the size of the system.img of your Ubports phone with some extra gigs. Ubuntu Touch specific: Make your rootfs writable sudo mount -o remount,...


2

As suggested in the comments, you have a number of filesystem-level options for backing up the Pi. If you do this you will need to exclude the virtual filesystems /proc and /sys. You also should be aware that it will not include the non-filesystem boot image or your partition table; I'll provide a suggestion for backing that up separately at the end of this ...


Only top voted, non community-wiki answers of a minimum length are eligible