24

I'm using this for creating backups securely (? - is it really secure? with a good password?):

# ENCRYPT
ORIGDIR="DIRECTORYNAMEHERE"; tar cvf - "${ORIGDIR}/" 2>/dev/null | gzip -9 - 2>/dev/null | openssl aes-256-cbc -salt -out "${ORIGDIR}.tar.gz.aes"

# DECRYPT
openssl aes-256-cbc -d -salt -in "ENCDIRECTORYNAMEHERE" | tar -xz -f -

Q: But how can I do this using 7z with max compression rate?

Creating temporary files besides the only OUTPUT file is not good, because if I need to compress ~100 GByte sized files/directories on a 180 GByte FS I wouldn't have enough free space (if ex.: the compressed file would take ~60 GByte).

6 Answers 6

38

This is covered in the man page of 7z:

-si    Read data from StdIn (eg: tar cf - directory | 7z a -si directory.tar.7z)
4
  • 1
    Can I put a "|" and further process it after the 7z? Aug 19, 2013 at 19:30
  • 7
    You can use -so to dump the compressed data on stdout instead of a file.
    – Wieland
    Aug 19, 2013 at 21:14
  • 1
    -so results in error E_NOTIMPL
    – phil294
    May 22, 2020 at 16:35
  • Please open a new question with an example to reproduce it.
    – Wieland
    May 23, 2020 at 7:28
11

Is there a reason you want to use 7z specifically, or do you just want better compression than gzip?

The xz utility uses the same compression algorithm as 7z (LZMA), and allows piped compression the same as gzip.

tar cvf ... | xz -9 | openssl ...
2
  • The reason I personally needed specifically 7z is ability to split to multiple volumes
    – Unirgy
    Dec 16, 2015 at 10:00
  • 1
    xz wasn't multi-threaded at the time (it may not be yet) so on a multi-core computer it could be a lot slower than 7z
    – Xen2050
    Apr 25, 2016 at 19:59
6

Some examples of reading from stdin and writing to stdout:

echo "hello world" | 7z a .xz -si -so | xz -dc # hello world
echo "hello world" | 7z a .xz -si -so | wc -b # 64
echo "hello world" | 7z a .xz -si -so | 7z x -txz -si -so # hello world
7z x -so db.sql.xz | less # preview compressed database file
  • a create archive. Requires a filename; we're using just .xz to abuse it to set the archive type, since we don't actually care about a name
  • -si read from stdin
  • -so write to stdout
  • x extract; must also specify -t the type since we didn't create an archive w/ filename
5

For the record as this thread is quite old.

7-zip does not allow streamed write of 7z format, it requires free seeking access as it writes, amongst the other stuff, header at the end of operation. It does allow to pipe standard *nix formats like bz2, gz, xz, though. It also allows to pipe single input without restrictions (with -si switch).

So one can compress single file into 7z format but cannot pipe it further.

$ cat archive.tar | 7z a -si archive.tar.7z

Other formats, like mentioned bz2, gz, xz can be written to stdout (with -so switch).

$ cat archive.tar | 7z a -tgzip -si -so -an > archive.tar.gz

==

$ 7z a -si -so -tgzip -an < archive.tar > archive.tar.gz

This obviously allows to pipe it further:

$ cat archive.tar | 7z a -tgzip -si -so -an | wc -c

Note -an switch that disables parsing of archive_name. What it actually does, god one knows - this program is bit magical and you have to cope with its quirks. Example of this behaviour can be found here.

1

To quickly create an remote backup preparation file of all my home scripts.I use the following:

# Scripts backup
ls ~/*.sh | cpio -ov | 7z a -si ~/Documents/SCRIPT_BACKUP_30062017.cpio.7z
# Scripts restore
7z x -so ~/Documents/SCRIPT_BACKUP_30062017.cpio.7z | cpio -iv   

Reason I don't backup my 'home' root is that I specifically sync and send only some directories. Notice how cpio will skip and check for newer existing files on restore. This is powerful. Your work will not be overwritten.

bud@

-1

just got it to work, postgres pgdumpall to a 7zip file: use set pgpassword=xxxxx before issuing command:

pg_dumpall -U postgres | c:\"program files"\7-Zip\7z.exe a -si e:\directory\output.file.sqlc

it just took 4 days to get it dumped and compressed! 50201104790 bytes (47GB)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .