2

I use Ubuntu 16.04 with Nginx and Bash. I know that it's not possible to directly pipe data into zip.

For example, if you host websites on Apache/Nginx webserver, this command set would fail after filling in the password:

drt="/var/www/html"
mysqldump -u root -p --all-databases | zip "$drt/db-$date.zip"

What will be your workaround if you really desire the end file to be a zip file?

  • 2
    A workaround would be to use gzip, bzip2, or even xz instead of zip, and accept that Unix and Windows are different platforms with different software capabilities. – roaima Apr 10 '18 at 19:42
  • hahaha, that's a comment with a funny part. I accept that, but workarounds can be done, sometimes, as you know. – user9303970 Apr 10 '18 at 19:43
  • 2
    Actually, I'm really quite serious. You're backing up MySQL databases. Why not use native compression tools? – roaima Apr 10 '18 at 19:44
  • I know you are. I don't know gzip and I'm not sure I won't have problems uploading these to MySQL and MySQL like databases. It's not likely I'll have any problems but I'm not sure and I would prefer to stay with Zip that I know better, there is a small enough workaround, though I'm open to gzip or anything working slightly with these databases anyway. – user9303970 Apr 10 '18 at 19:46
4

If you really want to use zip, you can use Jeff Schaller’s trick:

drt="/var/www/html"
mysqldump -u root -p --all-databases | zip "$drt/db-$date.zip" -

This will create a ZIP file containing a file named - whose contents are the database dump. This is mentioned in the zip manpage:

zip also accepts a single dash ("-") as the name of a file to be compressed, in which case it will read the file from standard input, allowing zip to take input from another program. For example:

tar cf - . | zip backup -

You could also use /dev/stdin instead:

mysqldump -u root -p --all-databases | zip -FI "$drt/db-$date.zip" /dev/stdin

This would result in an archive containing a file named dev/stdin which might be harder to handle properly. - is a common short-hand to tell programs to use standard input or output; it’s not something that the shell handles, it has to be supported by each individual program.

In both cases you’d probably want to use funzip to extract the data; it extracts the first member of an archive to its standard output.

  • .. or unzip -p and redirect – Jeff Schaller Apr 10 '18 at 20:49
  • I must say I'm still confused about why it must be a named pipe and not a regular pipe, I mean, why would the zip utility developers care if use an anonymous or a named pipe. Maybe there isn't a clear answer to that question. – user9303970 Apr 11 '18 at 11:33
  • So instead having say mysql.sql inside the end zip I'll have - or dev/stdin with the sql data. That's a tiny bit depressing but I might need to eat more non depressing foods like Chocolate, to handle that. – user9303970 Apr 11 '18 at 11:34
  • It can be a regular pipe, if you use a hyphen. – Stephen Kitt Apr 11 '18 at 11:34
  • I didn't figure it out from the answer. :\ my apologies. – user9303970 Apr 11 '18 at 11:35
8

If you are open to alternative compression tools, try this almost identical alternative.

drt="/var/www/html"
mysqldump -u root -p --all-databases | gzip >"$drt/db-$date.gz"

If you prefer you can substitute gzip with bzip2 or xz, and the typical extension from gz to bz2 or xz.

  • xz is often better than bzip2 these days. Both are slow to compress, but xz decompresses faster and often gets a better compression ratio (tukaani.org/xz claims 15% better than bzip2) It uses LZMA2 like 7-zip. – Peter Cordes Apr 10 '18 at 21:11
  • @Peter ah yes. I'd mentioned it in my original comment (under the question) but omitted to include it here. Good reminder, thanks. – roaima Apr 10 '18 at 22:05
3

If you really prefer a zip file over using gzip or tar:

drt="/var/www/html"
scratch="$(mktemp)"
if mysqldump -u root -p --all-databases > "$scratch"; then
  zip "$drt/db-$date.zip" "$scratch"
fi
rm -f  "$scratch"

Optionally, you can use a temporary directory to have a more useful filename:

drt="/var/www/html"
scratchdir="$(mktemp -d)"
if mysqldump -u root -p --all-databases > "${scratchdir}/db-${date}.sql"; then
  zip -j "$drt/db-$date.zip" "${scratchdir}/db-${date}.sql"
fi
rm -rf "$scratchdir"
  • The name scratch is a must here? – user9303970 Apr 10 '18 at 20:16
  • 3
    No, that's just my default stand-in variable name for a "scratch file", i. e. one written to and discarded after use. mktemp creates a new file with a random name, so you need to capture that name to a variable if you want to actually do anything to it with a script. Helpfully, it outputs the path to the created file, which is why varname="$(mktemp)" works. – DopeGhoti Apr 10 '18 at 20:17
  • 1
    mktemp is guaranteed to create a new file which does not already exist. Sanity checking for existing procedural file names are left as an exercise for the OP as they are beyond the scope of the question at hand. – DopeGhoti Apr 10 '18 at 20:27
  • My only suggestion here would be to chain the mysqldump, zip, and rm together with && – Jeff Schaller Apr 10 '18 at 20:37
  • The scratch file would need to be removed either way, but I have added a sanity check to only create the archive if mysqldump succeeds. As to the complaint about renaming the scratch file, keep in mind that using a defined name was a retrofit of the earlier example in case the filename actually matters. – DopeGhoti Apr 10 '18 at 20:40

Not the answer you're looking for? Browse other questions tagged or ask your own question.