1

I was called to solve a situation of recovering a backup of emails from a legacy server that died.

Having Maildir files, and with some header data, I could easily determine with some detective work the server used to run a combination of postfix+dovecot, contrary to what I was being said of postfix+courier.

So I setup a Debian 9 server with Postfix 3.1.6 and dovecot-imapd 2.2.27.

After a couple of tweaks, we were already seeing the folders, however the emails looked garbled.

Inspecting the cur and new directories, the emails files looked the normal Maildir format, however opening them they seemed garbled.

Ran a command file to find the messages are all in a gzipped format.

What is happening here, was I given a backup where the admin gzipped all email box messages files before backing up them?

3

As I further explored the subject, felt something was out of place definitively.

My suspicions also grew even more, as when ungzipping some messages, the name was actually updated to reflected the new bigger size when dovecot saw them.

So upon further investigation, found dovecot can have compression on place.

From Enable compression on Dovecot storage

I’ve just discovered than compression can be applied on Dovecot mail storage.

Note that compression will only apply to “new” emails.

From Compressed Dovecot Maildir on Debian.

Add this into /etc/dovecot/conf.d/10*

# Enable zlib plugin globally for reading/writing:
mail_plugins = $mail_plugins zlib

# Enable these only if you want compression while saving:
plugin {
    zlib_save_level = 6 # 1..9; default is 6
    zlib_save = gz # or bz2, xz or lz4
}

Add this into /etc/dovecot/conf.d/20*

protocol imap {
   mail_plugins = zlib
}
protocol pop3 {
   mail_plugins = zlib
}

Restart the service dovecot:

service dovecot restart

The compression settings will then apply to all messages created after the configuration change.

There are directions on Dovecot WiKi (Zlib plugin) that would help you compress already stored emails.

From the Dovecot WiKi (Zlib plugin)

The Zlib plugin can be used to read compressed mbox, maildir or dbox files. It can be also used to write (via IMAP, LDA and/or LMTP) compressed messages to dbox or Maildir mailboxes. Zlib plugin supports compression using zlib/gzip, bzlib/bzip2, liblzma/xz (v2.2.9+) and liblz4/lz4 (v2.2.11+).

......

Maildir

When this plugin is loaded Dovecot can read both compressed and uncompressed files from Maildir. If you've enabled both gzip and bzip2 support you can have files compressed with either one of them in the Maildir. The compression is detected by reading the first few bytes from the file and figuring out if it's a valid gzip or bzip2 header. The file name doesn't matter.

Alternatively, not wanting to deal with compressed mailboxes due to performance reasons, one of the approaches to decompress them in bulk is writing a bash script.

The current script has a few shortcomings, however the general guideline is:

#!/bin/bash

TMPDIR="/tmp"

service dovecot stop

find  /home/*/Maildir -type f |

while read file
do
    file "$file" | grep gzip > /dev/null
    if [ $? -eq 0 ]
    then
        echo "$file"" gzipped...decompacting"
        mv "$file" $TMPDIR/$$.gz
        gunzip -c $TMPDIR/$$.gz > "$file"
        rm -f $TMPDIR/$$.gz 
    fi
done

find /home -name "dovecot.index.cache*" -delete

service dovecot start
  • That looks like a very weird feature. What tools support reading from this "individually compressed messages" Maildir variant? – dhag Feb 28 '18 at 21:31
  • @dhag Dovecot....it got me by surprise too. Actually, this was not the original article I found. – Rui F Ribeiro Feb 28 '18 at 21:38
  • @dhag Updated it again, this time should be more explicit. – Rui F Ribeiro Feb 28 '18 at 22:12
  • 1
    Oh I think I understand: Dovecot uses this format as its internal format, which is then only exposed through IMAP, etc., but never accessed directly from the filesystem by anything like Mutt. OK, that makes a little bit of sense now. – dhag Mar 1 '18 at 1:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.