Openldap (specifically version 2.4) stores transaction history in log files by default in the ldap data directory (so
/var/lib/ldap/log.###########). Currently these log files take up a lot of space, are never removed automatically, and grow infinitely. Manual removal for old logs works fine, but I'd like to limit the amount of logs
slapd keeps automatically.
I know that these transaction logs are used to recover ldap in case of a catastrophic failure. In my scenario ldap is regularly wiped and populated via a script (this isn't used for system login accounts). Because of this I don't need to concern myself with recovery, in case of a failure it's acceptable to run the script again. On the other hand, the regular wipe/population of ldap includes a lot of transactions, so these transaction logs build up pretty quickly.
logrotate has potential here, but if the most recent transaction log is ever removed then
slapd will fail to start (it will complain about needing to perform recovery). Because I can't rely on the log names (because
slapd keeps many small logs, incrementing the log file number as it goes) I'd like to use the Berkeley DB settings which creates these logs. I can count on the access/creation dates (most recent modify date is the most recent transaction log), but I'd still prefer to use Berkeley if possible.
The settings for the transactions logs are said to be controlled by the Berkeley DB settings in
/var/lib/ldap/DB_CONFIG. The example
DB_CONFIG that comes with openldap specifies some transaction log settings:
set_lg_regionmax 262144 set_lg_bsize 2097152
According to the Oracle documentation on Berkeley:
set_lg_regionmax: Set the size of the underlying logging area of the Berkeley DB environment, in bytes. The log region is used to store filenames, and so may need to be increased in size if a large number of files will be opened and registered with the specified Berkeley DB environment's log manager.
So this seems to just set the size of the file that tracks the transaction log files.
set_lg_bsize: Set the size of the in-memory log buffer, in bytes.
This seems to control how much RAM is allotted to the transaction buffer.
log.########### files in the ldap data directory are all
10485760 bytes which seems to correspond closely to
10485760 / 5 = 2097152 = set_lg_bsize) though I'm not sure if this is a coincidence.
My interpretation of this is that
$lg_bsize amount of transaction history is stored in memory at a time. When this limit is exceeded it pushes some of the transaction history to the most recent log file, and creates a new log if the current log reaches a certain size.
According to the Berkeley documentation transaction logs can be removed by setting the flag
DB_LOG_AUTOREMOVE in the
DB_LOG_AUTOREMOVE: If set, Berkeley DB will automatically remove log files that are no longer needed.
However when I added this to the
slapd I didn't notice a difference. I removed the old transaction logs and ran the ldap population script that I have, and was able to rack up
290MB in transaction logs. It still doesn't seem to be limiting the logs at all. The reason for this may be related to the phrase:
that are no longer needed
How does one configure the automatic removal of
slapd's transaction logs using the Berkeley DB