4

Openldap (specifically version 2.4) stores transaction history in log files by default in the ldap data directory (so /var/lib/ldap/log.###########). Currently these log files take up a lot of space, are never removed automatically, and grow infinitely. Manual removal for old logs works fine, but I'd like to limit the amount of logs slapd keeps automatically.

MY SCENARIO

I know that these transaction logs are used to recover ldap in case of a catastrophic failure. In my scenario ldap is regularly wiped and populated via a script (this isn't used for system login accounts). Because of this I don't need to concern myself with recovery, in case of a failure it's acceptable to run the script again. On the other hand, the regular wipe/population of ldap includes a lot of transactions, so these transaction logs build up pretty quickly.

LOGROTATE

logrotate has potential here, but if the most recent transaction log is ever removed then slapd will fail to start (it will complain about needing to perform recovery). Because I can't rely on the log names (because slapd keeps many small logs, incrementing the log file number as it goes) I'd like to use the Berkeley DB settings which creates these logs. I can count on the access/creation dates (most recent modify date is the most recent transaction log), but I'd still prefer to use Berkeley if possible.

DB_CONFIG

The settings for the transactions logs are said to be controlled by the Berkeley DB settings in /var/lib/ldap/DB_CONFIG. The example DB_CONFIG that comes with openldap specifies some transaction log settings:

set_lg_regionmax 262144
set_lg_bsize 2097152

According to the Oracle documentation on Berkeley:

set_lg_regionmax: Set the size of the underlying logging area of the
Berkeley DB environment, in bytes. The log region is used to store 
filenames, and so may need to be increased in size if a large number of 
files will be opened and registered with the specified Berkeley DB 
environment's log manager.

So this seems to just set the size of the file that tracks the transaction log files.

set_lg_bsize: Set the size of the in-memory log buffer, in bytes. 

This seems to control how much RAM is allotted to the transaction buffer.

The log.########### files in the ldap data directory are all 10485760 bytes which seems to correspond closely to set_lg_bsize (10485760 / 5 = 2097152 = set_lg_bsize) though I'm not sure if this is a coincidence.

My interpretation of this is that $lg_bsize amount of transaction history is stored in memory at a time. When this limit is exceeded it pushes some of the transaction history to the most recent log file, and creates a new log if the current log reaches a certain size.

DB_LOG_AUTOREMOVE

According to the Berkeley documentation transaction logs can be removed by setting the flag DB_LOG_AUTOREMOVE in the DB_CONFIG.

DB_LOG_AUTOREMOVE: If set, Berkeley DB will automatically remove log 
files that are no longer needed. 

However when I added this to the DB_CONFIG:

set_flags DB_LOG_AUTOREMOVE

and restarted slapd I didn't notice a difference. I removed the old transaction logs and ran the ldap population script that I have, and was able to rack up 290MB in transaction logs. It still doesn't seem to be limiting the logs at all. The reason for this may be related to the phrase: that are no longer needed

Actual Question

How does one configure the automatic removal of slapd's transaction logs using the Berkeley DB DB_CONFIG file?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.