OpenLDAP: ldap db not backed up by tklbam

If you use the '-s' switch (for simulate) it will show you everything that will be backed up...

If you have some suggestions on what should be being backed up but isn't please feel free to detail it.

Thanks for posting this. Sounds like it should be included in the TKLBAM OpenLDAP profile.

Also I have just raised a bug on the TKL Isue Tracker.

In the meantime I found that it is definitely not ok to backup or restore /var/lib/ldap without shutting down slapd first.

Therefore I added a tklbam hook /etc/tklbam/hooks.d/ldap as follows:

#!/bin/bash

tklbam_operation=$1
tklbam_state=$2

if [ "$tklbam_state" == "pre" ]; then
    service slapd stop
elif [ "$tklbam_state" == "post" ]; then
    service slapd restart
fi

Thanks to point this out, I have changed the scripts as follows and it works great.

Hook:

#!/bin/bash -e
op=$1
state=$2
if [ "$state" = "pre" ]; then
    service slapd stop
elif [ "$state" = "inspect" ]; then
    if [ "$op" = "restore" ]; then
        echo "Only manual restore"
    elif [ "$op" = "backup" ]; then
        slapcat -n0 -l /root/ldap/config.ldif
        slapcat -l /root/ldap/data.ldif
    fi
elif [ "$state" = "post" ]; then
    service slapd restart
else
    echo "bad hook invocation"
fi

Restore

#!/bin/bash
service slapd stop
# config
rm -rf /etc/ldap/slapd.d/*
slapadd -n0 -F /etc/ldap/slapd.d/ -l /root/ldap/config.ldif
chown -R openldap:openldap /etc/ldap/slapd.d/*
# data
rm -rf /var/lib/ldap/*
slapadd -F /etc/ldap/slapd.d/ -l /root/ldap/data.ldif
chown -R openldap:openldap /var/lib/ldap/*
service slapd start

The LDAP schema's and index definitions are normally restored because they are stored in "/etc/ldap/slapd.d/" (and below).

To backup the LDAP data fields I use the (modified example) script "/etc/tklbam/hooks.d/ldap" (Note: /root/ldap/backup.ldif wil be normally backuped and restored):

#!/bin/bash -e
op=$1
state=$2
if [ "$state" = "pre" ]; then
    service slapd stop
elif [ "$state" = "inspect" ]; then
    if [ "$op" = "restore" ]; then
    elif [ "$op" = "backup" ]; then
        slapcat -l /root/ldap/backup.ldif
    fi
elif [ "$state" = "post" ]; then
    service slapd restart
else
    echo "bad hook invocation"
fi

I manually restore the LDAP data fields because a clean appliance doesn't have the ldap hook script with the following script:

#!/bin/bash
service slapd stop
rm -f /var/lib/ldap/*
slapadd -F /etc/ldap/slapd.d/ -l /root/ldap/backup.ldif
chown openldap:openldap /var/lib/ldap/*
service slapd start

Regards

Howard, thanks for the hint! Unfortunately I am not an expert on OpenLDAP or BerkeleyDB.

In the OpenLDAP admin guide, it says

19.4. Migration

The simplest steps needed to migrate between versions or upgrade, depending on your deployment type are:

1. Stop the current server when convenient 
2. slapcat the current data out 
3. Clear out the current data directory (/usr/local/var/openldap-data/) leaving DB_CONFIG in place 
4. Perform the software upgrades
5. slapadd the exported data back into the directory
6. Start the server

So I guess

• this procedure could be used in the tklbam pre/post hooks and
• this would the safest way to go (especially in the case of restoring to a new instance of the OpenLDAP appliance that has a newer version of slapd).

Do you have any sample backup/restore scripts based on slapcat/slapadd and covering all the dbs?

I have added additional notes to the bug report referring back to this thread.

Using my scripts I discover that the first time I make a backup the config.ldif and data.ldif files doesn't appear in the backup! In following backup's they are in the backup.

This means, to my opinion, that the check of files to be backuped is done before calling the hook scripts. I don't know where to change the order.

Hi guys,

Thanks a lot for your work! I would have looked into this myself after Howard's comments, but didn't find the time. I will test the latest version of Eric's scripts and let you know my findings.

Christoph

I can confirm that the script above works, thanks Christoph.

I have noticed errors from the "libnss-ldapd" package during backup, so I stop/start also the "nslcd" service.

Another point I found is that during "--simulate" of tklbam-backup and tklbam-restore the script is normally executed :-( Is it possible that the script is aware of the "--simulate" option?

I can confirm the issue that Eric mentions above. It can be resolved by performing the backup in the operation=backup / state=pre phase.

Also, in Eric's script, an error would occur if the backup directory didn't exist yet.

Here is my latest version of the script with backup and restore working perfectly for me: :-)

#!/bin/bash

tklbam_operation=$1
tklbam_state=$2

ldap_backup_dir=/root/ldap_backup

if [ "$tklbam_operation" == "backup" ] && [ "$tklbam_state" == "pre" ]; then
    service slapd stop

    mkdir -p $ldap_backup_dir
    slapcat -n0 -l $ldap_backup_dir/config.ldif
    slapcat -l $ldap_backup_dir/data.ldif

    service slapd start
fi

if [ "$tklbam_operation" == "restore" ] && [ "$tklbam_state" == "post" ]; then
    service slapd stop

    # restore config
    rm -rf /etc/ldap/slapd.d/*
    slapadd -n0 -F /etc/ldap/slapd.d/ -l $ldap_backup_dir/config.ldif
    chown -R openldap:openldap /etc/ldap/slapd.d/*

    # restore data
    rm -rf /var/lib/ldap/*
    slapadd -F /etc/ldap/slapd.d/ -l $ldap_backup_dir/data.ldif
    chown -R openldap:openldap /var/lib/ldap/*

    service slapd start
fi

I just closed the issue on the tracker.

I monkey patched the TKLBAM profiles for OpenLDAP that live on the Hub to include this hook so that that slapd migration happens out of the box and you don't need to install hooks to /etc/tklbam/hooks.d

It's been possible to do this since TKLBAM v1.4. From the RelNotes-1.4:

* Cryptographically signed profile-level hooks are now supported. It's a
  generic mechanism but the original motivation for this was to allow us
  to automatically fix migrations between two versions of an appliance
  (E.g., TurnKey Redmine 12 and TurnKey Redmine 13)

So that I don't have to monkey patch tklbam profiles for future versions I added the final version of the hook by Christoph to tklbam-profiles:

https://github.com/turnkeylinux/tklbam-profiles/commit/d12462fa18d2f721a...

Great news! Thanks for the confirmation. :)

Thanks a lot!

I just retested backup and restore to a new cloud server without my script present in /etc/tklbam/hooks.d, and everything worked fine. :-)