You are here:Home » Frequently Asked Questions » Backup and Migration (TKLBAM) » Usage » Frequently Asked Questions - Usage
- How do I exclude a database or table from my backup?
- I forgot my passphrase, and I "lost" my escrow key. Can you help me?
- Is there a limit to how frequently I can backup?
- How often does a full backup happen, how can I configure this?
- What's the difference between a full backup and an incremental backup?
- How does TKLBAM know what to backup on my system?
- How do I add a directory to my backup?
- How do I remove a file or directory from being included in my backup?
- How do I backup to local storage (instead of S3)?
- How do I tune and optimize a TKLBAM backup?
- Can I have multiple TKLBAM backups on a single system?
- Can I use TKLBAM to only backup a single directory?
By adding a negative database override to /etc/tklbam/overrides:
# exclude drupal5 database echo -mysql:drupal5 >> /etc/tklbam/overrides # exclude sessions table in drupal6 database echo -mysql:drupal6/sessions >> /etc/tklbam/overrides
Or on the command line:
tklbam-backup -- -mysql:drupal6/page_cache
By default ALL databases are backed up so adding a negative database override excludes only that database or table from the backup.
Excluding a table only excludes its data. The schema is still backed up as long as the database is included.
Specifying a positive database override changes the default behavior so that only the database or table specified in the override is included in the backup.
You can mix positive overrides with negative overrides.
Sorry, if your server is gone (e.g., terminated EC2 instance) nobody can help you. Next time either save an escrow key somewhere[s] safe or don't set a passphrase.
Don't misunderstand, we'd love to help if we could, but we can't. The encryption key for your backup was generated locally on your server not ours. We designed passphrase protection to use special cryptographic countermeasures to make typical cracking techniques (e.g., dictionary attacks) very difficult even for someone with access to massive amounts of computer resources.
Note, if the system you backed up is still available, just log into it as root and change the passphrase (you don't need to know the old passphrase):
No, but if you backup more frequently (e.g., hourly instead of daily), we strongly recommend creating full backups more frequently - daily or weekly instead of monthly (I.e., the default).
The reason is that long backup chains are inefficient and more vulnerable if something goes wrong as links in the chain depend on one another.
If you backup daily, and do a full backup monthly, your backup chains will consist of a full backup with at most 31 incremental backups linked to it. But if you backup hourly, by the end of the month your backup chain could consist of up to 744 incremental backups, all of which have to be downloaded and extracted when you restore.
To configure automatic hourly backups with a full backup every 7 days:
mv /etc/cron.daily/tklbam-backup /etc/cron.hourly chmod +x /etc/cron.hourly/tklbam-backup echo full-backup 7D >> /etc/tklbam/conf
By default, a full backup will happen if the last full backup is older than 30 days. Between full backups, all backup sessions are incremental.
We recommend enabling the daily backup cron job so that daily incremental backups happen automatically:
chmod +x /etc/cron.daily/tklbam-backup
You can override the default by setting the full-backup parameter in the tklbam configuration:
# create a full backup every 14 days echo full-backup 14D >> /etc/tklbam/conf
A full backup is a backup that can be restored independently of any backup other backup. An incremental backup links with the last backup before it and only includes changes made since.
Backup chains are links of backup sessions which start with a full backup, and then a series of incremental backups each recording only the changes made since the backup before it. Incremental backups are useful because they are fast and efficient.
Restoring an incremental backup requires retrieving the volumes of all backup sessions made before it, up to and including the full backup that started the chain. The longer the backup chain, the more time it will take to restore.
Every TurnKey appliance that TKLBAM supports has a corresponding backup profile, which is downloaded from the Hub when you initialize TKLBAM. The profile is used to calculate the list of system changes we need to backup. It usually describes the installation state of a TurnKey appliance and contains a list of packages, filesystem paths to scan for changes and an index of the contents of those paths which records timestamps, ownership and permissions.
You can also generate your own custom profiles with the following command:
The backup profile is stored in /var/lib/tklbam/profile and contains the following text files:
- dirindex.conf: a list of directories to check for changes by default. This list does not include any files or directories maintained by the package management system.
- dirindex: appliance installation state - filesystem index
- packages: appliance installation state - list of packages
Users can override which files and directories are checked for changes by configuring overrides (See below).
By adding an override to /etc/tklbam/overrides:
echo /mnt/images >> /etc/tklbam/overrides
Or on the command line:
Make sure you understand the implications of doing this. For example, if you add a directory handled by package management this may break package management on the system you restore to.
By adding a negative override to /etc/tklbam/overrides:
echo -/var/www/*/logs >> /etc/tklbam/overrides
By default TKLBAM is designed to work with S3 automatically, and this is the easiest and safest option. In manual mode, TKLBAM can also work with non-S3 storage addresses, but this complicates usage and carries additional risks which you should make sure you understand first to avoid data loss.
Of all non-S3 manual storage targets, the local filesystem is the simplest options because you don't need to mess around with authentication credentials.
So assuming you want to store your backup at /mnt/otherdisk:
tklbam-backup --address file:///mnt/otherdisk/tklbam/backup tklbam-escrow /mnt/otherdisk/tklbam/key
And restore like this:
tklbam-restore --address file:///mnt/otherdisk/tklbam/backup --keyfile=/mnt/otherdisk/tklbam/key
Not as easy as the Hub-enabled "automatic" mode, but still easier than a conventional backup process. Linux supports mounting most types of storage devices (e.g., external harddrive, local network file share) to the filesystem though this can require extra configuration at the operating system level.
One of the main disadvantage of using a local storage target, besides the more complicated setup and maintenance process is that you won't be able to restore/test your backup in the cloud, or from a VM running in another office branch (for example).
Also keep in mind that a physical hard disk, even a RAID array, provides much lower data reliability than the 11 nines (99.999999999%) of Amazon S3
For this reason we recommend users use local backups to supplement cloud backups (e.g., providing fast local access).
One of my favorite ways to do this:
# step 1: generate a backup dump tklbam-backup --dump=/tmp/mybackup # step 2: interactively review the dump's file contents & disk usage cd /tmp/mybackup apt-get install ncdu ncdu # step 3: add includes or excludes, go back to step 1, rinse, repeat vim /etc/tklbam/overrides # Everything perfect? tklbam-backup --upload-raw=/tmp/mybackup
By default, TKLBAM will automatically determine what paths and databases need to be backed up on a given TurnKey system according to the backup profile it gets from the Hub. The default profile tracks changes to the user-servicable, customizable parts of the filesystem (e.g., /etc /root /home /var /usr/local /var /opt /srv) while ignoring changes in areas maintained by the package management system.
You can "override" the default backup profile configuration by specifying overrides, either on the command line, or preferably by editing the /etc/tklbam/overrides configuration file.
Yes. For example, let's say your default TKLBAM backup is several gigabytes in size and you'd like to create a lighter 100 MB backup that will be updated more frequently and take less time to update/restore:
cp -a /etc/tklbam /etc/tklbam.light echo -/var/www/\*/logs >> /etc/tklbam.light/overrides echo -/home/liraz/bigfiles >> /etc/tklbam.light/overrides echo -mysql:mydatabase/bigtable >> /etc/tklbam.light/overrides export TKLBAM_CONF=/etc/tklbam.light mkdir /var/lib/tklbam.light export TKLBAM_REGISTRY=/var/lib/tklbam.light tklbam-init tklbam-backup
For convenience you may want to create a script that sets the TKLBAM_REGISTRY and TKLBAM_CONF environment variables:
cat > /usr/local/bin/tklbam-backup-light << EOF #!/bin/bash export TKLBAM_CONF=/etc/tklbam.light export TKLBAM_REGISTRY=/var/lib/tklbam.light tklbam-backup EOF chmod +x /usr/local/bin/tklbam-backup-light
Yes. Here are a couple of recommended ways to do this:
Create a separate backup with an empty backup profile:
export TKLBAM_REGISTRY=/var/lib/tklbam.srv export TKLBAM_CONF=/etc/tklbam.srv tklbam-init --force-profile=empty tklbam-backup --skip-packages --skip-database -- /srv
Use the --raw-upload option
This lobotomizes TKLBAM so instead of creating a system level backup it just backs up the directory you specify. In other words, --raw-upload turns TKLBAM into a directory-level backup tool rather than a system-level backup tool.
For example, let's say you have a collection of big files at /srv that you don't want to include in your system backup (e.g., because you don't want to bloat your backup).
So you configure an overrides to exclude the /srv directory from your default backup and create another TKLBAM backup just for the big files:
echo -/srv >> /etc/tklbam/overrides export TKLBAM_REGISTRY=/var/lib/tklbam.srv-raw tklbam-backup --raw-upload=/srv
Later, you'll need to use the --raw-download option to restore:
tklbam-restore --raw-download=/srv <your-backup-id>
If you don't use the raw-download option, TKLBAM will assume you are trying to restore a system-level backup and you'll get an error.
- Top downloads
- About TurnKey
- Apps (105)
- TurnKey Hub