Ryan's picture

Hello,

I've been reading through the forums and read a number of posts re: backups, TKLBAM, and TKLBAM's place as a system-level versus file-leve backup and restore mechanism.

TKLBAM does a fantastic job at the system level backup/restore stuff, and can technical work to restore files too from what I read, but it's not it's primary mission in life and it's not intended for that solution.

Scenario for me is:  I use the file server appliances as central backup servers for my customers.  With it, I have written a script that backups up all the other servers on their network (be in Windows, SQL, Linux, Proxmox VM's, etc.) to itself.  (It's a pretty cool little script for what i need it to do! :-) )

Whle I intend to install rsnapshot on this central backup VM (which will give me the ability to restore files from snapshots), I need to have a reliable TRUE backup mechanism for the data I rsync to this box.  I would like to use Duplicity to backup the /svr/storage folder (where I rsync all my backups from my other servers) so that I can restore at a file level.

(I hope I'm explaining this correct, so far.)

So my question is this:  Can I install (using apt-get install) Duplicity on the Fileserver appliance, even though TKLBAM is on there and hooks into Duplicty somehow?  Turnkey doesn't seem to think Duplicity is already installed, but, I don't know if I do an 'apt-get install duplicity' if that will break TKLBAM or if there are other negative side effects that I am not aware of.  I am really hoping I can have TKLBAM and Duplicity live on the same VM at the same time, all in harmony, as that will be the perfect solution for my problem.

On a side note-- I can't use TKLBAM on EC2 to backup/restore a fle by 1) launching a VM on AWS  2) restoring backup from TKLBAM console  3) recovering file  4) destroying instance.  This would be an acceptable solution to me IF i could get the appliance backed up in it's entirety (to Amazon) but, because of the volume of data (plus slow broadband links at customers), I can not backup to AWS (and in fact, exclude /svr/storage in the /etc/tklbam/overrides file to allow for at least system level backups to AWS.)

 

Any feedback or direction would be GREATLY appreciated, thank you!

Forum: 
Eric (tssgery)'s picture

 I think you could install duplicity alongside tklbam-duplicity. The version that tklbam uses is in a package called tklbam-duplicity and is installed under /usr/lib/tklbam so I don't suspect any conflicts.

Since you're using it for a customer though, I would either:

  • try it out out on a new applaince somewrere that won;t imact the customer if something happens
  • snapshot the customers appliance (if using VMware or similar hyervisors) so you can restore an image if necessary
Ryan's picture

Thank you Eric, appreciate the feedback!

 

I will do that-- give it a try on a test instance.  But it also made me wonder about something else; now that you have shard w/ me where the TKL Duplicity is, should I just use that for all my duplicity needs, or should I actually apt-get the Duplicity package?  From my understanding, the one TKL uses IS the same one avail as a package.  Thoughts?

Jeremy Davis's picture

I'm certainly no server guru or anything but I can't see why you couldn't... 

It isn't quite the same one as in the package manager, actually it's a slightly modified version of the Wheezy package. Have a look at the tklbam-duplicity commit history...

Obviously it's still worthy of testing though! The thing I'd be most interested in testing is if you try to run your backup while TKLBAM is running too...

Ryan's picture

Hi Eric,

I am running Debian-6-Turnkey-Fileserver-12.01, but there was not tklbam-duplicity on /usr/lib/tklbam (I also looked for it as a hidden file.)  I do a 'whereis tklbam-duplcity' and nothing turns up, same with a 'find / | grep tklbam-duplicity'

Am I doing something wrong?

Eric (tssgery)'s picture

The package is called tklbam-duplicity, the executable script is 'duplicity'.

whereis will only search the path, I think, so won't find it.

I found it using the 'locate' utility, in:

/usr/lib/tklbam/deps/bin/duplicity

Liraz Siri's picture

Yes, you should be able to install "vanilla" Duplicity via apt-get install duplicity and not have it interfere with TKLBAM's Duplicity. At least, that's the way it's supposed to work. If anything breaks it's a bug.

TKLBAM uses a patched version of Duplicity, packaged into tklbam-duplicity, which is a dependency of the tklbam package:

# apt-cache show tklbam | grep Depends
Depends: python (>= 2.4), tklbam-squid, tklbam-duplicity (>= 0.6.18), tklbam-python-boto (>= 2.3.0), turnkey-pylib (>= 0.3), turnkey-version, python-crypto, pycurl-wrapper (>= 1.1+2), python-simplejson, ca-certificates, ntpdate
The best way to find where a package's files live:
# dpkg -L tklbam-duplicity
/.
/usr
/usr/lib
/usr/lib/tklbam
/usr/lib/tklbam/deps
/usr/lib/tklbam/deps/lib
/usr/lib/tklbam/deps/lib/python2.7
/usr/lib/tklbam/deps/lib/python2.7/dist-packages
/usr/lib/tklbam/deps/lib/python2.7/dist-packages/duplicity
/usr/lib/tklbam/deps/lib/python2.7/dist-packages/duplicity/collections.py
/usr/lib/tklbam/deps/lib/python2.7/dist-packages/duplicity/gpg.py
/usr/lib/tklbam/deps/lib/python2.7/dist-packages/duplicity/tarfile.py
/usr/lib/tklbam/deps/lib/python2.7/dist-packages/duplicity/patchdir.py
/usr/lib/tklbam/deps/lib/python2.7/dist-packages/duplicity/misc.py
/usr/lib/tklbam/deps/lib/python2.7/dist-packages/duplicity/robust.py
/usr/lib/tklbam/deps/lib/python2.7/dist-packages/duplicity/filechunkio.py
/usr/lib/tklbam/deps/lib/python2.7/dist-packages/duplicity/selection.py
/usr/lib/tklbam/deps/lib/python2.7/dist-packages/duplicity/_librsync.so
/usr/lib/tklbam/deps/lib/python2.7/dist-packages/duplicity/statistics.py
/usr/lib/tklbam/deps/lib/python2.7/dist-packages/duplicity/tempdir.py
/usr/lib/tklbam/deps/lib/python2.7/dist-packages/duplicity/urlparse_2_5.py

[ ... snipped ..]

Add new comment