damelco's picture

I am using the latest Wordpress App

and executing;

tklbam-backup --dump=/tmp/mybackup

 it results in a 8,797 byte dump without error.

 

Do I need to do something else to get it to dump the www-data folder and other pertinent appliance data??

 

thanks

 

-DJ

Forum: 
Tags: 
Jeremy Davis's picture

Assuming that you have linked your server to the Hub, then TKLBAM is a "smart backup that will only backup things that have changed. So if nothing has changed yet, then it's expected that an initial backup will be essentially empty.

I suggest that you delete the /tmp/mybackup directory, add a new file to /var/www and try again. I.e.:

rm -rf /tmp/mybackup
echo "this is a test file - if you can read this we're all good" > /var/www/test.txt
mkdir /tmp/mybackup
tklbam-backup --dump=/tmp/mybackup

Then to double check it works as expected:

cat /tmp/mybackup/var/www/test.txt
damelco's picture

... along with a pretty large database also exists.

I did run your test script. it created an empty file. I manually added content and re-ran

tklbam-backup --dump=/tmp/mybackup

still not included in the dump.

thoughts?

Jeremy Davis's picture

I was working on the assumption that you are linked to the TurnKey Hub!? Is that the case, or are you running in "solo" mode?

If you're running in "solo" mode, that might explain it. To do that, you either need to generate your own profile, or use an empty one. If you use an empty one, then it will only backup the DB by default, to include other files, you'll need to add them to the overrides file (/etc/tklbam/overrides.conf).

If that's not the case, then I'm not really sure what could be wrong? Perhaps try manually clearing the cache files? I.e.:

rm -rf /var/cache/tklbam/*
rm -rf /var/cache/duplicity/*

Then run it manually again.

If it still doesn't work, please post the full output of TKLBAM, plus the output of these:

turnkey-version
apt policy tklbam
df -h
damelco's picture

as far as I know, so I would guess I'm in "solo" mode. though I don't remember reading about that label anywhere. my only link is a mounted local nas to push the backups to.

odd though??? you indicate it should be backing up the database by default at least ... but it is not?!

as for "profile" ... I was under the impression each appliance had a default/preset backup-profile tuned to that app ... you are indicating that is not so, right?

so the turnkey aspect of tklbam is more-or-less completely non-function in this "solo" mode, eh? seems silly for the profile to be missing for local use ... since the appliance still has the same functionality.

or does the default profile need to be manually copied from somewhere?!

Thanks for the feedback

-DJ

Jeremy Davis's picture

Apologies if my previous post wasn't as clear as I hoped.

So to provide some more background, by default, the first step is to link TKLBAM to your TurnKey Hub account via 'tklbam-init'. E.g.:

tklbam-init YOUR_HUB_API_KEY

(Where YOUR_HUB_API_KEY is your actual TurnKey Hub API key).

Doing that should do all the required setup, including linking to your TurnKey Hub account and downloading the relevant backup profile for your server. The profile tells TKLBAM what to backup.

If you choose not to link TKLBAM to the Hub, you can use it in 'solo' mode. Doing that you have 2 choices.

The first is to provide a backup profile yourself. To do that you will need to build it first. For more details have a read here:

tklbam-internal create-profile --help

Then armed with a profile, TKLBAM can be initialised like this:

tklbam-init --solo --force-profile=path/to/backup/profile

(Where 'path/to/backup/profile' is the actual path to your generated profile).

The alternative 'solo' option is to not use any profile at all. That is done like this:

tklbam-init --solo --force-profile=empty

Then you will need to explicitly configure what will be backed up via the overrides file (/etc/tklbam/overrides). The default overrides file should include plenty enough documentation:

cat /etc/tklbam/overrides

Re-reading the thread I suspect that you have done the latter but not actually configured it to backup anything yet?!

Hopefully this helps? Please ask for more clarification if needed.

damelco's picture

it seems there is no method of getting an API_KEY without creating a "nexus" with Amazon !

if that IS the case, I wish I would have known TKLBAM is a Amazon product before I got this far down the rabbit hole. what a disappointment.

thanks for your time ... perhaps suggest to the "ptb", they should make that little detail a bit more apparent!

thanks again nontheless.

-DJ

Jeremy Davis's picture

TKLBAM is a open source backup tool created by us; TurnKey Linux! And is NOT an Amazon product! We are a small independent open source collective.

The primary point of TKLBAM is to allow anyone, anywhere in the world to easily create bombproof, secure backups of their TurnKey server(s).

To do that, we need a globally accessible storage provider with incredible reliability and reasonable storage costs. Currently AWS are the front runner in that field. Ideally we would like to support alternate remote storage backends as well, but we've just never got to that.

If you want automated, bombproof, encrypted remote backups, with quick and easy setup, with a web based dashboard to manage your backups, then I would encourage you to sign up to the TurnKey Hub (and link it to an AWS account for storage).

If you are adamant that you do not want to use AWS, then that's fine. As I've noted earlier, you can still use TKLBAM and you have 2 options re configuring it to backup what you want:

  1. You can build a profile (should take no more than 15-20 minutes); or
  2. You can explicitly define what you want backed up via the overrides file.

Also, I'm not really clear on who "ptb" is? And I'm not sure how you ended up trying to use TKLBAM without realising the "easy" path (i.e. via the TurnKey Hub) requires AWS? That's part of the reason that I didn't initially understand your issue. I'd genuinely be interested in understanding how that came to be as you are the first person in my 8 years supporting TKLBAM to get as far as you did without realising that...

Also FWIW TurnKey is Debian under the hood, so whilst it won't be supporting us, you can use any Debian compatible backup product or service you'd like...

damelco's picture

"ptb" is simply "powers-that-be" ... meaning whomever calls the shots.

not sure how you would consider a solution to be open-source if it requires a commercial entity's product in order to function locally (even without the commercial product). .. a bit ironic, don't you think? don't get me wrong, it may be a great option, but clearly not in the true spirit of open-source to require it just to get the needed "default profile", which is needed both locally and remotely, right?

it would seem logical for the default-profile for a given app package could be "part of the initial open-source download" ... or at least downloadable from somewhere "open-sourced" ... without the necessity of engaging in something commercial where it is not needed.

to answer your query ... quiet honestly ... in my time I never ran across an open-source project that required a commercial nexus to complete the open-source solution. so I had no reason to search for such an odd condition?! and since my initial comment made was regarding the confusion as to why it generated only a 8k backup ... quite clearly I expected that "default profile" to be part of the app!

as for an "easy" path ... wouldn't an user's choice to use a local storage option include an "easy" path too?!?! instead of requiring them to have an intimate knowledge of the file structure of multiple pieces of software in order to build an appropriate "default profile" themselves. especially since that file ALREADY EXISTS.

thanks for your assistance, nonetheless.

-DJ

Jeremy Davis's picture

It seems that you have not properly read my previous posts or made any attempts to read any of the documentation I've pointed you towards. If you have some sort of comprehension issues and/or reading/learning disability, I apologise for my frustration, but otherwise, it feels to me that your attitude is overly entitled. Essentially we are giving you a gift and you're saying "that's a crap gift, you should spend more of your time and effort (for free) to give me a better gift"!

The irony is that if you'd spent as much time and effort reading and following the available documentation as you have complaining about the fact it doesn't work how you think it should, you'd have automated backups working already (for free, with no requirement for any commercial services)!

Anyway, let me try one more time...:


First, some response to your post with some more background and context:

TKLBAM is free, open source software; licenced under GPLv3. The term "open source" has a fairly clear and quite specific meaning (which TKLBAM clearly fulfils). "Open source" means that you have full access to the code and can modify it and use it however you like. It doesn't mean that you get to dictate to us how our software works and/or the way which we provide it.

[...] my initial comment made was regarding the confusion as to why it generated only a 8k backup ... quite clearly I expected that "default profile" to be part of the app!

But to get to the point where you found yourself in that position, you would have needed to initialise TKLBAM with the option '--force-profile=empty' (as well as '--solo') which is clearly documented as:

    --force-profile=empty          "empty" is a special profile_id value that creates an empty
                                   backup profile. Backup configurations will only be taken 
                                   from /etc/tklbam.

So if you'd taken a little more care when setting it up, unless you configured what to backup within /etc/tklbam, the 8kb backup should not have come as any surprise...

And if you'd asked nicely about how to generate a profile (instead of using an empty profile) then I would have happily directed you on how to do that. If you had humbly reported difficulties understanding/doing that and/or asked for assistance, I would have happily held your hand, taking the time to give you the exact commands so you could have simply copy/pasted it straight into the terminal.

Anyway, as I've noted on numerous occasions and really should be completely clear by now: YOU DO NOT NEED TO LINK TO AWS!!!! (Or any commercial/third party service).

TKLBAM was designed from the ground up to be used with the TurnKey Hub (which is not open source and does depend on third party commercial services), to direct remote storage and permissions, security key management and distributing the latest backup profiles. The point of providing TKLBAM as open source software is primarily so that anyone can view the code and it can be demonstrated to be trustworthy. It was developed to be "best of breed" backup solution, tailored to our servers, not so it can be used for free.

However, we decided to also support users like yourself; who did not wish to link to a commercial service and/or pay for backups. So we tok the time and effort to support that use case. We have provided 2 different ways that TKLBAM can be used completely independent of any commercial and/or 3rd party services.

Free option #1 that DOES NOT REQUIRE anything commercial:

As noted in the docs, the commandline help and my previous post, you can simply configure what you want to include/exclude from your backup via the "overrides" (/etc/tklbam/overrides) config file. The default file includes tons of documentation which should get you going quickly and easily. I'm also here to assist (again for free) if need be...

Free option #2 that DOES NOT REQUIRE anything commercial:

Generate your own profile using all the details and config that we provide completely free of charge. It shouldn't take more than a few minutes to do. Sure, I get that you would prefer not to have to do anything, but it's really not that hard...


Sorry again for my tone and my frustration, but your attitude (or at least my interpretation of it) really does feel like a slap in the face. Considering that I have poured my heart and soul into TurnKey for 10 years (on a very modest salary) and we give away 90+% of what we do for the love of it, it feels quite confronting to be told that it's not good enough and we should give more away.

damelco's picture

what's with the "FREE" whining ...

FREE and COMMERCIAL are not opposites ... and how dare you gaslight me in assuming "COST" be my motivation ... or that somehow my question is an indicator of the level of my appreciation of your product or that my ability to comprehend is at fault ... who do you think you are?!?!

if the "open-source" arena gives you this much grief, perhaps you need another path in life, brother. but either way you will likely experience less grief when you realize everyone communicates a bit differently on this journey and not assign judgement solely on your "disappointment" with other's manner-of-communication.

but let's bring this to a close. the bottom-line being ... forget I ask ... life is TOO short.

and again ... thanks nonetheless.

regards,

-DJ

Jeremy Davis's picture

It seems that you're reading stuff I didn't write (certainly that I didn't intend)! And conversely I'm clearly not understanding what you're trying to convey either. I like to think that I'm open to constructive criticism and I certainly try to encourage it. But I am aware that I can be a bit precious about TurnKey. Clearly in this instance, I got a bit emotionally triggered and as a result, a bit snarky. For that I apologise. Bottom line, I agree, let's just leave it at that.

Although for the record, I love working in the FOSS arena. Every few years I have a frustrating encounter, but 99.99% of the time it's so awesome. I've met so many awesome people and learned so much and then had the honour of sharing my knowledge with others that are interested to learn.

Despite my frustration at (my interpretation of) your attitude, genuine thanks for trying out TurnKey anyway. And good luck with all your future endeavours.

If you do want to try again sometime and need a hand setting up TKLBAM with a self-generated profile, please ask. ;)

Add new comment