You are here
Good day,
I'm trying to migrate a DokuWiki 11.3 virtual machine to a 12.0 one.
- 12.0 installs and updates fine, links to the hub and everything.
- I have a full backup of the 11.3 machine stored to local filesystem, transferred to the new VM, and restored there.
- TKLBAM says the restore went fine, and I could see our DokuWiki files fly by in the terminal in the process
- TKLBAM suggests I reboot the machine
Here's where the problem begins: Despite the fact that the machine was completely and properly initialized before, the appliance goes into first boot mode, asking me to set root and admin passwords, wanting to initialize TKLBAM, etc., etc. (if I try the latter, it complains that's already been done).
Even if I do go through the motions again, the previously working -if empty- DokuWiki now says
DokuWiki Setup Error
The datadir ('pages') does not exist, isn't accessible or writable. You should check your config and permission settings. Or maybe you want to run the installer?
This is also reproducible. After the first target machine was trashed by this procedure, I deleted it, loaded a new one, and made sure to reboot after each step to check if the machine was still working properly. (A suggestion from the last post of this previous thread, which details a similar issue.) The same procedure trashed the machine again, but this time I had made snapshots during installation/sys-upgrade, so I could jump back to the working machine before the backup.
Did that, did the restore again, lo and behold: It's broken again.
I would really appreciate advice, a patch or anything to get this fixed, especially since the 11.3 one now has its own issue to deal with, so I can't run it much longer. (Closing that one as "Fix Released" without ensuring the migration path worked properly is just bad style.)
Lastly, the machines in question are VMs in a VMWare vSphere, created from the provided OVF files. The 11.3 one has been running fine for months, the 12.0 one start and initializes fine, it's just the restore that kills it.
So I'm ruling out evil influence by the environment, broken installation media or similar issues.
Thank you for your time
Carl Kiendl
A couple of things...
Firstly, the v11 bug you link to is an upstream Ubuntu bug, not specifically a TKL bug. As such it is completely out of the control of the devs. There's only 2 of them after all - and if Ubuntu with it's millions of dollars of resources and it's massive community of volunteers and enthusiasts can't/won't fix it then I think it's asking a bit much for 2 TKL guys basically working for free to resolve it. Besides, the workaround is clearly spelt out in the forum thread where I first posted about it and if you don't update udev then it shouldn't ever be a problem anyway...[/end rant]
Ok now I've got that out of my system, remember it has been stated that tweaks may be required to some appliances to successfully migrate from one version of TKL to the next. It looks like DocuWiki is one of these...
My first guess from the error message is permissions - either file permissions or DB permissions. Although the fact that you needed to rerun the init scripts makes me think perhaps DocuWiki is unable to contact the DB at all (assuming it has a DB backend that is - I know nothing about DocuWiki).
Have you tried clicking the 'run the installer' link to see what that says? It may give you some hints on exactly what has/is going wrong. (I suspect it will give you an error message of some sort that will be a bit more helpful. Another place that may be helpful in working out what you issue is is the Apache log(s), you can find them in /var/log/apache2 IIRC.
The other possibility that strikes me is that perhaps DocuWiki have changed their DB schema between the versions included in v11 vs v12 (again assuming that it has a DB backend). If that's the case, then upstream software providers often have a script or explanation of steps required when updating from one version of their software to another. If you don't know what the relevant versions are then you can checl via the TKL appliance changelog (link is on the appliance page).
Good luck! And be great if you can post back regardless of how things go.
Re: A couple of things...
I wasn't implying you were necessarily supposed to fix the bug yourself, but marking it as fixed while it clearly isn't, under the pretense that 12.0 works, sounds like mockery to those of us still running 11.3 who can't upgrade (be it for technical or procedural reasons), or just plain didn't get around to it yet.
Yes, certainly, I can search half the sector to research what I can do on my own, but would it have been that hard to post clear instructions of what one can do in the same post closing the issue?
Or at least say something like "the post linked in comment #foo contains a workaround"?
If you already know what can and can't be done, why don't you make it easy to find the answer?
Why is it necessary that every single user has to restart researching all over again?
It's not about additional work, it's about the exact opposite: Make it easy for everyone to find the proper way to deal with this. That's a one time thing, with minimum effort if you already know how to do it, helping every single user encountering this issue.
Though, that being said, if you know about the issue, you could have released a patch simply doing exactly that: Preventing udev from being upgraded and downgrading it if necessary. No need to fix udev yourself.
Instead, every updating user runs into the issue, perpetuating the problem...
[/end counter-rant :P]
DokuWiki doesn't have a DB, it runs on flat files (hence the reference to a "datadir" named "pages"). Therefore, I can rule out missing databases and schema-changes.
After some more digging into this, I realize I misinterpreted the statement "Upgraded to latest upstream package.".
I interpreted "upstream" as "the source of DokuWiki". I realize now it refers to Debian.
Which means the provided version of DokuWiki is three years old, and the exact same one as in 11.3.
Our own installation is more or less current, I assume this could be the source of the issue. I'll check if upgrading the stock appliance's DokuWiki before migration helps.
I know the joke goes that Debian comes in "Rusty", "Stale" and "Ancient", but a three-year-old version is pretty depressing...if that turns out to be the issue, I'll check if I have the time to generate a tklpatch. No promises, though. I only ever used that thing once to update Redmine.
A couple more things...
The terms can be a little confusing and I guess in some sense the labelling was not strictly 100% true. Although it could be argued that the fix that was 'commited' (meaning the new version doesn't have the bug but it remains in this version) was 'released' (ie this version doesn't have the bug) with the release of v12 (Debian based appliances).
FYI there are at least 3 links to the forum thread (which includes the workaround) posted in the bug report (one of which points directly to the forum post with the workaround spelt out). Also the thread is the first result when searching 'udev' here in the forums (top right) and the second result when searching 'udev error tkl' on google (the first result when searching 'udev error workaround tkl'). A number of different variations of the workaround are within that thread (at least 3 posts by me that have some version of it, including an effort to consolidate the knowledge and repost once it got burried). That's hardly searching "half the sector"...! If you'd taken 2 minutes to quickly scan the bug report posts or 10 minutes to read the thread you would have all that knowledge...
Yeah sure I could of probably done more, but there are a million other things that I could've done around here too... Besides, I just volunteer! I also have a full-time job, a family, a house that needs renovating and beyond that I try to have a life too!! As it is I spend a heap of my time of time here trying to help others out and trying to make things easier. Perhaps I'm being overly touchy, but I must say that it pisses me when someone blows in (who from what I gather has contributed nothing to TKL) and says how it should be done!! Well perhaps it should be done the way you think, so perhaps you should roll up your sleeves and do it!?! [/end counter-counter-rant]
Ok thanks for the info re DocuWiki. So as you say, that obviously rules out DB issues... So my next guess is file permissions. Like I said I'd try hitting install.php and see what happens.
It looks like the devs have made a mistake when updating the DocuWiki changelog. That shouldn't read 'upstream' it should read 'package management' (as it does on the appliance page). When they use the term upstream, they mean that they installed it from upstream source (as you would expect).
It is no great surprise that the version in Debian package management is the same as Ubuntu 10.04. Ubuntu 10.04 is based on a snapshot of Debian 6 (IIRC while it was 'frozen' in testing, or not long before it was frozen) so most of the packes will be the same, it's just that Debian had an additional 4-5 mths of testing and bug fixing prior to stable release. I can only assume that when deciding what to do, the devs felt that the added features to the upstream version weren't enough to offset the security and stability gained by using the Debian repo version. Sometimes the trade off is worth it, but other times not and that is always going to be a somewhat subjective decision.
Sorry if I took it all a bit personally
Just wanted to apologise if I have taken your critism of TKL a bit too personally. I think that your criteque is not completely unreasonable but for some reason the way that you put it pushed my buttons a bit...
Upgrades and stuff
I'm the same way when it comes to my projects, no offense taken. I know the feeling of users doing jack shit to help. ;)
I originally wrote the rest of this post yesterday, but the forum corruption ate it, so here it is again:
Alright, since I figured out that the current appliance's DokuWiki is on Debian's three-year-old version (0.0.20091225c-10+squeeze2/2009-12-25c "Lemming") and I had updated our local DokuWiki in the meantime, I can't rule out my migration issues are related. It was my understanding that TKLBAM should include all changed files (that's the whole point of a backup, after all), but, as said, I can't rule it out.
In order to avoid this problem in the future (or at least make it easier to handle), I've researched how to upgrade DokuWiki in a more migration-friendly way, and have resorted to selectively pulling the dokuwiki package from Debian's testing repositories. While TKLBAM will likely not automatically set this up on migration, it'll be easier to upgrade DokuWiki like this on a new version of the appliance and then migrate than constantly manually upgrading DokuWiki after each release and never being able to use TKLBAM. Since the DokuWiki-package-setup on the new appliance would be equal to the one installed currently, a future migration should run smoothly.
Testing currently doesn't contain the latest DokuWiki release 2012-10-13 "Adora Belle", but it does, at least, contain 2012-01-25a "Angua", and there's hope that testing will at least get updated at some point.
I am not claiming this to be the sanest or best way. But it is the way I'm using right now, and it at least seems like Angua is running properly (in its empty state).
So here are the instructions for anyone else who wants to attempt this:
Obviously, as the very first thing, make backups. As many of them as necessary to ensure you can restore the full machine to its current state. Afterwards...
As root in a terminal of your choice (SSH, web-SSH):
That line adds testing to your repositories. Do not trigger upgrades or installations at this point.
Note that that is the German mirror; replace the "de" subdomain with your own country-code, e.g. deb http://ftp.us.debian.org/debian testing main, or something geographically close.
Next, do
and append the following content to the file:
(Hint: Ctrl-O saves, Ctrl-X quits)
This tells apt to give very low priority to testing in general, but to use testing's DokuWiki-package if it's newer than the one installed.
Now you can use apt again. Do an
Next, issue the following commands:
Both commands should show you new package versions from testing in the version table, like this:
root@dokuwiki ~# apt-cache policy php5 php5: Installed: 5.3.3-7+squeeze14 Candidate: 5.3.3-7+squeeze14 Version table: 5.4.4-7 0 1 http://ftp.de.debian.org/debian/ testing/main i386 Packages *** 5.3.3-7+squeeze14 0 500 http://security.debian.org/ squeeze/updates/main i386 Packages 500 http://ftp.debian.org/debian/ squeeze/main i386 Packages 100 /var/lib/dpkg/status root@dokuwiki ~# apt-cache policy dokuwiki dokuwiki: Installed: 0.0.20091225c-10+squeeze2 Candidate: 0.0.20120125b-1 Package pin: 0.0.20120125b-1 Version table: 0.0.20120125b-1 500 1 http://ftp.de.debian.org/debian/ testing/main i386 Packages *** 0.0.20091225c-10+squeeze2 500 500 http://ftp.debian.org/debian/ squeeze/main i386 Packages 100 /var/lib/dpkg/statusIf everything went as we want, there will be one significant difference: The Candidate version.
php5's candidate should be equal to the version provided by the squeeze/stable repositories.
dokuwiki's candidate should be equal to the version provided by testing.
If php5's candidate version is the testing one, do not upgrade or install new packages. That's a sign that your pinning/priority settings are incorrect. Go back to the /etc/apt/preferences part and make sure you transferred the settings correctly.
If the Candidate situation is as expected, you can now run
apt-get install dokuwiki
it will install a few new dependencies and then upgrade dokuwiki.
In the process, the Package Configuration will pop up:
When done, edit /etc/dokuwiki/htaccess [sic] and restore those settings of your old .htaccess you need.
should now show
dokuwiki: Installed: 0.0.20120125b-1 Candidate: 0.0.20120125b-1 Package pin: 0.0.20120125b-1 Version table: *** 0.0.20120125b-1 500 1 http://ftp.de.debian.org/debian/ testing/main i386 Packages 100 /var/lib/dpkg/status 0.0.20091225c-10+squeeze2 500 500 http://ftp.debian.org/debian/ squeeze/main i386 Packages 2012-01-25b "Angua"Lastly, do an
and check your installation.
Disclaimer: I have only tested this on an "empty" appliance. I make no guarantees for already content-filled wikis. Try at your own risk and make backups.
Next, I'll detail how to migrate your existing wiki data from the old appliance to the new one.
Thanks Carl, nice work!
Awesome post and I'm sure others will find it of use. TBH that would have to be one of the clearest and most straightforward explanations of pinning that I have ever seen! So bonus points right there! :)
I agree that who knows what might happen if you applied that to a DocuWiki appliance that already contained data. However my suspicion is that it would be ok. Debian has a reputation of providing a fairly smooth upgrade between versions (as opposed to Ubuntu which I have only done once and NEVER again! And that wasn't even between LTS versions! It would've been quicker and easier to start from scratch than fix the mess I ended up with...) Having said that there is risk that the new package may break things for your existing data (and may break things even after this if you again upgrade to the next version that apears in testing). AFAIK it is only once testing is frozen ready for the next stable release that a concerted effort is made to support existing data from the stable version to the version in testing. But perhaps I'm just being a bit paranoid. I know that some people use testing as there OS of choice and not sure if they still do but AFAIK Linux Mint Debian uses it as a base (although I think they maintain their own repo as well so they can fix stuff that breaks ASAP).
On TKLBAM: It does backup files that have changed (as one would expect) however it also plays strictly by the Debian packaging rules. This means that anything that is included in 'naughty' places (ie areas of the file system that 'should' only be touched by package management and not by end users) will be ignored.
OTTOMH there are a few ways you could work around this:
1) Instead of using the Dokuwiki appliance you could use an alternative appliance (such as Core or LAMP) and install the software you need (including from upstream). As long as you put all your upstream files in places handled by TKLBAM (eg /var/www) then you'll have no issues. TKLBAM is even smart enough to install packages that aren't included by default (eg Apache on Core). Although it will only install from the default repos (TKL, Debian stable main and security), so anything from non standard repos (eg packages from Debian testing) will be missed out. Doing it this way you will have a larger backup because all of the DocuWiki files will be backed up (ie the full contents of /var/www) - not just the relevant data files. In the grand scheme of things though it's probably still not going to be a huge backup and you'll have the convienience of your DocuWiki being in exactly the same state as it was previously (ie same upstream version).
2) Another possibility would be to be 'naughty' and simply overrule TKLBAM's choices by adding an inclusion and force it to include the 'naughty' places in the backup.
3) Possibly the best of both worlds would be to use a TKLBAM hook to enable testing and update DocuWiki prior to addition of your data.
Something else to keep in mind is that packages in testing don't always get timely security updates. In fact they rarely ever get security fixes backported like stable does. Usually they'll just get the next version of the software that doesn't incude the security bug (once it has passed through Sid - min 2 weeks AFAIK). I recall reading of packages with security bugs sitting unfixed in testing for months.
So I would only recommend the first method in a production environment. I guess 2 or 3 would be ok if it wasn't running on the open internet and you were the only one maintaining it (and/or the data is backed up and non-sensitive).
Don't get me wrong I'm not trying to poo-poo your suggestions. I really appreciate it and personally if I was running DocuWiki I'd probably do what you've done regardless of the points I've raised. But I do think that it is important to make educated decisions with an understanding of what future implications there may be.
Add new comment