You are here
Hello,
So I powered a Magento2 Turnkey VM back up after a few months of it being offline, and new I'd need to renew its certificate.
I ran the following:
/usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper
and got:
--------------
[2019-10-01 09:07:43] dehydrated-wrapper: INFO: started
[2019-10-01 09:07:44] dehydrated-wrapper: INFO: found apache2 listening on port 80
[2019-10-01 09:07:44] dehydrated-wrapper: INFO: stopping apache2
[2019-10-01 09:07:44] dehydrated-wrapper: INFO: running dehydrated
+ ERROR: An error occurred while sending post-request to https://acme-v01.api.letsencrypt.org/acme/new-authz (Status 400)
Details:
{
"type": "urn:acme:error:badNonce",
"detail": "JWS has no anti-replay nonce",
"status": 400
}
[2019-10-01 09:07:46] dehydrated-wrapper: FATAL: dehydrated exited with a non-zero exit code.
[2019-10-01 09:07:46] dehydrated-wrapper: WARNING: Something went wrong, restoring original cert & key.
[2019-10-01 09:07:46] dehydrated-wrapper: INFO: starting apache2
[2019-10-01 09:07:46] dehydrated-wrapper: INFO: starting stunnel4
[2019-10-01 09:07:47] dehydrated-wrapper: WARNING: Check today's previous log entries for details of error.
------------------
From what i can find while googling, it seems as though the dehydrated script isn't handling the nonce sequence properly.
Any idea on how to fix this?
Hey Ortho
Yes, it appears that Let's Encrypt have changed their server config and the change has broken the older version of Dehydrated that we have been using (installed from Debian repos).
There is some discussion regarding this (and some other issues) in another thread (specifically this post and this one). But probably your best bet is to have a read of the full info in the relevant issue on GitHub.
FWIW, I have just updated the issue with a link to the relevant Debian bug and a note that it appears to be possible to just edit the Dehydrated script itself to resolve the issue. I haven't tested that myself, but I figured it was worth noting.
Regardless, I should probably do a blog post (and send out a newsletter) about this as it will hit all users eventually (as their certificates expire). I'll be speaking with Alon tonight (my time) and we'll decide on a plan of attack...
So I tried...
I tried both changes, and updating dehydrated did seem to work after a reboot.
Unfortunately I've seemed to have run into an LE Rate limit (I hate how it doesn't tell you which limit you've hit).
I tried then to change the dehydrated url to the test url, but now I always just get 400 error.
Is there a way to run the dehydrated wrapper but in test mode just to validate that everything is working once the rate limit clears?
Bugger... :(
AFAIK the Let's Encrypt rate limit is 20 per week.
It's not well documented, but Confconsole actually ships with a alternate config designed to be used against the "Let's Encrypt staging server". It was created with the intention of being "dropped in" instead of the default config (it overrides the default URL that it hits, so just adding the relevant lines in the config file is another option). You can find a note about it within a discussion on GitHub.
Having said that, it was a fair while ago since it was last tested (at least by me). So I'm not even 100% sure that it still works. It'd be great if you wanted to test it out and let us know...
It might be best to save the current config, just in case. I.e.:
Then give it a run:
Good luck mate.
And I have got new error...
Hmmm that is a new one... Looks like updating Dehydrated is reqd
Ah yes, that is a new error. Reading the error message suggests that we'll need to update the API URL that Dehydrated uses. According to a Let's Encrypt announcement disallowing new registrations for APIv1 wasn't supposed to happen until next month. Existing users have until June 2020 apparently, although this early breakage of v1 suggests that it's probably best not to wait...
I documented on the bug report how to install from upstream (it's a bit dirty, but in this case it's not "unsafe"). Not sure if you've done that, or the just "fixed" the single line (i.e. adjusting the single line in Dehydrated)?
Regardless, either of those will resolve the Nonce issue, but without further changes, it will still attempt to use the v1 API. I haven't done any testing yet, but a quick google suggests that we'll need to adjust the URL that Dehydrated connects to (stored in the CA variable). Currently it is:
For new users, that now needs to be:
It's probably also worth existing users changing that too and double checking that everything works as it should.
It's probably worth noting that there is now a newer version of Dehydrated in stretch-backports now (actually, it's the latest version). So installing from backports is another (cleaner) way to upgrade it.
As noted above, I'm yet to do any testing to confirm my understanding, but essentially, these steps should resolve the issue(s):
And then everything should be all systems go!
I'll aim to look into this further ASAP (within the next few days) but I'm currently a bit bogged down with other stuff.
PS - I hope you don't mind me editing your post to make it a bit easier to read... :)
I solved the problem this way:
Great, thanks Igor!
Thanks for the neat and tidy roundup Igor, that's great!
Although FWIW, you don't need certbot. Dehydrated does pretty much the same thing (i.e. they're both ACME clients which get certificates from Let's Encrypt).
Links in /etc/dehydrated / conf console.config
Yes add them in
Yes add them in.
FWIW Dehydrated has built in defaults for a number of settings, so adding new values (such as CA and CA_TERMS) in the config file will override the builtin defaults.
Ryan has noted a (possible) short term workaround for #1360
Ryan (via support) has just provided me with a (possible) short term workaround for the issue related to failures with multiple domains configured. I have posted it to the relevant GH issue, but will give a brief overview here.
Essentially he suggests doing the update (as noted above by Igor or on issue #1359). Once that has been configured, reduce the domains that you are requesting certs for to one and run the wrapper script. Then re-add the other domains and re-run the wrapper script.
I haven't tested it myself, but thought it worth sharing. If anyone else tests this out, please post back with feedback.
Unfortunately, I still haven't had a chance to complete the work I've started to provide a "proper" fix for #1360, but fingers crossed, I'll have the final piece for that soon. If anyone is willing and able to help out with that, please let me know.
new error after updating dehydrated
Perhaps try clearing Dehydrated's data?
TBH, I'm not 100% sure, but I think that perhaps it's the switch from the v1 API to the v2 API. I think it's worth trying to clear Dehydrated's data. I.e. try this:
And then retry:
If that still doesn't work, could you please show me the contents of your config file:
got one step further
It looks like you don't have the updated hook script.
Judging from the error message "this_hookscript_is_broken__dehydrated_is_working_fine__please_ignore_unknown_hooks_in_your_script", I'm guessing that you haven't updated the hook script. To do that (as per step 2 of the workaround noted on the issue):
Hopefully that should get you up and running...
Hi all, there's now a "proper" fix! :)
Please note that I've just published Confconsole v1.1.1. It's not yet available from our repos (although will be soon) and still requires some specific steps to install and set up on a v15.x server (although better than instructions published previously).
Please note that users who have already updated via various other means are still recommended to install this update as it includes reliability fixes for add-water; our custom challenge mini-server. Please see the release notes for full step by step setup and further info - instructions cover both new and existing users.
Any issues, please ask. Any feedback (e.g. anything that isn't clear, etc). Please ask.
Heads up on an issue with the update...
It's been bought to my attention that after installing the v1.1.1 Confconsole update, the add-water service is being inadvertently enabled. That means that on reboot, it will start up and will likely block Apache (or other webserver) from starting!
The fix is easy:
Please note that on any server where you have already run the v1.1.1 update, you need to apply the above line and there is no value in updating to the newer package. For any new servers you launch though (or for anyone else who hasn't applied any fixes and stumbles across this thread) the newer v1.1.2 release resolves this issue (it's exactly that same as v1.1.1, but doesn't auto enable the add-water service on install).
Add new comment