Dehydrated and multiple domains

Andi Northrop's picture

Hi there, I posted this over in the confconsole docs as a comment, but I guess the forums are better monitored? Anywho, to recap:

OK, I may be missing something obvious but I can't see what I should be doing after the first bullet point to make dehydrated pick up the additional domains and run the wrapper to get certificates for those new domains?

Will it just pick up the changes when the daily cron runs or do I need to invoke it myself somehow?

Cheers for any help!

Since then I've figured that I can call dehydrated from the terminal (duh) but I can't get it working.

To start with dehydrated by itself looks for domains.txt (instead of confconsole.domains.txt), a tweak to dehydrated's config file sorts that but now I'm getting a challenge error of 403:

Processing resources.citygatechurch.net
 + Signing domains...
 + Generating private key...
 + Generating signing request...
 + Requesting challenge for resources.citygatechurch.net...
 + Responding to challenge for resources.citygatechurch.net...
ERROR: Challenge is invalid! (returned: invalid) (result: {
  "type": "http-01",
  "status": "invalid",
  "error": {
    "type": "urn:acme:error:unauthorized",
    "detail": "Invalid response from http://resources.citygatechurch.net/.well-known/acme-challenge/Th_2SY-ekF4kPgs1Ea-UHpttUjM_80bifGs_bLcQw-0: \"\u003c!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\"\u003e\n\u003chtml\u003e\u003chead\u003e\n\u003ctitle\u003e404 Not Found\u003c/title\u003e\n\u003c/head\u003e\u003cbody\u003e\n\u003ch1\u003eNot Found\u003c/h1\u003e\n\u003cp\"",
    "status": 403
  },

Any ideas? I'm wary of trying too many times without advice lest I be blocked by letsencrypt.

This all runs fine from Confconsole but I've got new domains I need to add in so I need to get it sorted this way (I'm running 14.2 LAMP).

Cheers!

Andi Northrop's picture

Having rummaged around in the dehydrated docs, my best guess is that I need to add an alias to Apache in the alias.conf mod to the tune of:

Alias /.well-known/acme-challenge /var/www/dehydrated

Am I on the right lines?

Also, if I change dehydrated settings will that break the Confconsole method (not a major issue now but I may wish to use it in the future)?

Andi Northrop's picture

That Alias should probably read:

Alias /.well-known/acme-challenge /var/lib/dehydrated/acme-challenges

 

Jeremy Davis's picture

It's been a little while since I worked on it, but IIRC you should just need to add your additional domains to /etc/dehydrated/confconsole.domains.txt. Each new domain goes on a separate line, with up to 4 sub-domains. E.g.:
example.com www.example.com blog.example.com
example.net www.example.net forums.example.net
domain.com www.domain.com
The last in the list should write to /etc/ssl/private/cert.pem so will also be the cert that will work for Webmin and Webshell.

You can then run it manually (to check it works) by directly invoking the TurnKey dehydrated-wrapper:

/usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper

If you want to do some testing without risking getting blocked by Let's Encrypt, please add the staging server to your conf file (/etc/dehydrated/confconsole.config). Make sure you remove it when you're done testing though.

Want some info about it? Use the --help switch. Want more info about what it's doing? Try the --log-info switch or if you want really verbose output, set DEBUG. I.e.

export DEBUG=y
/usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper
If you use the Confconsole dehydrated-wrapper to launch dehydrated, you won't need to worry about setting aliases etc as it uses it's own mini-webserver to serve the challenges.

If you don't like that long path and want to make dehydrated-wrapper a "command"; then add a symlink to /usr/local/bin:

ln -s /usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper /usr/local/bin/dehydrated-wrapper
Then you should be able to launch it simply by typing "dehydrated-wrapper".

Once it's done it's thing, then you should find your certificates in /etc/ssl/private/DOMAIN, e.g. using my domain list above, I should have:

/etc/ssl/private/example.com/
/etc/ssl/private/example.net/
/etc/ssl/private/domain.com/

TBH, I didn't exhaustively test with multiple domains, so there is a chance that I missed something.

If you want to keep going the way you are, then it's possible, but you'll need to write your own hook script to write out the certificates that it gets. The hook script that ships with confconsole is designed to integrate with the bundled mini-server.

Regardless, it seems clear that we either need to improve the docs a bit (giving a step-by-step walkthrough) and/or make confconsole support multiple domains...

Andi Northrop's picture

Thanks Jeremy, that makes more sense.

It runs now but it's only writing to /etc/ssl/private/cert.pem meaning that only the domain (and subs) on the last line of confconsole.domains.txt is getting a certificate.

It's not writing any of the extra domains to directories e.g. /etc/ssl/private/example.com/

/var/lib/dehydrated/certs/ contains directories for each domain but nothing in /etc/ssl/private

Andi Northrop's picture

Hi Jeremy, just wondering if there's been any progress on this?

Cheers, Andi

Jeremy Davis's picture

Sorry for the radio silence. I've been tied up trying to finalise v14.2 and get the basics in place for v15.0. I'm pretty close, but still some minor issues.

I've fairly sure that this should be a pretty easy fix. I just need to put an hour or 2 aside and sit down with it.

I can't promise, but I'll try to get you something to test this week.

Please do not hesitate to bump this if your curious how I'm going.

Jeremy Davis's picture

Deep apologies on the delay on this.

But you know what?! It works as expected for me!?

Reading back through your posts, the issue is the path that you're looking for the certificates in! I'm really sorry that I didn't spot that sooner. As per the docs, please check /var/lib/dehydrated/certs/ for the specific per domain certificates. I'll replay what I did to confirm it's working as expected:

Double check DNS is configured right (from my desktop):

user@ninjux ~$ dig jeremydavis.org lamp.jeremydavis.org lamp-tester.tklapp.com +short
52.65.87.99
52.65.87.99
52.65.87.99

That looks good! So ready to run the dehydrated-wrapper on my lamp server. First, my confconsole.domains.txt:

root@lamp ~# cat /etc/dehydrated/confconsole.domains.txt

# please use this file with confconsole or
# alternatively use dehydrated with it's appropriate
# configuration directly
jeremydavis.org lamp.jeremydavis.org
lamp-tester.tklapp.com

Now to run dehydrated. You shouldn't need the --force switch, but I do as I've already had this working and want to demonstrate the full process:

root@lamp ~# dehydrated-wrapper --force
[2017-11-28 23:22:00] dehydrated-wrapper: INFO: started
[2017-11-28 23:22:00] dehydrated-wrapper: INFO: found apache2 listening on port 80
[2017-11-28 23:22:00] dehydrated-wrapper: INFO: stopping apache2
[2017-11-28 23:22:01] dehydrated-wrapper: INFO: running dehydrated
[2017-11-28 23:22:13] confconsole.hook.sh: SUCCESS: Cert request successful. Writing cert.pem & cert.key for jeremydavis.org to /etc/ssl/private
[2017-11-28 23:22:19] confconsole.hook.sh: SUCCESS: Cert request successful. Writing cert.pem & cert.key for lamp-tester.tklapp.com to /etc/ssl/private
[2017-11-28 23:22:19] dehydrated-wrapper: INFO: dehydrated complete
[2017-11-28 23:22:19] dehydrated-wrapper: INFO: Cleaning backup cert & key
[2017-11-28 23:22:19] dehydrated-wrapper: INFO: starting apache2
[2017-11-28 23:22:20] dehydrated-wrapper: INFO: starting stunnel4
[2017-11-28 23:22:20] dehydrated-wrapper: INFO: dehydrated-wrapper completed successfully.
Now check in /var/lib/dehydrated/certs for the certificates:
root@lamp ~# tree /var/lib/dehydrated/certs
/var/lib/dehydrated/certs
|-- jeremydavis.org
|   |-- cert-1511911322.csr
|   |-- cert-1511911322.pem
|   |-- cert.csr -> cert-1511911322.csr
|   |-- cert.pem -> cert-1511911322.pem
|   |-- chain-1511911322.pem
|   |-- chain.pem -> chain-1511911322.pem
|   |-- fullchain-1511911322.pem
|   |-- fullchain.pem -> fullchain-1511911322.pem
|   |-- privkey-1511911322.pem
|   `-- privkey.pem -> privkey-1511911322.pem
`-- lamp-tester.tklapp.com
    |-- cert-1511911333.csr
    |-- cert-1511911333.pem
    |-- cert.csr -> cert-1511911333.csr
    |-- cert.pem -> cert-1511911333.pem
    |-- chain-1511911333.pem
    |-- chain.pem -> chain-1511911333.pem
    |-- fullchain-1511911333.pem
    |-- fullchain.pem -> fullchain-1511911333.pem
    |-- privkey-1511911333.pem
    `-- privkey.pem -> privkey-1511911333.pem

2 directories, 20 files
Looks like everything is in there. So all should be well. Although you'll still need to configure the individual webserver sites to use the relevant certificates. We haven't automated that because it will depend on the webserver being used and how you are configuring the sites. But it should be relatively straight forward if you understand how that works (it's is after all "advanced" functionality of confconsole).

Rereading the previous conversation again, I think what introduced your confusion is the log message "Writing cert.pem & cert.key for xyz.com to /etc/ssl/private".

FWIW, the docs do mention that the last cert to be written will be written there (and it is). Whats actually happening is that each cert is being written there, so each one overwrites the previous. In retrospect, I perhaps should have considered how I might make the log messages a bit clearer when multiple domains are used.

TBH, it was a last minute addition to support more complex use cases, and it wasn't implemented as well as it could have been. Also as I've noted, the docs aren't as clear as they should be. I'll spend a little more time on this and try to tidy it up a bit.

If you have any specific feedback on might be best, please let me know.

Jeremy Davis's picture

Let's Encrypt have updated their user agreement, and for it to work, it needs to be pointing to the latest version of that. The fix is easy, have a look here.

I'll have a look at improving the log messages and we'll hopefully be updating confconsole soon.

Jeremy Davis's picture

If you don't want to have them set as separate webserver sites (i.e. virtual hosts) then you can actually create a single certificate for all domains and subdomains! All you need to do is put them all on the one line of confconsole.domains.txt. E.g.:
root@lamp ~# cat /etc/dehydrated/confconsole.domains.txt

# please use this file with confconsole or
# alternatively use dehydrated with it's appropriate
# configuration directly
jeremydavis.org lamp.jeremydavis.org lamp-tester.tklapp.com
root@lamp ~# dehydrated-wrapper --force
[2017-11-28 23:50:23] dehydrated-wrapper: INFO: started
[2017-11-28 23:50:23] dehydrated-wrapper: INFO: found apache2 listening on port 80
[2017-11-28 23:50:23] dehydrated-wrapper: INFO: stopping apache2
[2017-11-28 23:50:25] dehydrated-wrapper: INFO: running dehydrated
[2017-11-28 23:50:41] confconsole.hook.sh: SUCCESS: Cert request successful. Writing cert.pem & cert.key for jeremydavis.org to /etc/ssl/private
[2017-11-28 23:50:41] dehydrated-wrapper: INFO: dehydrated complete
[2017-11-28 23:50:41] dehydrated-wrapper: INFO: Cleaning backup cert & key
[2017-11-28 23:50:41] dehydrated-wrapper: INFO: starting apache2
[2017-11-28 23:50:42] dehydrated-wrapper: INFO: starting stunnel4
[2017-11-28 23:50:42] dehydrated-wrapper: INFO: dehydrated-wrapper completed successfully.
And here is the certs directory (but this time they're all written to /etc/ssl/private). Note that I deleted all the directories that were there initially so this is new for this run of dehyrdated-wrapper.
root@lamp ~# tree /var/lib/dehydrated/certs #
/var/lib/dehydrated/certs
`-- jeremydavis.org
    |-- cert-1511913025.csr
    |-- cert-1511913025.pem
    |-- cert.csr -> cert-1511913025.csr
    |-- cert.pem -> cert-1511913025.pem
    |-- chain-1511913025.pem
    |-- chain.pem -> chain-1511913025.pem
    |-- fullchain-1511913025.pem
    |-- fullchain.pem -> fullchain-1511913025.pem
    |-- privkey-1511913025.pem
    `-- privkey.pem -> privkey-1511913025.pem

1 directory, 10 files
Andi Northrop's picture

Thanks Jeremy, I haven't had time to look at this today but thanks for digging into it. Hopefully this sorts us out!

Jeremy Davis's picture

You're most welcome.

FWIW I'll be looking to refactor the confconsole code to allow users to create a single certificate with up to 5 separate domains (be they sub domains, or completely separate domains). I'm not sure how long it will take, but I plan to work on that over the coming week or so. Unfortunately you may need top make do with what we have for now, but hopefully we'll have something better soon!

You will still need to manually adjust the confconsole.domains.txt if you want multiple certificates. But hopefully that will make it more useful. There's generally no need to create separate certificates, even if you are serving the sites completely separately. I.e. by using named based virtual hosts.

I have the certs generated but not sure which of the file si ma supposed to point to ?

 

ssl_certificate /var/lib/dehydrated/certs/ydexchange.youdopet.com/cert.pem;
        ssl_certificate_key /var/lib/dehydrated/certs/ydexchange.youdopet.com/privkey.pem;

Is this correct

Thanks


Jeremy Davis's picture

Although TBH, I'm not sure OTTOMH. The first test is if it works! :)

Although actually, I just did a quick google and apparently it should be like this:

ssl_certificate /usr/local/etc/dehydrated/certs/example.com/fullchain.pem;

ssl_certificate_key /usr/local/etc/dehydrated/certs/example.com/privkey.pem;
Hope that helps.

Post new comment