Confconsole - Let's Encrypt

Overview

Confconsole Let's Encrypt plugin provides a simple way to get a free legitimate CA signed TLS/SSL certificate via Let's Encrypt. It supports both the HTTP-01 and DNS-01 "challenge" methods to validate your control of the (sub)domain(s) that you are registering.

HTTP-01 challenges are hosted by our custom mini webserver via port 80. If port 80 is already in use, the default webserver is temporarily stopped. DNS-01 challenges are fulfilled by temporarily setting a TXT DNS record via a supported DNS provider's API.

By default Confconsole supports registering a domain with up to 4 subdomains, in a single certificate which the existing webserver will use on all https ports. So Adminer, etc will also use the same certificate, as will Webmin (via it's built in miniserver). Wildcard certificates are also supported, but they require DNS-01 validation (not compatible with HTTP-01).

Beyond initial setup via Confconsole, no additional configuration of your server is generally required. Common usage scenarios should "just work". More uncommon and/or complex ones are still possible, but may require additional configuration.

Confconsole - Let's Encrypt

By default, Confconsole should "just work" with these servers/webservers:

  • Apache
  • LigHTTPd
  • Nginx
  • Tomcat
  • Webmin

If you wish to see support for alternate servers, please post on the forums, and/or open a issue on our tracker if you're open to contributing code, that's warmly welcomed.

Cert auto renew

Selecting this option makes the default TLS/SSL certificate renewal cron job (/etc/cron.daily/confconsole-dehydrated) executable (or not). Only executable files within /etc/cron.daily are triggered automatically by cron.

Note: Until you get your initial certificate (which also configures dehydrated), the cron job doesn't exist in the cron.daily directory. This ensures that the cron job can't be enabled until the dehydrated-wrapper has been run (and hopefully a Let's Encrypt TLS/SSL cert has been generated).

Note: To generate wildcard certificates with the TurnKey Let's Encrypt integration, you will need to use DNS-01 validation. DNS-01 validation is not available prior to Confconsole v2.1 (first appears in TurnKey v18.0).

For more info about what the cron job actually does, please see Cron job details below.

Get certificate

Selecting this option will allow you to set a single fully qualified domain name (FQDN) for you server.

Another option is to set a root domain and up to 4 separate subdomains.

If you wish to set more than one root domain and/or more than 4 subdomains, manual configuration is required. Please see Advanced - usage with multiple domain names below. If you have another scenario not covered, please sign up and post on the forums.

Getting a certificate - HTTP-01

This option will validate your ownership of the domain(s) via looking up public DNS records of the domain(s) and checking that the appropriate challenge data is being served via port 80 of your server.

Requirements:

  • Port 80: port 80 (default vanilla HTTP port) must be open on your server and publicly available
  • DNS record(s): authoritative 'A'/'AAAA' or 'CNAME' record(s) which resolve to your server (for all domains/subdomains you are getting certs for)

Port 80: Beyond the port being externally accessible, no changes to existing web server config are required. E.g. web server config redirecting all traffic from HTTP to HTTP will be ignored, so does not need to be changed.

DNS record(s): If you have just made the required changes with your DNS registrar, please be aware that it can take time for DNS records to publicly propagate. To double check that your DNS records are publicly available, please check via 'dig'. E.g. Google's free online Dig tool.

Important Note: Please ensure that you have your Domain nameservers correctly configured and port 80 is publicly available prior to running this. Failure to do so will cause the Let's Encrypt challenges to fail (so you won't get a certificate). Repeated failures may cause your server to be blocked (for a week) from further attempts.

Getting a certificate - DNS-01

This option will validate your ownership of the domain(s) via looking up a special public DNS TXT record of the root domain and ensuring that it contains the appropriate challenge data. No external access is required to your server, so this option is ideal for internal/intranet servers. This option also supports wildcard certificates (e.g. '*.example.com').

Requirements:

  • Supported DNS provider: your domain must be registered with a supported provider
  • Credentials: User credentials and/or API key (as relevant) for accessing your DNS provider's API

Supported DNS provider: The required TXT DNS record needs to be created with your DNS registrar. Confconsole leverages Dehydrated which in turn leverages dns-lexicon to created the required TXT DNS record. As such, your domain needs to be registered with a supported provider.

Credentials: To create the required DNS record, access to your DNS provider (via API) is required. Each provider has different requirements, so please consult the dns-lexicon docs. On that doc page, find your provider and click the link. That should take you to the relevant section of that page. The relevant fields are formatted as dot points, with the key noted in monospace font, colored orange, with an explanation after.

Note that the config file may contain comments (lines starting with a '#' prefix) which will be ignored. See a few config examples below:

AWS Route53: (lexicon docs)

# AWS Route53 example
auth_access_key: YOUR_AWS_ACCESS_KEY
auth_access_secret: YOUR_AWS_ACCESS_SECRET
private_zone: False # generally you'll always want public zone
zone_id: YOUR_ZONE_ID

Cloudflare: (lexicon docs)

# Cloudflare example 1 - using global api key
auth_username: YOUR_CF_USERNAME
auth_token: YOUR_CF_API_TOKEN

Or

# Cloudflare example 2 - using unscoped API token
auth_token: YOUR_CF_UNSCOPED_API_TOKEN

Or

# Cloudflare example 3 - using scoped API token
auth_token: YOUR_CF_SCOPED_API_TOKEN
zone_id: YOUR_CF_ZONE_ID

For other DNS providers, please consult the relevant docs by clicking your provider in the list at the top of the DNS Lexicon provider doc page.

Special note re Wildcard certificates

Please note that generating a Wildcard certificate is only available when using the DNS-01 challenge method.

A wildcard certificate, is a special certificate that is generated for a domain that starts with an asterisk: '*', e.g. '*.example.com'. This special certificate that will match any subdomain (within limits). E.g. '*.example.com' will cover sites with domains such as 'www.example.com', 'blog.example.com' and 'docs.example.com'. However the same cert won't work with sub-sub-domains, e.g. a '*.example.com' certificate won't work with 'www.docs.example.com'. For 'www.docs.example.com' to work with a wildcard cert, you'd need to get a cert for '*.docs.example.com'.

It is possible to register specific sub domains at the same time as registering a wildcard domain, however, the wildcard domain must come first. Multiple wildcard domains are also allowed.

Also, when registering a wildcard domain, Dehydrated needs to give it an alias. You will see instructions online to separate the domain(s) with a '>' followed by the alias. You can do that if you wish, but Confconsole will ignore it and generate it's own alias.

The more advanced features supported when using Aliases with Dehydarated require using it standalone.

Important Note: Please ensure that you have your DNS registrar info correct. Incorrect authentication details will cause the Let's Encrypt challenges to fail (so you won't get a certificate). Repeated failures may cause your server to be blocked (for a week) from further attempts.

Getting a certificate - Behind the scenes

The process goes like this:

  • Confconsole Let's Encrypt plugin writes the domain (and subdomains) to /etc/dehydrated/confconsole.domains.txt
  • Confconsole calls dehydrated-wrapper
  • dehydrated-wrapper stops webserver listening on port 80
  • dehydrated-wrapper checks for /etc/dehydrated/confconsole.config (config file); if it doesn't exist, it copies it the default file from: /usr/share/confconsole/letsencrypt/dehydrated-confconsole.config
  • dehydrated-wrapper checks for /etc/dehydrated/confconsole.hook.sh (hook script); if it doesn't exist, it copies it the default file from: /usr/share/confconsole/letsencrypt/dehydrated-confconsole.hook.sh
  • dehydrated-wrapper checks for /etc/cron.daily/confconsole-dehydrated (cron script); if it doesn't exist, it copies it the default file from: /usr/share/confconsole/letsencrypt/dehydrated-confconsole.cron
  • dehydrated-wrapper calls dehydrated, passing confconsole.config, confconsole.hook.sh & confconsole.domains.txt (all stored within /etc/dehydrated/)
  • dehydrated contacts Let's Encrypt and gets the challenge (to prove you control the domain)
  • Whilst hosting the challenge, add-water temporarily redirects all web traffic except for the challenge (i.e. 304 - temporarily moved) to the web root. A simple "Under Maintence" message is displayed. To provide a custom html page, please see Advanced - custom maintence message below.
  • via the hook script, dehydrated serves Let's Encrypt challenges using add-water server (minimalist python webserver)
  • when done, add-water is killed (via hook script)
  • dehydrated writes certificate to /etc/ssl/private/cert.pem (via hook script); original certs generated by dehydrated remain in /var/lib/dehydrated/certs/DOMAIN
  • dehydrated hands back to dehydrated-wrapper
  • dehydrated-wrapper restarts webserver
  • dehydrated-wrapper restarts stunnel (so Webmin & Webshell also use new cert)
  • dehydrated-wrapper hands back to confconsole

Cron job details

The cron job checks the expiry date of the default certificate (/etc/ssl/private/cert.pem) and if it will expire within 30 days or less, it runs dehydrated-wrapper (using the --force switch).

Advanced - custom maintence message

As noted, whilst it is serving the challenges, add-water will redirect all other urls to the web root. By default it will display a simple "Maintenance" message via a basic index.html file.

If you wish to display a custom message, then you can add a custom index.html file to /var/lib/confconsole/letsencrypt/. E.g. to copy across the default and then tweak it:

mkdir -p /var/lib/confconsole/letsencrypt/
cp /usr/share/confconsole/letsencrypt/index.html \
  /var/lib/confconsole/letsencrypt/index.html

add-water will serve /var/lib/confconsole/letsencrypt/index.html if it exists, or otherwise will fall back to the default.

Note: The custom file must be named index.html and contain only valid HTML (and CSS). PHP or other server side languages are not supported. Embedded (i.e. "inline") JavaScript is also supported as that is processed client side.

Advanced - usage with multiple domain names

The interactive Confconsole plugin only supports a single domain with up to 4 subdomains. However, the dehydrated-wrapper can handle multiple root domains, plus multiple subdomains. To support that you will need to manually make configuration adjustments in a number of places:

  • add additional domains to /etc/dehydrated/confconsole.domains.txt
    • ensure that each line starts with the base domain, followed by a space separated list of any subdomains.
    • WARNING: If you re-run confconsole's Let's Encrypt plugin, your custom additional domains will be removed!
  • adjust your webserver virtual hosts to use the relevant certificates in /var/lib/dehydrated/certs/DOMAIN. Each domain will have it's own subdirectory under /var/lib/dehydrated/certs/.
  • By default, the last domain (and any subdomains) configured, will be the one which will work for Webmin & Webshell (and Adminer, if it's included). To change this behaviour, you can adjust the hook script, or simply rearange the order of your domains and put the one you want Webmin & Webshell available from, last.
  • Note: the cron job only checks the expiry of /etc/ssl/private/cert.pem. So if you adjust the hook script to no longer update /etc/ssl/private/cert.pem, you will also need to adjust the cron job to check the expiry of a certificate you are updating. Failure to do so will result in daily certificate updates, which may get your server temporarily blocked from accessing the Let's Encrypt servers.

Comments

Andi Northrop's picture

OK, I may be missing something obvious but I can't see what I should be doing after the first bullet point to make dehydrated pick up the additional domains and run the wrapper to get certificates for those new domains?

Will it just pick up the changes when the daily cron runs or do I need to invoke it myself somehow?

Cheers for any help!

Bill Carney's picture

Andi, did you ever figure out how to get the system to pick up the additional domains? My system retrieves the certs only for the first domain listed, and none of the others.

Jeremy Davis's picture

Hi Bill

Andi also posted in the forums. I probably should have noted the forum thread in response to Andi's question here (I missed this post at the time).

Hopefully that resolves your questions too Bill?! If you need more clarification or it's directly relevant, perhaps post on his thread. If it's not directly relevant to that thread, please open a new thread (perhaps link to Andi's in your new thread to as I'm sure it'll be at least vaguely relevant).

Jeremy Davis's picture

It looks like your post got picked up by the spam filter and I completely missed it. Sorry about that...

Unfortunately, Confconsole doesn't currently support certificates where the server isn't publicly available. It'd be nice to do, but we currently leverage the Debian package of Dehydrated, which is a older version, 0.3.1.

I note that newer versions of Dehydrated now supports dns01 challenges. And it appears that this newer version of Dehydrated is packaged as a Debian backport.

So it should be possible to update to the backported version and then use the dns01 challenge to get a certificate. However, the Confconsole plugin was never designed to be used in that way, so you'd need to leverage Dehydrated directly.

Obviously there are other Let's Encrypt client options, but personally, I quite like Dehydrated...

Bill Carney's picture

By default, the last domain (and any subdomains) configured, will be the one which will work for Webmin & Webshell (and Adminer, if it's included). To change this behaviour, you can adjust the hook script, or simply rearange the order of your domains and put the one you want Webmin & Webshell available from, last.

Following these directions, I've kept my domains for Webmin and Webshell as the very last line in the confconsole.domains.txt config file.  However, every time I add a new line (at the top) for another website I'm hosting, I get certificate errors because Webmin and Webshell start using the most recently created cert.  It's not a problem for me as I'm a one-person shop and I recognize that I've just added a new certificate, but this may be an issue for others.

Bill Carney's picture

My management cert (was) a subdomain, and I had the top domain and subdomains as the last line.  When I moved the management subdomain to the last line, by itself, the issue appears to go away.

was:

mysite.com www.mysite.com webmin.mysite.com

now it's:

mysite.com www.mysite.com
webmin.mysite.com

Jeremy Davis's picture

And thanks tons for posting back with your info.

FWIW I need to update these docs for v15.0 as the Let's Encrypt Confconsole plugin actually works slightly differently in v15.0 (Confconsole 1.1.0+). IIRC the source of the docs (on GH) should be up to date.

Ken's picture

I finished config ssl for my main domain successful but when i access turnkeyweb admin with ip_of_domain:12321

i saw in browser there are show that "your connection to this site is not secure"

Could some one help me in this problem config ssl for ip address

Jeremy Davis's picture

Hi Kevin, first up; you're best posting in the forums. You'll generally get more timely response there.

Anyway, I suspect that you're hitting one of the relatively recent issue that has plagued this since Let's Encrypt made some changes late last year. Ideally we should have re-released our appliances to include this fix, but we're working on a new major release, so we decided to just keep focused on that.

Regardless, I suspect that if you follow the instructions noted in the latest (v1.1.2) Confconsole release notes, that should get it working for you.

For context, please check to see if you're hitting the issues noted on these threads (the first is possibly better as it contains screenshots and was asked after we'd fixed everything; the second was much earlier and contains a lot of back and forth discussion as we worked through the multiple issues):

Bill Carney's picture

I believe you have to access the site via a domain name and not IP address (and of course have a cert for that domain name) in order for the connection to show as secure in the browser. 

Jeremy Davis's picture

Oops, I completely missed that!

You are absolutely right! Even if you have a valid Let's Encrypt cert, it's for the domain name, not the IP address! Well spotted! :)

Ken's picture

Many thanks for your tips

I have another problem with ssl could you look a bit in

https://www.turnkeylinux.org/forum/support/fri-20200221-2202/insecure-si...

Guest's picture

Hello! I'm hosting a website with a few subdomains and i added them all to the file, but no where in this document says what the dehydrated command is that this runs or how often it checks or if i can just add domains and that next time it checks they will be added or not.. and if that's not the case will it work if i just delete the certificates until it runs again? any help would be highly appreciated.
Jeremy Davis's picture

As noted in the 'Cron job details" session, it checks the expiry of the default cert (/etc/ssl/private/cert.pem) and if it will expire within the next 30 days, it will attempt cert renewal.

To manually force it to get a new certificate (after updating /etc/dehydrated/confconsole.domains.txt as desired), run our dehydrated wrapper script directly, with the '-f' (or '--force') switch (to force it to get a new cert) like this:

/usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper -f

You can read the script's help first by passing the '--help' (or '-h') switch (instead of the '-f' switch).

I hope that helps.

Zorroken's picture

Hi,

First, thanks very much for Turnkey Linux which is a great way for me to learn lot of things with linux, virtulalization, webserver etc... :-)

In my autolearning life with Turnkey LAMP server, I encountered an issue with the deshydrated script to renew let's encrypt certifcate using the confconsole or using the '/usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper -f' command.

The script seems to work find

My apache conf contains this line :

But when I go to the site, https://revp01.mydomain.com

I got this issue : the certificate is for the name "mydomain.com" and not "revp01.mydomain.com"

And the certificate is not valid because the script renew for good reason the "revp01.mydomain.com"

But a wrong certificate is used...

I don't understand why and want to reset this from scratch but I don't know how to do this...

I tried to delete :

/usr/local/share/ca-certificates/cert.crt

and

/etc/ssl/private/cert.key

Then renew the certificate with the "/usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper -f" command

But it doesn't change anything...

Can you tell me how to reset from scratch the certificate used by the webserver and correctly renew the certificate with the good subdomain "name" ?

Thanks in advance...

Jeremy Davis's picture

The file that notes/sets the domain to get a cert for is /etc/dehydrated/confconsole.domains.txt. If you just want a cert for a single domain, just ensure that that is the only contents of the file (lines starting with '#' are ignored). E.g. if I wanted to get a cert for my.domain.com, the contents of /etc/dehydrated/confconsole.domains.txt would look like:

# please use this file with confconsole or
# alternatively use dehydrated with it's appropriate
# configuration directly
my.domain.com

Also, make sure that you clear cache and cookies from your browser (or better still - do testing in a private/incognito browser window - and close it once done).

The above should work fine and there should not be any need to "start again". However, if you really want to, don't worry about the file in /usr/local/share/ca-certificates/cert.crt - that isn't used. The files that your server uses should be /etc/ssl/private/cert.key (private key only) & /etc/ssl/private/cert.pem (cert, key and dh params combined into one file). And they are overwritten by a successful run, so you don't need to remove them. To clear all existing Dehydrated (the tool we use to get certs) config, run this command:

find /etc/dehydrated -type f -name "confconsole*" -exec rm {} \;

Then running Confconsole will copy new default config into place. Then configure as desired.

I hope that helps.

The certificates are