John I.'s picture


I have some trouble to use my own certificate with stunnel.
I'm trying to setup a TurnKey PostgreSQL 15.0, but first of all a want the turnkey's web interfaces to have a valid SSL certificate.

My certificate is provided by an external authority, with its pem and its private key.

The thing is at boot time only the stunnel unit failed to load with the specific error:

/etc/init.d/stunnel status

* stunnel4.service - LSB: Start or stop stunnel 4.x (TLS tunnel for network daemons)
   Loaded: loaded (/etc/init.d/stunnel4; generated; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2019-05-07 10:48:02 CEST; 30min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 438 ExecStart=/etc/init.d/stunnel4 start (code=exited, status=1/FAILURE)

May 07 10:48:02 rhodecode stunnel4[438]: [!] error queue: 140DC002: error:140DC002:SSL routines:use_certificate_chain_file:system lib
May 07 10:48:02 rhodecode stunnel4[438]: [!] error queue: 20074002: error:20074002:BIO routines:file_ctrl:system lib
May 07 10:48:02 rhodecode stunnel4[438]: [!] SSL_CTX_use_certificate_chain_file: 2001002: error:02001002:system library:fopen:No such file or directory
May 07 10:48:02 rhodecode stunnel4[438]: [!] Service [shellinabox]: Failed to initialize TLS context
May 07 10:48:02 rhodecode stunnel4[438]: failed
May 07 10:48:02 rhodecode stunnel4[438]: You should check that you have specified the pid= in you configuration file
May 07 10:48:02 rhodecode systemd[1]: stunnel4.service: Control process exited, code=exited status=1
May 07 10:48:02 rhodecode systemd[1]: Failed to start LSB: Start or stop stunnel 4.x (TLS tunnel for network daemons).
May 07 10:48:02 rhodecode systemd[1]: stunnel4.service: Unit entered failed state.
May 07 10:48:02 rhodecode systemd[1]: stunnel4.service: Failed with result 'exit-code'.

But, logged as root, if I start the unit manually, either by /etc/init.d/stunnel start or systemctl start stunnel4.service everything is working as expected; stunnel is starting and the turnkey's web interfaces show up my certificate.

The only edit on /etc/stunnel/stunnel.conf is the certificate location:

; **************************************************************************
; * Service defaults may also be specified in individual service sections  *
; **************************************************************************

; Certificate/key is needed in server mode and optional in client mode
;cert = /etc/ssl/private/cert.pem
cert = /var/run/authority-ca/cert.pem
key = /var/run/authority-ca/privkey.pem

/var/run/authority-ca is a permanent directory and exists at boot time, files included.

I tried many things without success, I may have misstep few.
Moving the certs to /etc/stunnel/authority-ca, or to /etc/ssl/{certs,private}, suspecting the chroot, changing permissions, with no luck.

Did you ever experienced something like these?
Or have some hints on any aspects I may have overlooked?


Jeremy Davis's picture

TBH, I don't understand why it appears to work ok when you manually start the service, whereas on reboot it fails?! That seems a little weird to me. I would expect it to fail consistently, or "just work".

As background, it maybe worth noting that Stunnel provides a SSL/TLS front end for both Webmin and Webshell. We do it that way as it's the easiest way to use a single cert across all browser accessible services. Webshell in particular has some limitations with regard to SSL certs (note; technically they're TLS certs these days). So if you don't need/want Webshell (you could use SSH instead for CLI usage?!) then you could try re-configuring Webmin to using your certificate directly (and remove use of Stunnel altogether). I won't go into details of that here yet, as it would be a little time consuming and convoluted, but if that's an appealing option, please let me know and I can try to point you in the right direction.

FWIW it took us a bit of tweaking to get it so that Stunnel, Apache, Nginx, etc could all use the same certificate(s). IIRC some services require the key to be included in the certificate file (i.e. they use just use a single pem file), whereas others need the key to be within a separate key files (and will ignore the key within the cert.pem file). I think that the order of the contents of the pem file may also matter (but I don't 100% recall - it's been a while)?!

The first thing that is possibly worth noting is that if this server has incoming internet connection, then you could use the Confconsole Let's Encrypt plugin to generate (free) Let's Encrypt certificates. You'd then end up with a trusted certificate that should "just work" with all the default TurnKey services (for free too!).

Although if you need to use a particular certificate, then you'll need to persevere. Beyond permissions, the actual location of the files shouldn't matter. And there is no reason why you couldn't (re)construct the file specifically for Stunnel (if it's too fiddly to get a single cert/key combo to work with all services). FWIW because we include the key within the cert file, we lock things down pretty tight re permissions on /etc/ssl/private and only root has access, so that will cause issues for any app that doesn't at least at some point have elevated permissions.

Whilst it's not directly relevant in this case, I wrote a blog post last year which may be of some assistance. It's specifically talking about creating pem file certs from alternate formats, but it does cover some aspects of the way the pem files are laid out and how to adjust them.

So unfortunately, I still haven't directly answered your question. But hopefully I've given you a few options to explore further. If you want/need some further input, please ask. Also regardless of which direction you go, it'd be great to hear how things work out.

John I.'s picture

Hi, thanks for your reply.

First of all, I followed your blogpost by dropping my certificate alongside its key to /etc/ssl/private/{cert.pem,cert.key}; the /etc/ssl/private/cert.pem is chained with the content of the /etc/ssl/private/cert.key after the operation. The /etc/stunnel/stunnel.conf is the shipped one.

It is working this way, a script will manage to update this cert when /var/run/authority-ca is updated. It is good enough for me.

Regarding the Let's Enscrypt solution, I have:

  1. The service is not visible nor accessible outside the intranet;
  2. The required certificate is a wildcard certificate;
  3. The DNS doesn't provide any API to update its, at least, TXT records.

The dehydrated script doesn't yet handle the wildcard certificate from Let's Encrypt; I tried to backport a newer version with no success. But with a DNS which doesn't allow a remote update, the solution is to setup something like acme-dns. I left that subject out.


Jeremy Davis's picture

Fantastic that you worked it out! Thanks for confirming.

Re Let's Encrypt; ah ok. Yes the only way that I'm aware of getting wildcard certs is via Dehydrated is DNS authentication. Because it's all automated, and Let's Encrypt certs can support numerous sub-domains, you generally don't need a wildcard cert. So that alone isn't really a showstopper. Although it does make initial setup a bit more of a pain (each time you use a new subdomain, you need to either extend the existing cert or get a new one).

In theory you could leverage the DNS auth capabilities of Dehydrated and that would also work around the issue of no internet access. However, our current implementation is purely designed for use of the HTTP authentication method (which is obviously of no value to you...!).

Add new comment