You are here
JackF - Fri, 2022/12/09 - 00:08
On a virtual machine with a private IPv4, and a public IPv6 interfaces/addresses, running the gitea appliance
root@g ~# turnkey-version turnkey-gitea-17.1-bullseye-amd64 root@g ~# lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 11 (bullseye) Release: 11 Codename: bullseye
Dehydrated fails to obtain a cert with a time out error.
Investigation shows that the python script that Dehydrated spawns is only listening on 0.0.0.0:80
Disabling the IPv4 interface (ifdown) and keeping the IPv6 as the only option for connectivity changes nothing.
The DNS is set up properly with the host name pointing to a single IPv6 address. The issue is that python is not listening on IPv6.
Here are the listening processes while waiting for the verification:
<!--break-->
root@g ~# ss -lptn
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 80 127.0.0.1:3306 0.0.0.0:* users:(("mariadbd",pid=481,fd=48))
LISTEN 0 4096 0.0.0.0:5355 0.0.0.0:* users:(("systemd-resolve",pid=137,fd=12))
LISTEN 0 5 0.0.0.0:80 0.0.0.0:* users:(("python3",pid=11630,fd=5))
LISTEN 0 4096 127.0.0.1:10000 0.0.0.0:* users:(("miniserv.pl",pid=9948,fd=5))
LISTEN 0 4096 127.0.0.53%lo:53 0.0.0.0:* users:(("systemd-resolve",pid=137,fd=17))
LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=369,fd=3))
LISTEN 0 1 127.0.0.1:9977 0.0.0.0:* users:(("python3",pid=11630,fd=3))
LISTEN 0 100 127.0.0.1:25 0.0.0.0:* users:(("master",pid=4686,fd=13))
LISTEN 0 128 127.0.0.1:12319 0.0.0.0:* users:(("shellinaboxd",pid=9132,fd=4))
LISTEN 0 4096 0.0.0.0:12321 0.0.0.0:* users:(("stunnel4",pid=9081,fd=9))
LISTEN 0 4096 [::]:5355 [::]:* users:(("systemd-resolve",pid=137,fd=14))
LISTEN 0 128 [::]:22 [::]:* users:(("sshd",pid=369,fd=4))
LISTEN 0 4096 *:12320 *:* users:(("stunnel4",pid=9101,fd=9))
Dehydrated log:
root@g ~# /usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper -r
[2022-12-08 21:57:02] dehydrated-wrapper: INFO: started
[2022-12-08 21:57:02] dehydrated-wrapper: INFO: No process found listening on port 80; continuing
[2022-12-08 21:57:02] dehydrated-wrapper: INFO: running dehydrated
ERROR: Challenge is invalid! (returned: invalid) (result: ["type"] "http-01"
["status"] "invalid"
["error","type"] "urn:ietf:params:acme:error:connection"
["error","detail"] "2a11:4c8:173:462::16:1: Fetching http://gt.domain.com/.well-known/acme-challenge/SgxwEdm4aWjV5eQtAfH_UEd9qtrGnL1XWtwU1pq4r: Timeout during connect (likely firewall problem)"
["error","status"] 400
["error"] {"type":"urn:ietf:params:acme:error:connection","detail":"2a11:4c8:173:462::16:1: Fetching http://gt.domain.com/.well-known/acme-challenge/SgxwEdm4aWjV5eQtAfH_UEd9qtrGnL1XWtwU1pq4r: Timeout during connect (likely firewall problem)","status":400}
["url"] "https://acme-v02.api.letsencrypt.org/acme/chall-v3/185464525747/IN0IoQ"
["token"] "SgxwEdm4aWjV5eQtAfH_UEd9qtrGnL1XWtwU1pq4r"
["validationRecord",0,"url"] "http://gt.domain.com/.well-known/acme-challenge/SgxwEdm4aWjV5eQtAfH_UEd9qtrGnL1XWtwU1pq4r"
["validationRecord",0,"hostname"] "gt.domain.com"
["validationRecord",0,"port"] "80"
["validationRecord",0,"addressesResolved",0] "2a11:4c8:173:462::16:1"
["validationRecord",0,"addressesResolved"] ["2a11:4c8:173:462::16:1"]
["validationRecord",0,"addressUsed"] "2a11:4c8:173:462::16:1"
["validationRecord",0] {"url":"http://gt.domain.com/.well-known/acme-challenge/SgxwEdm4aWjV5eQtAfH_UEd9qtrGnL1XWtwU1pq4r","hostname":"gt.domain.com","port":"80","addressesResolved":["2a11:4c8:173:462::16:1"],"addressUsed":"2a11:4c8:173:462::16:1"}
["validationRecord"] [{"url":"http://gt.domain.com/.well-known/acme-challenge/SgxwEdm4aWjV5eQtAfH_UEd9qtrGnL1XWtwU1pq4r","hostname":"gt.domain.com","port":"80","addressesResolved":["2a11:4c8:173:462::16:1"],"addressUsed":"2a11:4c8:173:462::16:1"}]
["validated"] "2022-12-08T21:57:07Z")
[2022-12-08 21:57:19] dehydrated-wrapper: FATAL: dehydrated exited with a non-zero exit code.
[2022-12-08 21:57:19] dehydrated-wrapper: WARNING: Python is still listening on port 80
[2022-12-08 21:57:19] dehydrated-wrapper: INFO: attempting to kill add-water server
[2022-12-08 21:57:19] dehydrated-wrapper: WARNING: Something went wrong, restoring original cert, key and combined files.
[2022-12-08 21:57:19] dehydrated-wrapper: INFO: (Re)starting stunnel4@webmin.service
[2022-12-08 21:57:19] dehydrated-wrapper: INFO: (Re)starting stunnel4@shellinabox.service
[2022-12-08 21:57:19] dehydrated-wrapper: WARNING: Check today's previous log entries for details of error.
Any idea how to get this to work?
Thank you.
Note: names and addresses changed for privacy.
Forum:
Thanks for the report! I'll have a look ASAP.
It's an oversight on our behalf, but you're right, it certainly should support it, ideally OOTB - but at least via a config option. Let me have a play and I'll get back to you.
Thanks again for reporting. I have a work around! :)
It's a pretty easy fix. I didn't actually test getting a certificate, but I confirmed that it listens on IPv6 (and still on IPv4 too) after this change. Please be aware that if you retried lots of times, there is a chance that your IP has been blacklisted. Unfortunately, if that's happened, you'll either need to wait - or get a new IP :(
So to apply the workaround, you'll need to edit /usr/lib/confconsole/plugins.d/Lets_Encrypt/add-water-srv. On the very last line, change what is there:
to
And you should be good to go! :)
I've opened an issue on our tracker so it doesn't get forgotten and I've also opened a PR that will fix it in the code base.
Works like a treat for my use case.
Thank you so much for the fast responses, Jeremy.
Much appreciated!
Awesome! Thanks for confirming.
I was fairly confident that it would work, but thanks for the confirmation.
Also thanks too for the quality of your bug report. With a detailed report like that with lots of info, you really do make my life easier. If only everyone where so good at sharing info about problems they hit! :)
Add new comment