Updated Webmin packages in TurnKey testing repository

[Update: 2021-02-12] Updated Webmin packages which include Webmin v1.970 (plus the previously mentioned tweak to the MySQL module) have been pushed to the "main" TurnKey repo. So enabling the TurnKey testing repo is no longer required (I've edited the post to hopefully make that clear).

[Update: 2020-11-06] Updated Webmin packages which include Webmin v1.960; plus a tweak to the MySQL module.


I'm happy to announce new Webmin packages are available in the TurnKey buster-testing repository. Whilst we have done some testing and they should work fine, I'd really love to have some other testers install them and report back! :)

New Webmin v1.955 v1.960 v1.970 packages for TurnKey v16.x

I have just built new TurnKey Webmin v1.955 v1.960 v1.970 packages and uploaded them to the TurnKey buster-testing buster (main) repository. I also intend to upload them to main TurnKey buster repository too; but I'd really like to hear from at least a couple of other users to confirm our internal testing results.

These new Webmin packages primarily achieve 2 3 ends:

  1. Firstly the Webmin update provides an improved service file which should make the Webmin service more reliable on TurnKey; particularly on LXC containers. Closes bug #1480.
  2. They also provide a newer version of Webmin; namely the current v1.955 v1.960 v1.970. The changes since v1.941 (the default version in TurnKey Linux v16.0) are not super significant and don't contain any security updates but I figured that I may as well update Webmin whilst providing the above noted bugfix(es). The full list of changes can be viewed on the Webmin website but changes since the previous packages include:
    • Improved support for user and permission management in newer MySQL / MariaDB versions. [2020-10-30]
    • Updated Authentic theme (bugfixes and improvements).
    • Support for Postfix SNI certificate maps.
    • Caching for LDAP and MySQL connections for Webmin users.
    • Optional automatically generated translations for all languages, and switched all encodings to UTF-8
    • Removed several noisy messages from the error log.
    • Many, many small/minor bugfixes and features.
[2020-10-30] Also includes a tweak to the default MySQL admin config; namely changes the default username to 'adminer' (was 'root'). This addresses a situation where a user could easily and inadvertently break their MySQL (MariaDB) server instance (on any TurnKey server which includes MySQL/MariaDB).

How to update to the new "testing" packages

Please note that these instructions only apply to v16.x users. It may be possible to install these newer packages on previous versions of TurnKey, but we advise that you either migrate your data or upgrade your instance.

These instructions assume that you are logged in as root. If that's not the case, please run this command first:

sudo su

Then enable the TurnKey buster-testing repository and update the package lists:

mv /etc/apt/sources.list.d/turnkey-testing.list.disabled /etc/apt/sources.list.d/turnkey-testing.list
apt update
apt update

You can now view the upgradeable packages like this:

apt list --upgradable

This will list all the packages on your system which are upgradable (not just Webmin). To collect just the installed (and upgradeable) Webmin packages, run this:

webmin_pkgs=$(apt list --upgradable 2>/dev/null | grep ^webmin | cut -d/ -f1)

You can double check the list of packages like this:

echo "$webmin_pkgs"

My test machine returns this:

webmin-apache
webmin-authentic-theme
webmin-custom
webmin-fail2ban
webmin-fdisk
webmin-filemin
webmin-firewall
webmin-lvm
webmin-mount
webmin-net
webmin-passwd
webmin-postfix
webmin-raid
webmin-shell
webmin-software
webmin-sshd
webmin-syslog
webmin-updown
webmin-useradmin
webmin

Assuming that yours looks somewhat similar, you can save this list of packages (just in case you want to rollback) like this:

echo "$webmin_pkgs" > ~/webmin_pkgs.txt

Then upgrade the packages:

apt upgrade $webmin_pkgs

If it all looks good, then you can approve the change by hitting 'y'. If you want to upgrade all the upgradeable packages on your server, you can instead use a similar command, but omit the '$webmin_pkgs' bit - i.e. 'apt upgrade'.

Once the upgrade has completed, you should be all good and if you browse to port 12321 on HTTPS you should be greeted by the latest Webmin UI. You can double check on the "Dashboard". It should display: "Webmin version - 1.955" & "Authentic theme version - 19.54" "Webmin version - 1.960" & "Authentic theme version - 19.61" "Webmin version - 1.970" & "Authentic theme version - 19.71" (or similar - I don't have the exact version in front of me).

Troubleshooting

If for some reason it's not running, please try restarting 'webmin.service' and 'stunnel4@webmin.service' like this:

systemctl restart webmin.service stunnel4@webmin.service

If you continue to have issues, please post back and provide the output of the following commands:

systemctl status webmin.service stunnel4@webmin.service

Worst case scenario; rollback

If things really aren't working and you want to rollback to the previous version, then this should work:

apt install $(cat ~/webmin_pkgs.txt | sed "s|$|=1.941-turnkey+2+g51d4dbf|")

Disable TurnKey buster-testing repo again

Disabling the TurnKey buster-testing repository again is super simple; just rename the file again:

mv /etc/apt/sources.list.d/turnkey-testing.list /etc/apt/sources.list.d/turnkey-testing.list.disabled

Regardless of your experience, please post feedback

Hopefully your testing goes as well as mine did and everything "just works" with the new version. But regardless, please let me know. You can post a comment below or get hold of me via other channels.

Comments

Jeremy Davis's picture

Firstly you say "still doesn't work". I assume this means that you were previously having issue with Webmin running prior to performing this Webmin update? Could you please confirm?

Regardless, from what you've posted, the issue that you appear to be hitting doesn't seem related to Webmin itself, but the Stunnel service ('stunnel4@webmin.service') that Webmin depends on. It's not clear to me exactly why, but it appears that Stunnel is not starting. That appears to be causing Webmin to fail to start (because it depends on the stunnel4@webmin.service).

I see in the status that you've shared that Stunnel does note that the "Start request repeated too quickly." so perhaps it's just an issue restarting Stunnel?

I had nearly finished writing an extensive post responding to you, but during my investigations I've actually discovered some shortcomings in the Stunnel service template that we provide (stunnel4@.service). I have documented these in on our issue tracker.

TBH, I'm not sure that any of these shortcomings are directly related to your issue. Regardless, I have created a new template file to address the issues. I have done some basic testing and I figure that it's worth sharing it with your to see if it helps (and also get your input on whether it fixes your issue). So I've rewritten this post to download and use the new Stunnel server template file. It should be an improvement and if it doesn't fix the issue, then we can do some more troubleshooting.

So first thing to do is to download the updated template file:

FILE=usr/lib/systemd/system/stunnel4@.service
COMMIT_ID=5cd698673671dea499e6af9f72dac69e5c93c4f5
BASE_URL=https://raw.githubusercontent.com/JedMeister/common
URL=$BASE_URL/$COMMIT_ID/overlays/turnkey.d/stunnel/$FILE
wget $URL -O /$FILE

Then run the following commands to load the new file and (re)start the services:

systemctl daemon-reload
systemctl start stunnel4@webmin.service
systemctl start webmin.service

Hopefully Webmin should now be running and accessible via port 12321. You can double check the service's status again via systemctl:

systemctl status stunnel4@webmin.service
systemctl status webmin.service

If it's still not working, I suggest a reboot and see if that helps at all.

If after a reboot it's still failing, first please double check the status of stunnel4@webmin.service to double check that it's still the same Stunnel not starting issue. Assuming so, please post the output of:

journalctl -u stunnel4@webmin.service

Also it might be useful to see what the status of the relevant targets are:

systemctl status network.target network-online.target remote-fs.target
Jeremy Davis's picture

I'm not sure why, but Jerry's post didn't get autoposted. So here it is:


Hi Jeremy,

Thank you very much for the detailed reply.

I am running it as LXC container, downloaded via Proxmox templates (16.0-1). After running the container, I did an update.

1. It was not working with the previous version of webmin (just like what is described in the bug report.). Then I followed the procedure to install new version 1.955, and still doesn't work due to a stunnel server error. I am not expert in these. Just a amateur (a researcher at University trying to set up a server for lab members) with a little bit tech and computer science background.

2. Still fails to start stunnel, even after reboot.

I really appreciate your help.

*wget $URL -O /$FILE*
--2020-09-17 14:54:33--
https://raw.githubusercontent.com/JedMeister/common/5cd698673671dea499e6af9f72dac69e5c93c4f5/overlays/turnkey.d/stunnel/usr/lib/systemd/system/stunnel4@.service
Resolving raw.githubusercontent.com (raw.githubusercontent.com)...
151.101.0.133, 151.101.64.133, 151.101.128.133, ...
Connecting to raw.githubusercontent.com
(raw.githubusercontent.com)|151.101.0.133|:443...
connected.
HTTP request sent, awaiting response... 200 OK
Length: 330 [text/plain]
Saving to: '/usr/lib/systemd/system/stunnel4@.service'

/usr/lib/systemd/system/stunnel4@.service
100%[====================================================================================================>]    330  --.-KB/s    in 0s

2020-09-17 14:54:34 (44.5 MB/s) - '/usr/lib/systemd/system/stunnel4@.service' saved [330/330]
*root@Turnkey-SMB-Server ~# systemctl status stunnel4@webmin.service*
* stunnel4@webmin.service - Universal SSL tunnel for network daemons (webmin)
   Loaded: loaded (/lib/systemd/system/stunnel4@.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2020-09-17 14:43:04 UTC; 2min 27s ago
  Process: 512 ExecStart=/usr/bin/stunnel4 /etc/stunnel/webmin.conf (code=exited, status=226/NAMESPACE)

Sep 17 14:43:03 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Control process exited, code=exited, status=226/NAMESPACE
Sep 17 14:43:03 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Failed with result 'exit-code'.
Sep 17 14:43:03 Turnkey-SMB-Server systemd[1]: Failed to start Universal SSL tunnel for network daemons (webmin).
Sep 17 14:43:04 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Service RestartSec=1s expired, scheduling restart.
Sep 17 14:43:04 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Scheduled restart job, restart counter is at 5.
Sep 17 14:43:04 Turnkey-SMB-Server systemd[1]: Stopped Universal SSL tunnel for network daemons (webmin).
Sep 17 14:43:04 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Start request repeated too quickly.
Sep 17 14:43:04 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Failed with result 'exit-code'.
Sep 17 14:43:04 Turnkey-SMB-Server systemd[1]: Failed to start Universal SSL tunnel for network daemons (webmin).
*root@Turnkey-SMB-Server ~# systemctl start stunnel4@webmin.service*
Job for stunnel4@webmin.service failed because the control process exited with error code.
See "systemctl status stunnel4@webmin.service" and "*journalctl -xe*" for details.
*root@Turnkey-SMB-Server ~# journalctl -xe*
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- The unit stunnel4@webmin.service has entered the 'failed' state with result 'exit-code'.
Sep 17 14:45:54 Turnkey-SMB-Server systemd[1]: Failed to start Universal SSL tunnel for network daemons (webmin).
-- Subject: A start job for unit stunnel4@webmin.service has failed 
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- A start job for unit stunnel4@webmin.service has finished with a failure.
-- 
-- The job identifier is 779 and the job result is failed.
Sep 17 14:45:56 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Service RestartSec=1s expired, scheduling restart.
Sep 17 14:45:56 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Scheduled restart job, restart counter is at 5.
-- Subject: Automatic restarting of a unit has been scheduled
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- Automatic restarting of the unit stunnel4@webmin.service has been scheduled, as the result for
-- the configured Restart= setting for the unit.
Sep 17 14:45:56 Turnkey-SMB-Server systemd[1]: Stopped Universal SSL tunnel for network daemons (webmin).
-- Subject: A stop job for unit stunnel4@webmin.service has finished
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- A stop job for unit stunnel4@webmin.service has finished.
-- 
-- The job identifier is 826 and the job result is done.
Sep 17 14:45:56 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Start request repeated too quickly.
Sep 17 14:45:56 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- The unit stunnel4@webmin.service has entered the 'failed' state with result 'exit-code'.
Sep 17 14:45:56 Turnkey-SMB-Server systemd[1]: Failed to start Universal SSL tunnel for network daemons (webmin).
-- Subject: A start job for unit stunnel4@webmin.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- A start job for unit stunnel4@webmin.service has finished with a failure.
-- 
-- The job identifier is 826 and the job result is failed.
*root@Turnkey-SMB-Server ~# journalctl -u stunnel4@webmin.service*
-- Logs begin at Thu 2020-09-17 14:42:58 UTC, end at Thu 2020-09-17 14:45:56 UTC. --
Sep 17 14:42:58 Turnkey-SMB-Server systemd[1]: Starting Universal SSL tunnel for network daemons (webmin)...
Sep 17 14:42:58 Turnkey-SMB-Server systemd[226]: stunnel4@webmin.service: Failed to set up mount namespacing: Permission denied
Sep 17 14:42:58 Turnkey-SMB-Server systemd[226]: stunnel4@webmin.service: Failed at step NAMESPACE spawning /usr/bin/stunnel4: Permission denied
Sep 17 14:42:58 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Control process exited, code=exited, status=226/NAMESPACE
Sep 17 14:42:58 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Failed with result 'exit-code'.
Sep 17 14:42:58 Turnkey-SMB-Server systemd[1]: Failed to start Universal SSL tunnel for network daemons (webmin).
Sep 17 14:42:59 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Service RestartSec=1s expired, scheduling restart.
Sep 17 14:42:59 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Scheduled restart job, restart counter is at 1.
Sep 17 14:42:59 Turnkey-SMB-Server systemd[1]: Stopped Universal SSL tunnel for network daemons (webmin).
Sep 17 14:42:59 Turnkey-SMB-Server systemd[1]: Starting Universal SSL tunnel for network daemons (webmin)...
Sep 17 14:42:59 Turnkey-SMB-Server systemd[483]: stunnel4@webmin.service: Failed to set up mount namespacing: Permission denied
Sep 17 14:42:59 Turnkey-SMB-Server systemd[483]: stunnel4@webmin.service: Failed at step NAMESPACE spawning /usr/bin/stunnel4: Permission denied
Sep 17 14:42:59 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Control process exited, code=exited, status=226/NAMESPACE
Sep 17 14:42:59 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Failed with result 'exit-code'.
Sep 17 14:42:59 Turnkey-SMB-Server systemd[1]: Failed to start Universal SSL tunnel for network daemons (webmin).
Sep 17 14:43:00 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Service RestartSec=1s expired, scheduling restart.
Sep 17 14:43:00 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Scheduled restart job, restart counter is at 2.
Sep 17 14:43:00 Turnkey-SMB-Server systemd[1]: Stopped Universal SSL tunnel for network daemons (webmin).
Sep 17 14:43:00 Turnkey-SMB-Server systemd[1]: Starting Universal SSL tunnel for network daemons (webmin)...
Sep 17 14:43:00 Turnkey-SMB-Server systemd[501]: stunnel4@webmin.service: Failed to set up mount namespacing: Permission denied
Sep 17 14:43:00 Turnkey-SMB-Server systemd[501]: stunnel4@webmin.service: Failed at step NAMESPACE spawning /usr/bin/stunnel4: Permission denied
Sep 17 14:43:00 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Control process exited, code=exited, status=226/NAMESPACE
Sep 17 14:43:00 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Failed with result 'exit-code'.
Sep 17 14:43:00 Turnkey-SMB-Server systemd[1]: Failed to start Universal SSL tunnel for network daemons (webmin).
Sep 17 14:43:01 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Service RestartSec=1s expired, scheduling restart.
Sep 17 14:43:01 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Scheduled restart job, restart counter is at 3.
Sep 17 14:43:01 Turnkey-SMB-Server systemd[1]: Stopped Universal SSL tunnel for network daemons (webmin).
Sep 17 14:43:01 Turnkey-SMB-Server systemd[1]: Starting Universal SSL tunnel for network daemons (webmin)...
Sep 17 14:43:01 Turnkey-SMB-Server systemd[506]: stunnel4@webmin.service: Failed to set up mount namespacing: Permission denied
Sep 17 14:43:01 Turnkey-SMB-Server systemd[506]: stunnel4@webmin.service: Failed at step NAMESPACE spawning /usr/bin/stunnel4: Permission denied
Sep 17 14:43:01 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Control process exited, code=exited, status=226/NAMESPACE
Sep 17 14:43:01 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Failed with result 'exit-code'.
Sep 17 14:43:01 Turnkey-SMB-Server systemd[1]: Failed to start Universal SSL tunnel for network daemons (webmin).
Sep 17 14:43:03 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Service RestartSec=1s expired, scheduling restart.
Sep 17 14:43:03 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Scheduled restart job, restart counter is at 4.
Sep 17 14:43:03 Turnkey-SMB-Server systemd[1]: Stopped Universal SSL tunnel for network daemons (webmin).
Sep 17 14:43:03 Turnkey-SMB-Server systemd[1]: Starting Universal SSL tunnel for network daemons (webmin)...
Sep 17 14:43:03 Turnkey-SMB-Server systemd[512]: stunnel4@webmin.service: Failed to set up mount namespacing: Permission denied
Sep 17 14:43:03 Turnkey-SMB-Server systemd[512]: stunnel4@webmin.service: Failed at step NAMESPACE spawning /usr/bin/stunnel4: Permission denied
Sep 17 14:43:03 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Control process exited, code=exited, status=226/NAMESPACE
Sep 17 14:43:03 Turnkey-SMB-Server systemd[1]: stunnel4@webmin.service: Failed with result 'exit-code'.
Sep 17 14:43:03 Turnkey-SMB-Server systemd[1]: Failed to start Universal SSL tunnel for network daemons (webmin).
*root@Turnkey-SMB-Server ~# systemctl status network.target network-online.target remote-fs.target*
* network.target - Network
   Loaded: loaded (/lib/systemd/system/network.target; static; vendor preset: enabled)
   Active: active since Thu 2020-09-17 14:42:58 UTC; 6min ago
     Docs: man:systemd.special(7)
           https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget

Sep 17 14:42:58 Turnkey-SMB-Server systemd[1]: Reached target Network.

* network-online.target - Network is Online
   Loaded: loaded (/lib/systemd/system/network-online.target; static; vendor preset: enabled)
   Active: active since Thu 2020-09-17 14:42:58 UTC; 6min ago
     Docs: man:systemd.special(7)
           https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget

Sep 17 14:42:58 Turnkey-SMB-Server systemd[1]: Reached target Network is Online.

* remote-fs.target - Remote File Systems
   Loaded: loaded (/lib/systemd/system/remote-fs.target; enabled; vendor preset: enabled)
   Active: active since Thu 2020-09-17 14:42:58 UTC; 6min ago
     Docs: man:systemd.special(7)

Sep 17 14:42:58 Turnkey-SMB-Server systemd[1]: Reached target Remote File Systems.
Jeremy Davis's picture

So, the thing that jumps out at me from reading through these logs is the note of issues involving namespacing; i.e.:

stunnel4@webmin.service: Failed to set up mount namespacing: Permission denied
stunnel4@webmin.service: Failed at step NAMESPACE spawning /usr/bin/stunnel4: Permission denied
stunnel4@webmin.service: Control process exited, code=exited, status=226/NAMESPACE

So it seems that this is related to namespacing within the LXC container. Whilst I haven't heard of issues with Stunnel before, it seems almost identical to other issues I have read about regarding MariaDB (MySQL drop in replacement) and Apache. All of these issues appear to be related to privileged LXC containers based on Debian Buster based (as TurnKey v16.x is - or similar equivalent Ubuntu version).

AFAIK, the workaround is to either run a non-privileged container; or enable nesting for the container. Please note that enabling nesting does have security implications related to guest systems potentially having access to the host. So I'd recommend usage of unprivileged containers ideally.

Having said that, seeing as we're providing the Stunnel service template, then perhaps we can implement a better fix?! Maybe something like what has been suggested for Apache (although apparently Apache now runs ok in unprivileged guests?)

Unfortunately I don't actually have immediate access to v6.x Proxmox to test this out, but hopefully we can work it out.

BTW, here are some links to pages that provided some relevant info:

Jeremy Davis's picture

Thinking about this more, it'd be good to get a little more info about your setup if you don't mind.

Is this privileged or unprivileged container?

If it's already an unprivileged container, it'd be great to get you to test out some more tweaks to see how they go.

Jeremy Davis's picture

Thanks for posting back to confirm my suspicion! :)

Resty's picture

Need to try those steps hoping there will be no bumps ahead.

Jeremy Davis's picture

I've had a few positive reports now, so fingers crossed it works well for you too! :)

Pages

Add new comment