Syncthing doesn't start after adding LXC id mapping

It seems like perhaps the user ID mapping is not working as it should? I recommend checking the UID of the 'syncthing' user within the container both with the UID mapping and without. In theory, it shouldn't change within the container, but perhaps it is somehow/has for some reason? Perhaps a bug in LXC?

I.e. double check permissions of /home/syncthing/.config/:

ls -la /home/syncthing/.config

Check to see who owns it. Then double check that against the UID noted for the syncthing user:

grep ^syncthing /etc/passwd

[update] I've had a closer look at lxd.idmap and TBH, I'm more confused... Without knowing about exactly what you are trying to achieve, I can't be sure whether it's working as intended or not?! Also I'm not super familiar with the inner working of LXC. Perhaps you could share what you are actually trying to do with the user remapping? Understanding that and the UIDs within and outside the container might make it clearer? IIRC the syncthing user is a "system" user so should have a UID somewhere between ~100 - 999. So let's lay out what I think your conf is meant to do. To simplify, let's just look at your user mapping (note my understanding is from the lxc.container.conf man page so perhaps I've misunderstood something?).

lxc.idmap = u 0 100000 999
lxc.idmap = u 999 1000 1
lxc.idmap = u 1000 101000 64535

So the first line is mapping container UIDs from 0-999 to host UIDs 100000-100999. Then the next line is mapping container UID 999 to host UID 1000. Then the final line is mapping container UIDs 1000-65535 to host UIDs 101000-166535. I wonder if the internal UID 999 being mapped twice is causing the issue? I would expect the later mapping would overwrite the former, but perhaps not?

Hm, the error is due to syncthing not able to reach its home directory. Wonder why this can happen.

SYNC syncthing[1077]: WARNING: Failure on home directory: mkdir /home/syncthing/.config/syncthing: permission denied

Thank you for the extensive answer. As far as I can see this doesn't have to do with Turnkey or Syncthing specifically.

What I want to achieve is that all containers can use a common shared bind mount as the same user (Host owner is uid 5000, container share user - in this case syncthing user - also modified to be user #5000) to avoid any ownership issues. The problem is that when I do the mapping (below - it has to be valid, as it kinda works) ownerships go wrong.

On Turnkey/Debian: all container folders owned by mapped user go to nobody
On Alpine (just wanted to try): only /home folder of user is mapped to nobody

Otherwise the mapping works: in the bind mount, I can see the files owned by respective #5000 user of Guest/Host.

I wish I'd know what the problem is...

lxc.idmap: u 0 100000 5000
lxc.idmap: g 0 100000 5000
lxc.idmap: u 5000 5000 1
lxc.idmap: g 5000 5000 1
lxc.idmap: u 5001 105001 60535
lxc.idmap: g 5001 105001 60535

Glad you got it working as desired.

FWIW; whilst I think your solution is vastly superior, (in case you're not already) it's worth being aware of another cool tool that is suitable for resolving tricky interoperability file permissions (or other desired file attribute issues). incron is sort of like cron, but is triggered by filesystem events (e.g. files being written) rather than time.

Thanks, this sounds indeed powerful - and way more efficient for things one would use a watcher or a cronjob for.

Just some additional notes after I realised this also has an effect: always check "Ignore permissions" in Syncthing Advanced Settings, in each folder. This resolves all remaining permissions-related issues.