TurnKey Linux Virtual Appliance Library

systemd sysv init compatibility mode: how it works and troubleshooting when it breaks

systemd sysv init compatibility mode is magical. That is in the sense that it tries to handle compatibility with sysv init scripts while you are distracted looking somewhere else.

When it works it works well, but when things break it makes troubleshooting more difficult. Especially if you don't understand what's going on behind the curtain.

The first thing you need to understand is that this probably doesn't do what you expect:

Bug hunting with the scientific method

I've never come across a problem I couldn't solve using the following technique / problem solving methodology, and I've come across some tough ones.


How to debug a broken cron job

I just fixed a broken cron job. It turned out it was a PATH issue. By default cron doesn't have /usr/local stuff in the PATH.

Tips on how to debug cron:

  • change the schedule for the cron job so it runs every minute. It's harder to debug something that happens infrequently.

  • Make sure syslog sends cron logs to /var/log/cron.log. On one of our servers this line was commented out for some reason:

Neat trick: invoking a Python debugger at an arbitrary point in your program

Do you find yourself occasionally wishing you could freeze a misbehaving program at an arbitrary point in time and then examining what was going on interactively?

That's exactly what the debugger is for, but sometimes it's just too much of a bother to run your program inside it, you have to set breakpoints, etc.

Well there's a really simple alternative: call the debugger from an arbitrary point in your program, like this...