George3430's picture

Hi Jeremy,

I downloaded and installed the New TurnKey Invoice Ninja version (17.2) ISO on a Proxmox vm. Everything works fine  until I do the security updates which updates the kernel from 5.10.0-19 to 5.10.0-20, then everything fails .. It seem that grub cannot find the root fs ... messages below

0.7170071 Call Trace:
0.717218] dump_stack•0x6b/0x83
0.7175101 panic•0x101/0x2d7
0.7177511 mount block root•0x301/0x317 - -
0.7180711 prepare_namespace+0x136/0x165
0.7183661 kernel init freeable•0x256/0x280 - -
0.718617] ? rest init•0xb1/0xb1 -
0.7189031 kernel init•0xa/0x10c -
0.7191531 ret from fork•0x22/0x30 - -
0.7195031 kernel Offset: 0xba00000 from 0xffffffff81000000
e: 0xffffffff80000000-0xffffffffbfffffff)
(relocation ra11g
[ 0.720061] ---[ end kernel panic - not syncing: UFS: Unable to mount root fs
on unknown-block(0,0) ]---

If I escape out of the default grub choice and select 5.10.0-19 manually it boots normally and everything works fine.. see the VM config below ...

I've installed it many times with different scsi emulations  ... nothing seems to work ... I'm Stumped ... any ideas?

Jeremy Davis's picture

Argh! What a pain! It looks like I've inadvertently reintroduced a bug that was previously fixed. The mistake I made was that when I initially realized the issue, I implemented a short term workaround (to our build infrastructure) to resolve the issue. I then rebuilt everything using this workaround to build the fixed appliances. In the meantime, one of my colleagues applied a proper fix to the build component that was breaking things. I then removed the workaround for updated builds, but I hadn't actually applied the proper fix. Doh! Gross oversight on my behalf - I guess a symptom of too many plates spinning...

Anyway, the proper fix has now been applied to the infrastructure, so it should not cause any further issues. I also have a new (improved) Invoice Ninja ISO waiting to be published (hopefully will happen within the next week).

That will resolve this issue (and others: PDFs not working properly - still, support setting API_SECRET & fix cron job), but in the meantime, you can manually fix your local VM not booting like this:

First boot into the (working) 5.10.0-19 kernel, then run the following:

apt update
apt install --reinstall -y initramfs-tools
dpkg-reconfigure linux-image-5.10.0-20-amd64

Then you should be able to reboot and it should now "just work".

Sorry about that and thanks for the bug report.

As for the other issues I noted above:

To fix PDFs working, install the packages highlighted in green on this page.

The API-SECRET only applies if you wish to use the API. If you do want to enable that, manually edit the .env file (i.e. /var/www/invoice-ninja/.env) and add a new line:


(Change 'my_api_secret' to a long, ideally random string (or at least something that would be considered a a "good password"). One way is to use mcookie, like this:

echo "API_SECRET=$(mcookie)" >> /var/www/invoice-ninja/.env

The fix for the cron job is to edit our custom 'turnkey-artisan' helper script to use the full path to runuser. I.e. edit /usr/local/bin/turnkey-artisan and change this line:

runuser $ARTISAN_USER -s /bin/bash -c "$COMMAND"

To look like this:

/usr/sbin/runuser $ARTISAN_USER -s /bin/bash -c "$COMMAND"

I.e. replace 'runuser' with '/usr/sbin/runuser'.

Hopefully that all gets you going. Apologies that it's currently so broken.

George3430's picture

Hi Jeremy


hmmm... Seems like it is not working 😐 😥 see below

root@vit-inv-ninja ~# apt update ✅
Hit:1 bullseye InRelease
Hit:2 bullseye InRelease
Hit:3 bullseye-security InRelease
Ign:4 bullseye-security InRelease
Ign:5 bullseye InRelease
Hit:6 bullseye-security Release
Hit:8 bullseye Release
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
root@vit-inv-ninja ~#
root@vit-inv-ninja ~#
root@vit-inv-ninja ~# apt install --reinstall -y initramfs-tools   ✅
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded.
Need to get 71.7 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 bullseye/main amd64 initramfs-tools all 0.140 [71.7 kB]
Fetched 71.7 kB in 0s (972 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
(Reading database ... 41104 files and directories currently installed.)
Preparing to unpack .../initramfs-tools_0.140_all.deb ...
Unpacking initramfs-tools (0.140) over (0.140) ...
Setting up initramfs-tools (0.140) ...
update-initramfs: deferring update (trigger activated)
Processing triggers for man-db (2.9.4-2) ...
Processing triggers for initramfs-tools (0.140) ...
update-initramfs: Generating /boot/initrd.img-5.10.0-19-amd64
I: The initramfs will attempt to resume from /dev/dm-1
I: (/dev/mapper/turnkey-swap_1)
I: Set the RESUME variable to override this.
Enumerating objects: 1668, done.
Counting objects: 100% (1668/1668), done.
Delta compression using up to 2 threads
Compressing objects: 100% (1042/1042), done.
Writing objects: 100% (1668/1668), done.
Total 1668 (delta 101), reused 1668 (delta 101), pack-reused 0
root@vit-inv-ninja ~#
root@vit-inv-ninja ~#
root@vit-inv-ninja ~# dpkg-reconfigure linux-image-5.10.0-20-amd64 ✅
⛔⛔⛔  dpkg-query: package 'linux-image-5.10.0-20-amd64' is not installed and no information is available  ⛔⛔⛔
Use dpkg --info (= dpkg-deb --info) to examine archive files.
/usr/sbin/dpkg-reconfigure: linux-image-5.10.0-20-amd64 is not installed
root@vit-inv-ninja ~#

I then rebooted  ... and the same issue prevailed ... 😣😥

I think I'll wait for the updated ISO ...

George V

Jeremy Davis's picture

It looks like you don't have linux-image-5.10.0-20-amd64 installed?! That's inconsistent with what you reported earlier. Although I just checked and the latest kernel package is actually linux-image-5.10.0-21-amd64, so perhaps that's the one you have installed? To find out what kernel packages you have installed, run this:

dpkg-query -l | grep linux-image

On my test system, that returns:

ii  linux-image-5.10.0-21-amd64    5.10.162-1                              amd64        Linux 5.10 for 64-bit PCs (signed)
ii  linux-image-amd64              5.10.162-1                              amd64        Linux for 64-bit PCs (meta-package)

So I only have 'linux-image-5.10.0-21-amd64' installed. Use that package name with the 'dpkg-reconfigure' command. I.e. in my case, I would run:

dpkg-reconfigure linux-image-5.10.0-21-amd64

Hopefully that gets you going?

George3430's picture

Hey Jeremy,

You nailed!! .. It never occurred never occurred to me to actually check which kernel was installed ..

Just followed the instructions blindly 😳🤓 anyhow she is alive !!!


Add new comment