MediaServer v16.1 RAM Usage and Proxmox

To be honest, I'm not sure.

I have a Media Server instance running on my Proxmox server too. Mine is an LXC container and only has 4GB assigned to it and idles at 1GB usage. Mine isn't backed by ZFS. I have the OS filesystem running from a thin LVM volume on a SSD, with the media storage provided by a number of WD red HDDs in a singe logical volume, bind mounted into my Media Server instance at /srv/storage.

Neither my TurnKey Media Server nor Proxmox are the latest release, which may or may not have any relevance.

I'd be inclined to try to pin down what is actually chewing up the RAM. From within the VM/container run:

free -h

For example, here's my Media Server:

              total        used        free      shared  buff/cache   available
Mem:          4.0Gi       978Mi        56Mi        16Mi       3.0Gi       3.0Gi
Swap:         6.0Gi       349Mi       5.7Gi

Note that you need to read the 'available' column. As you can see buffer/cache is "using" most of my RAM leaving only 56MB 'free', but that would be cleared immediately if it's required, so there is actually 3GB available.

If yours is showing lots of free RAM internally, but not within Proxmox, then I'm not sure what's going on there. Personally, I'd be inclined to ask on the Proxmox forums. I suspect that they'd have more insight into what might cause that. If you do post over there, please feel free to post a link to that here.

As to not responding to the Promxox interface for shutdown/restart, it depends. If yours is also an LXC container, then I'm not clear what could cause that. If you have installed to a "proper" VM (i.e. KVM) then there is more that can go wrong there, but generally it should "just work". I vaguely recall someone else having an issue with that ages ago and it was some setting within Proxmox that they changed that fixed it. If yours is the same case, then IIRC that was a setting about using ACPI calls?! If Proxmox is sending ACPI calls, then TurnKey should respond to those. However, there is also a way it can be handled via the QEMU guest agent. And TurnKey doesn't include the agent by default, so installing that is a possible alternate route.

If you want to install the qemu-guest-agent package, do this (includes a reboot):

apt update && apt install -y qemu-guest-agent && reboot && exit

Please let me know if that helps.

Whilst the RAM is still in use by buffer/cache and is not "free", if you look at the 'available' column, it is still available for use by other apps. So there isn't actually any issue there. If something needed to use the RAM, it would be instantly freed. Why the data remains cached in RAM after writing to disk I'm not sure, but what you've shown here shows that there isn't actually anything wrong or any issue here.

And it sounds like installing the qemu-guest-agent solved the PVE UI shutdown/restart issue?

Ran the free -h command prior to starting a transfer.

Then several times during a large transfer.

The internal count jives with proxmox on the way up. I waited and once the transfer completed, I watched the lights continue go (indicating the write is carrying on). Once the lights were off, I checked it again.

Then I looked into it and found a Debian command to clear the cache/buff.

Proxmox jives with the internal measurements after the cache clear. So Proxmox appears to be reading accurately.

root@mediaserver ~# free -h
              total        used        free      shared  buff/cache   available
Mem:           47Gi       365Mi        46Gi        10Mi       255Mi        46Gi
Swap:         5.9Gi          0B       5.9Gi
root@mediaserver ~# free -h
              total        used        free      shared  buff/cache   available
Mem:           47Gi       378Mi        42Gi        10Mi       4.6Gi        46Gi
Swap:         5.9Gi          0B       5.9Gi
root@mediaserver ~# free -h
              total        used        free      shared  buff/cache   available
Mem:           47Gi       379Mi        35Gi        10Mi        11Gi        46Gi
Swap:         5.9Gi          0B       5.9Gi
root@mediaserver ~# free -h
              total        used        free      shared  buff/cache   available
Mem:           47Gi       377Mi        32Gi        10Mi        14Gi        46Gi
Swap:         5.9Gi          0B       5.9Gi
root@mediaserver ~# free -h
              total        used        free      shared  buff/cache   available
Mem:           47Gi       429Mi        30Gi        10Mi        16Gi        46Gi
Swap:         5.9Gi          0B       5.9Gi
root@mediaserver ~# sync; echo 3 > /proc/sys/vm/drop_caches
root@mediaserver ~# free -h
              total        used        free      shared  buff/cache   available
Mem:           47Gi       425Mi        46Gi        10Mi       150Mi        46Gi
Swap:         5.9Gi          0B       5.9Gi

In so far as the power off/restart command issue: I'm not sure how to see what command Proxmox sends to the VM. I've got qemu agent disabled. I think I recall reading that thread yesterday. The VM is KVM. I have not dived into LXC at all and a quick look at the how-to, I've definitely not done that.

I've got ACPI enabled and verified packages installed on turnkey. Went ahead and installed QEMU on the mediaserver and enabled it on proxmox. QEMU agent works.

If you want to make it so Promxox sees (more closely) what the actual available RAM is, then you could write a script for that!? Something along the lines of checking the 'free' vs 'available' and if the difference is too high, runs the commands you posted before (i.e. 'sync; echo 3 > /proc/sys/vm/drop_caches'). Then run that script as a cron job.

It's a bit hacky, and as already discussed the issue is only cosmetic, but it should be fairly easy to achieve your desired outcome. If you're unfamiliar with bash, now is probably a great time to learn a bit! :)

As a hint, you can get the free and available RAM (in kb) like this:

free=$(grep ^MemFree /proc/meminfo | tr -s ' ' | cut -d' ' -f2)
avail=$(grep ^MemAvail /proc/meminfo | tr -s ' ' | cut -d' ' -f2)

And if your script is /usr/local/bin/clear-free-ram, then you can make a cron job that will run every 5 minutes like this:

cat > /etc/cron.d/clear-free-ram <

Whilst the RAM is still in use by buffer/cache and is not "free", if you look at the 'available' column, it is still available for use by other apps.

I get it isn't bad; the memory is there and available.

But it does give me a misconception that the VM is under provisioned on memory when it isn't and makes it hard to judge performance when the main dashboard is, effectively, a lie. But reading around further last night, knowing what I know now, I see this is a long-standing complaint.

Guess I'll test and see how long it tries to hold that data. Maybe after an hour or two, it'll drop it. Otherwise, after doing uploads, I'll just clear it manually of clean data.

And it sounds like installing the qemu-guest-agent solved the PVE UI shutdown/restart issue?

Apologies. My final sentence wasn't clear. Yes. QEMU agent resolved the issue.

Good luck with it all and please don't hesitate to post back with any other questions and/or feedback you have.

Yeah.. it is 'cosmetic' for the most part. Need to read up on Proxmox for scripting that sort of thing.

Put it on the list of things to do. After figuring out this VNC issue.

Thanks for the assist.

Donation via paypal!