Ewen's picture

Hi,

I'm hoping to build a general purpose TKL home server (on an HP Proliant Microserver n36l).

I'd like it to be able to be

  • a web / DB server - LAMP (with MariaDB probably) 
  • windows accessible file server
  • run backups (to local ext hdd) of its own content and that of local PCs
  • possibly be a media server and stream ripped (owned!) movies within the house
  • I'm also vaguely interested in running the Getting Things Done functionality which one of the appliances is optimised for.

Could people advise please on the best way to go about this?

Should I run some kind of VMWare equivalent and a bunch of TKL VMs, each being optimised for these different purposes? If so, what's the best way to go about that?

Or, should I just run a vanilla / core TKL appliance and then install the various utlities and applications on it? If so, could folk advise on how I do that? Can I get the applications here (on TKL) or do I need to venture out beyond TKL and find / download install the bits and bobs myself?

 

Thanks!
Ewen

 

Forum: 
Jeremy Davis's picture

I highly recommend installing Proxmox VE on your bare metal, It's an open source Debian based hypervisor that provides a powerful web UI and CLI tools. It provides KVM (for full VMs of any OS) and LXC (for Linux OS containers). The TurnKey Linux library (in LXC format) is available for download via the UI (or you can install from ISO to a full VM if you wish).

Then install the various appliances you want as LXC containers. There is some redundancy as each is running it's own full OS, but because LXC leverages that host's kernel, you get near enough to bare metal performance, that the only real "wastage" is disk space. Besides, I personally like the redundancy. It means that I can make tweaks to one "service" (i.e. server) without affecting any others.

Ewen's picture

Thanks Jeremy!

That sounds great. I hadn't heard of Proxmox VE but it looks good from an initial scan of their home page.  I have my weekend tinkering sorted :D

I will report back with how I get on (once I do get on) to keep the thread complete.

 

Domhnall Currie's picture

Proxmox looks really cool and so do the benefits of LXC, but I have not been able to get either of them to run for me for love nor money!  I've set up probably a dozen TKL servers and they always go off without a hitch, but I've pulled my hair out with Proxmox and LXC.  I'm not saying don't use them because they both have tons of users and get great reviews, just don't get discouraged if you're just starting out.  :)  Hang with it and don't be afraid to ask the guys here for help.

Ewen's picture

Thanks Domhall,

I will be patient and see how it goes with Proxmox! I have some RAM arriving tomorrow (just to up it to 8GB from 2GB). Once that's installed I'll set to.

Fingers crossed that ProxMox behaves for me - I'm planning on installing that from the ISO file before then adding TKL appliances / VMs. I will report back though on how everything goes.

Thanks!
Ewen

Ewen's picture

Sorry for mis-spelling Domhnall!

 

Domhnall Currie's picture

No problem.  :)  Not sure about that microserver, but I think most HP servers with built in NICs (at least the ones I've used) had proprietary drivers which are not included with Debian.  You'll have to download them from HP or get them off your distribution disks and get Debian to recognize them.  Not difficult and you may already know that, just wasn't sure how "noob" you were.  :)

Ewen's picture

Thanks for this too Domhnall. I am 80% noob on Linux, so I think a potential unresponsive NIC due to a lack of driver would have stumped me. I will grab the driver from HP and will take a look into how to get Debian to recognise them.

I'm still fully intending to report back on how I get on with it all. As yet, this extra RAM has still to appear, so no action just yet!

Thanks again!
Ewen

Jeremy Davis's picture

Yeah unfortunately it's not uncommon for NIC hardware to have proprietary drivers (although wifi is often worse). Having said that, I just did a quick bit of research for you.

According to HP (and a newer version here, plus a few other sources that seem to confirm they used the same NIC across different versions) the "HP ProLiant MicroServer" uses a "Embedded NC107i PCI Express Gigabit Ethernet Server Adapter". A bit more searching suggests that apparently the NC107i PCIe NIC is based on a Broadcom BCM5723 chipset. According to Debian (what both TurnKey and Proxmox are based on), that chipset uses the 't3' driver. And the 't3' driver (t3.ko) is bundled in the kernel.

So my quick searching suggests that the NIC should "just work".... Fingers crossed!

If for some reason it doesn't, then before you start trying to install drivers from all over the internet, I urge you to check the "non-free" firmware available from Debian. You'll need to download the deb file onto a USB, then manually mount it and install it. But lets cross that bride if/when we get to it... :)

Ewen's picture

Hi

Thanks for this Jeremy, and for all the digging and research that you've done. Sincere apologies for tardy reply on my part - I have not actually had a chance to sit down and get into the server build yet. Family issued a series of higher priority interrupts! 

I will give it a go on working on the assumption that it'll 'just work' (which would be great) and will report back.

Thanks again!

Jeremy Davis's picture

No problem at all. When you get to it...

Domhnall Currie's picture

I've had a "series of higher priority interrupts" myself.  :) 

Ewen's picture

Hi again, I've finally managed to get a look at this Proxmox / TKL installation. Jeremy - I followed your cue and went for the Proxmox install onto the HP Microserver just as it was. The install seemed to go OK, but I think there might well be an NIC / comms issue - some reasonably fundamental issue anyhow. After the install was done, the screen was filling with the same set of error messages, repeating themselves every second or two. They were:
status: { DRDY ERR }

error: { UNC } 
           blk_update_request: I/O error, dev sdb, sector 2048 op 0x0:(READ) flags 0x0 phys_seg 9 prio class 0

exception Emask 0x0 SAct 0x80000 SErr 0x0 action 0x0

irq_stat 0x40000008

failed command: READ FPDMA QUEUED

cmd 60/00:98:00:08/01:00:00:00:00/40 tag 19 ncq dma 131072 in
res 41/40:00:00:08:00/00:01:00:00:00/00 Emask 0x409 (media error) <F>

status: { DRDY ERR } 
... and repeats as above Looking at it while typing it out, I guess it might be an HDD issue? I'm not sure what to do about it though - any views or advice would be much appreciated! Thanks, Ewen  
Jeremy Davis's picture

It's definitely an I/O error and the note about "media error" certainly does suggest a HDD issue. According to what you've posted, it's having issues with whatever drive is connected as 'sdb'. Having said that, there is also a possibility that it's a symptom of a motherboard, HDD (or HDD power) cable, RAM and/or power supply issue(s). The fact that you say the error repeats over and over does at least suggest that it's not an intermittent issue (which are the worst to troubleshoot).

First up, I recommend some basic troubleshooting. Ideally, do that within the existing system. That is if you can - if it's just spamming your terminal too much, then you may need to boot from a live USB.

First thing to do is work out what drive 'sdb' is. This should help (all commands are assuming you're root, if not 'sudo su' first, or prefix each command with 'sudo'):

hdparm -I /dev/sdb

Hopefully that at least gives you enough info to recognise what the drive is. If the system isn't really usable (because of the errors spamming the terminal), then take note of the info so you can find it using a Live USB (read down for more info on that). If it is usable, then you could try installing smartmotools and doing a SMART scan of the drive:

apt update
apt install -y smartmontools
smartctl -a /dev/sdb

That should show all the SMART data about the drive and give you a clue as to the likelihood of whether it's failing. If that shows something like this:

SMART support is: Available - device has SMART capability.
SMART support is: Disabled

Then run this instead:

smartctl -s on -a /dev/sdb

If the system isn't usable, then you'll need to try with a Live USB (i.e. a Linux Live ISO "burned" to a USB). Seeing as Proxmox is based on Debian, then I'd recommend just a Debian Live image. If you don't need any "non-free" drivers, then I'd recommend either the lxde one, or the lxqt one from here. Otherwise, the current ones with "non-free" drivers are here (for future readers, double check for newer releases here).

FWIW, if a TurnKey ISO boots ok in Live mode on your hardware and has networking, then you could even use that (download Core if you don't already have a TurnKey ISO)! Any other Debian or Ubuntu based live USD would probably be fine too. If you're on Windows, then I suggest Rufus to "burn" it to a USB (don't use NetBootIn!).

Once you've booted into a live system, If it doesn't have smartmontools installed already (which is likely), install it as per above. Note though, that under the live system, the problematic drive may no longer be 'sdb'. So first check to see if it is (compare with the info from earlier):

hdparm -I /dev/sdb

If that's not it, then you'll need to try to work out what it is. This should give you all the options:

ls /dev/sd* | grep -vE [0-9]

Then use 'hdparm' as above, trying each one until you find it. Then use smartctl on it.

Domhnall Currie's picture

Does the server have a hardware RAID controller in it and did you configure that?  Or did you let linux set up software RAID using mdadm or some such?  You could possibly have some sort of conflict there if you're trying to use both.

Jeremy, on my latest HP install (reconfiguring that server we've been discussing in another thread which I'll update soon), I tried the "non-free" version of Debian and it loaded up the necessary HP drivers without a hitch.  That's a good option for a newbie if they run into any driver issues.

I'll also post this here in case anyone is searching..... doesn't really apply to you Ewen since Proxmox installed ok for you, but possibly applicable for someone running a different HP server....  Jeremy, I mentioned Proxmox giving me problems before.... again, I couldn't get it to boot for love nor money.  It booted fine on my test server, an HP Elite 8000 Tower, which is basically just one of their office desktop type machines.  My regular server is an old HP DL360 G6 I bought off eBay.  I tried everything I could figure to get it to boot with no luck, but then landed on a product called Ventoy, which I had never heard of before.  Veeeeeerrrryy interesting.  :)  It allows you to just format a USB thumb drive and then copy over as many ISO images as you'd like (other formats supported as well) and then when you boot the thumb drive, you just scroll down the menu to pick which ISO you want to boot.  I think the problem with my server is some type of GPT, MBR issue (which is maybe what caused Grub to crash on that server when I did that update discussed in the other thread).  IIRC, the default of Ventoy is an MBR boot.  I copied the Proxmox ISO on that thumb drive and it let me boot up and install with no problems at all.  I can't remember if this is why I gave up on Proxmox on the first go 'round or not, but it may have been.  Anyway, maybe that'll help some folks with funky install issues. 

Jeremy Davis's picture

Thanks for sharing about Ventoy. I'd never come across that before and it looks cool!

Also, I didn't realise that your Proxmox issue was getting the install medium (i.e. USB) to boot?! I'm guessing that you are a Windows user? If you're a Linux or Mac OSX user, you can simply write the ISO to a USB using 'dd' (a low-level CLI tool) and it should "just work". For future reference, if you are on Windows, rufus is a good option (although doesn't look at cool as Ventoy). It may be fixed now, but I recall NetBootIn is (was?) known to have issues with Debian.

Also, FWIW, the only difference between the "official" and "non-free" Debian installs is that the latter has the "non-free" Debian apt repo enabled by default and includes all of the non-free Debian firmware packages pre-installed. Usually, the only drivers likely to cause issues (on a server system) are network drivers. If you do a bit of research on the hardware you have and work out what the chipset is, then you can download the relevant driver package and install it and you should be good to go... If you need a hand with that, please let me know and I can help out.

Domhnall Currie's picture

I can't remember if the install medium not booting was my only problem originally, but it was my most recent problem.  :)  Nope, 'dd' didn't "just work".  I scoured the internet for half a day and tried everything known to man to get it to boot on the DL360 server and it just wasn't having it.  It booted fine on the HP Elite 8000 server first attempt, but nothing I tried would make it boot on the DL360.  I don't know if it was because of the RAID controller or what it was.  Ventoy finally allowed it to boot.  I still have a windows machine to run a couple old apps I still use, but remember I've been using Linux 20 years.  I know, I know, you answer so many emails you can't keep all of us straight.  LOL

Jeremy Davis's picture

Yeah that's pretty bad! Mistaking a 20 yo Linux vet for a Windows user... Please forgive me.

I wonder what it was that caused the issue. It's BIOS/UEFI must be a bit picky?! Anyway, great to hear that Ventoy resolved it!

Domhnall Currie's picture

Ewen, are you using SSD's or HDD's?  Looks like there's been a couple hiccups with SSD drivers/firmware causing some data errors.  Also, there's been a lot of issues with I/O errors resulting from floppy disk controllers configured under VM's.  I know right, who still uses floppies?!  :)  But you might have to blacklist the floppy drive controller kernel module.  A UEFI boot configuration creates a floppy controller by default which can make some hardware complain.  Don't see anything in your error message about a floppy drive, but most floppy drive controllers use the DMA channel and maybe that's what the FPDMA in your error message is referring to. 

Ewen's picture

Hi all,   Sorry for hugely-extended silence on this. I finally got back to looking at the little Microserver this week. It was indeed a dodgy disk which was causing the errors I was seeing. I removed that and Proxmox fired up properly. I've since downloaded a few TKL 'templates' to create VMs from. The first of those is a LAMP build. That went on OK, and even seems to run - despite having me involved in it! So, I just wanted to update this thread to say thanks for the input on getting me up and running. Of course, I now can't get virtual hosts on the Apache install to appear at the end of their assigned IP addresses (always goes to the main 3-option admin page for the main Apache installation). However, I will go and read up on that some more and continue the learning! Thanks again!
Jeremy Davis's picture

Awesome to hear that you now have it up and running! :)


Re Apache, as you're likely aware, you can configre Apache within Webmin. However, I personally find Webmin Apache config a bit confusing and frustrating. IMO manually editing the Apache config files (in /etc/apache2) and using the commandline utilities is much more straight forward and intuitive (YMMV as I am a CLI lover...).

If you do go that way, then the upstream Apache docs are pretty good IMO. The only thing to keep in mind is that instead of being a single monolithic config file, Debian splits the config up. E.g. "virtual host" sites go in /etc/apache2/sites-available/SITENAME.conf and ports are configured in /etc/apache2/ports.conf, etc. Separate virtual hosts files are then enabled/disabled via 'a2ensite SITENAME'/'a2dissite SITENAME' respectively. Note that config changes are only applied on restart:

systemctl restart apache2

Obviously if you're happy with Webmin, don't let me get in your way! :) And ask if you need a hand (although I may not be super helpful with Webmin...).

Add new comment