alex yarbrough's picture

I will try to keep this working environment is ESXi. My VM started out as an installation from the TKL Core ISO, which gave me a nice slim Debian 9.4 box. Now I have installed and configured everything that I want and I am ready to start developing my app.

Of course for distributing the virtual machine, exporting to OVA is easy enough but that will preserve my root password and other specific things about my test environment. I would like to be able to publish my appliance to a .ISO so that an end user may choose their own root password and other things like partition layout and all the other steps you would have to go through the installation wizard. This will also make it easy to install to a physical machine or other platform not supporting OVA.

What do people use for that kind of stuff these days, in the fragmented absense of remasterys, respin, pinguy builder, etc, all these tools seem to require a full blown desktop environment to work and are of poor quality. Does TKL provide such a tool?

I don't want a "live CD" per se, just a way to redistribute and install a configured and working debian stretch system in ISO format.

All ideas and suggestions welcome, thanks

Jeremy Davis's picture

Probably the "preferred" way would be to use the same tools we do. I.e. TKLDev. There is a doc page that provides a bit more info and links to further docs.

Essentially, that path will require scripting the entire build process including regenerating "secrets" and passwords. Although depending on what your appliance includes, much of the heavy lifting is already done for you (via our existing build code). So you can leverage what we've done already. The other advantage of going that route will mean that you could distribute your new appliance in any of our supported build types (not just OVA and/or ISO).

If you're interested in that approach, are using only open source components and think that there may be value to a broader audience, then I'm also happy to further discuss making your appliance part of the official TurnKey library. Either way though, I'm more than happy to give you some guidance and advice, even if adding it to the library isn't an option.

If that seems like "too much" and if you're ok with distributing as an OVA (or VM - rather than ISO) then another (easier) option would be to to just configure the existing firstboot scripts (i.e. inithooks) to rerun on (next) firstboot. That simply requires you to change the value of RUN_FIRSTBOOT in /etc/default/inithooks to true (see here).

To be complete, that would still require you to write scripts to regenerate any "secrets"/passwords on firstboot for any additional software which you install (which aren't already covered by the existing scripts).

If you wanted to create an ISO, then that is still possible, although TBH, I've never done it, so can't provide much insight. There is an old blog post by Alon, but it's quite dated so I'm not sure how much of it would still work... Regardless, it should still be possible, but would likely require a bit of research and trial-and-error.

Jeremy Davis's picture

As you guessed, if you want to use TKLDev and do it "the right way", you'll need to recreate what you've done to date it a series of scripts and/or overlay files. There isn't really anyway around that if you want to use TKLDev.

On the upside, that will make longer term maintenance easier and more robust. And will also allow others to understand what you've done (and the potential implications of that), assuming you wish to share. But how useful that will be will depend on your longer term plans with your project and who your intended market is.

If it's simply a "one off" that you only plan to do just this once, then that may indeed be overkill. Simply resetting the inithooks to re-run on next boot and creating an OVA may indeed be your best bet - assuming that fits your distribution plans. Also, I'm sure it would be possible to generate an ISO from your current OS, I'm just not sure how you'd go about it...

If you plan to maintain your custom appliance into the future (and/or wish to distribute via alternate build types), I would argue that whilst it'll certainly be more work in the short term, porting to TKLDev will create less work in the long term. Updated versions of your custom appliance can simply be rebuilt whenever you fancy (without or without additional tweaks). You can also keep much better track of changes that you've made so if/when things change radically, you can just adjust your changes to suit.

I'm not 100% sure how it'd go, but depending on what you've changed and how, creating a TKLBAM backup, may assist you to checkout at least some of the changes that you've made.

Sorry that I don't have any better news for you, but if you have any more specific questions, please feel free to ask.

Jeremy Davis's picture

I've been playing with TurnKey now for about 8 years and it's been my main focus for about 5 years. I don't have any plans of doing anything else anytime soon. I continue to really enjoy it. The only thing I wish is that I had more time to play with improving our custom tools and developing new appliances...

But to get back on topic; the way I usually develop a new appliance is start with the closest appliance to my desired end point (for example the LAMP appliance). I clone the appropriate repo to my TKLDev products directory. E.g.:

cd products
git clone

I'll then script (and/or overlay) as much as I can be sure should "just work". That may be as little as only downloading the desired application. Or as much as the whole install.

As you probably already realise, install script(s) go in conf.d/. Note that they need to be executable. By convention, unless it's particularly complex install, downloads go in conf.d/download and the scripted installation steps in conf.d/main. Any overlay files go into the overlay/ directory - relative to / of the installed system. E.g. a file destined to be /etc/something/conf would go to overlay/etc/something/conf. Any dependencies that are installable via apt can be added to the plan (plan/main).

Then I'll build all the way to ISO and install that ISO as a VM. Then within that VM, I'll continue the installation. I keep really detailed notes of what I do, as much as possible copy/pasting commands I run back into my conf script(s) in my TKLDev. I also tend to make lots of notes (as comments in my conf script(s)). I do have a distinct advantage as I know the TurnKey library fairly well, so I'll often rob snippets of code from other appliances that do something similar to what I'm trying to achieve. If you care to share what you're up to, I'm more than happy to share some of my knowledge if it'll help. I also often develop and test new appliance config code within my temporary VM, then either copy paste back into a conf script, or scp overlay files back to my TKLDev. Be sure to check you've got everything you need from the VM before destroying though! I've made that mistake a few times!

Once I have everything working as intended (or at least a significant amount), I'll go back and rebuild my ISO, rinse and repeat until I'm happy with the end result. I can certainly see how the sandbox would be handy for some people, but personally I never used it.

That may sound a bit convoluted, but TBH, I really enjoy building new apps so mostly I don't mind too much. I often have multiple things on the go (in multiple TKLDevs), so waiting for things to happen is rarely a big deal for me. The only time where it starts to get a bit frustrating is when I think I'm pretty much done and then just need to do some final tweaks. I've been caught out a few times, thinking that I'm done, without doing a final test of some last minute code I've added, only to find it fails later, or doesn't quite work as intended.

With regard to building (v15.x apps) on v14.2 TKLDev that should be possible (in fact it's how we did our early v15.0 development). In the unlikely event you (or someone else) also want to build v14.x apps, I'll also cover that.

To build v15.x on a v14.2 TKLDev, you'll need to set the base Debian distro release to 'debian/stretch'and you'll also need to download the Stretch bootstrap from the TurnKey mirror and unpack it to the relevant place. Actually, I've just tweaked the tkldev-setup script to support that. See here. I haven't merged that into the TurnKey repo yet, but until I do, you can update the script on a v14.2 TKLDev like this:

wget $URL -O /$SCRIPT

(Anyone coming along later after I've merged, just needs to replace 'JedMeister...' in the URL with 'turnkeylinux-apps/tkldev/tree/master/overlay/$SCRIPT').

Then set the RELEASE environmental variable, (re)run the setup script, then make your appliance. This should do the trick:

export RELEASE=debian/stretch

TBH, I don't have a v14.2 TKLDev handy, so I haven't tested it, but it should work. If not, please post the full error output and I'll tweak if need be.

If OTOH you want to build v14.x based appliance (unlikely, but just in case), skip the above (or reset RELEASE to debian/jessie). Then you'll need to checkout the 'v14.x' branch of common (master is the default and that's been updated for Debian 9/Stretch). I'd also suggest that you use a v14.2 TKLDev. So something like this:

cd common
git pull
git checkout v14.x

Hope that helps...

alex yarbrough's picture

Thanks for the extensive info Jeremy. The appliance is in simplest terms a resource monitor/notifier, but it will also contain a mix of the capabilities of System Center, vCenter and Xen Orchestrator in that it will be able to configure virtual hosts, deploy/export/import VM templates, schedule rolling snapshots of VMs and email reports of all the goings on in the hardware in my company. I currently don't know of a product that covers ESX, Xen and Hyper-V all in one, and I already have the nuts and bolts of some of these things working in the native product shells. The piece I am missing is combining it all into an appliance with a unified GUI so that I can centralize things and make administering the infrastructure in my company both pretty and noob accessible/auditable.

I'm good with scripting, automation and web development, but not quite on the level of authoring linux ISOs. It's not urgent but I do see a need for that overlay file system so I'm not sure how I will be able to press onward without Deck and the sandbox working in 15.x TKLDEv.

Jeremy Davis's picture

The sandbox is only designed to be used as an "on the fly" early development mechanism for adding additional bits, without needing to rebuild from scratch again. I.e. it's only intended as a development aid to speed up initial development, not as a long term place to add overlay files. Note that anything that you add to the sandbox is deleted next time you run make clean!

The proper long term place to add overlay files is the overlay/ directory of your appliance build code. E.g. The Apache vhost config file for the Mibew appliance needs to go in /etc/apache2/sites-avaialble/mibew.conf in the built appliance, so we add it to overlay/etc/apache2/sites-available/mibew.conf.

I hope that makes sense.

Add new comment