Dashamir Hoxha's picture

 

I think that it is perfectly possible that one can install a minimal system with debootstrap, install tklpatch on it, and then apply a patch that builds TKLCore. Is there such a patch? Where can I find it?

Also I am not able to find the patches for the other appliancies, like MySQL, LAMP, etc. Why would I need them? Just to study them and copy/paste some parts to my patch. Hey, this is open source after all, isn't it? The ISO-s are not sufficient.

Thanks for your help.

Forum: 
Jeremy Davis's picture

There are only patches for community submitted appliances - not the ones developed by TKL core devs.

This is not an intentional 'close' of the open source code for TKL, but something (that is far from ideal) which just happened. Basically TKL started as a side/hobby project of core devs Alon and Liraz and as such their build instrastructure was a fairly cobbled together, poorly documented 'black-box' that has grown well beyond it's original planned usage. As the project took off there were more and more requests for appliances (more than the devs could even nearly keep up with). There was also interest from the community in contributing back with community devloped appliances. To accomodate the community, the core devs developed TKLPatch - so that community members could create and share appliances. It also allowed the devs to much more easily create official versions of the appliances as time and resources allowed. But to be clear, the TKL devs have never actually used TKLPatch to create the official appliances themselves. Thus the TKLPatches only exist for appliances created by the community, and whilst official appliances that spawned from these exist, they aren't actually built from them (i.e. an appliance built with a TKLPatch, may not be the same as an official appliance - even if the official appliance was based on that patch).

This situation is far from perfect and the core devs are acutely aware of it! They have been rebuilding their build infrastructure (to accommodate things like 64 bit) with the added plan to open up the TKL build infrastructure for some time (so that community devs and core devs are on the same playing field). I know that they have been doing a lot of work on this behind the scenes over the last year or 2 but have yet to publicly release anything (other than v13.0RC 64 & 32 bit ISOs built on the proposed new build infrastructure). One of the (many) problems with TKL is that there are so many great ideas and grand plans, but only 2 core devs with limited time and resources. It's a bit of a catch-22 really; to allow greater community involvement would be a great thing, but to focus on the groudwork for that would involve stopping other important stuff...

And yes it is open source! All the TKL specific/custom code from appliances is available from their GitHub page and/or code.turnkeylinux.org and binaries from archive.turnkeylinux.org. Upstream code is available from Debian and/or individual upstream devs (depending on where TKL gets the code from - this is detailed on each appliance page/changelog/manifest). However build/install scripts are not currently available (nor are they required to be by open source license - despite that the devs are working on it as mentioned above...)

Out of curiosity why would you want to patch Debian bootstrap to include Core when you can download Core (which is based on Debian) in a multitude of different formats already? Do you have a specific usage scenario in mind?

Dashamir Hoxha's picture

I understand all the historical reasons and that nothing is perfect, but I think that this is something that needs to be fixed. I think that TKLPatch is powerful enough to allow Core itself to be built with just debootstrap+tklpatch, and also the other official appliances. These patches (Core and official appliances) can be placed on code.turnkeylinux.org . It is good that the core devs are acutely aware of this and intend to improve it.

Is it difficult? Well, maybe it is not such a huge task as it may seem at the first sight (something like building everything from the beginning). Recently I had to debug/audit a server of mine, I asked around for help, I got feedback from many people, and finally I arrived at the solution that is described on this blog: http://dashohoxha.posterous.com/how-to-find-all-the-modifications-in-the-conf

About the reasons why these patches would be useful, I already mentioned that it could help me to build my own patch if I see how the things are done on the core patch and on the other official patches. I think that they could be useful for the other non-official patches as well.

Another reason is modularity. If you build the Core (and other appliances) with debootstrap+tklpatch, then you can easily build an i386 and an amd64 version. This could be just a simple change in the arguments of debootstrap. So, it reduces the work for the core devs. But even further, you can easily build a Core version based on Ubuntu or any other Debian derivatives (maybe with very small modifications).

For example I am having troubles with the old version of PHP in Debian. Other packages that I use have old versions as well (like phpmyadmin). Some other packages that I need don't even exist on Debian and I have to use external repositories like dotdeb in order to install them (for example php5-fpm). I would feel more comfortable if my appliance is based on Ubuntu Server 12.04 LTS, where everything that I need is there, having the right version, and I don't have to do so many tricks.

Another reason is security. Building Core from debootstrap and tklpatch is much more transparent than getting and using an ISO. If I cannot trust even a server that I have built myself, and I am not sure what I have installed and changed inside it over time, how can I trust a server built by somebody else? Especially when this server is built in a closed process,  by two overloaded devs that are too busy with many other important stuff? Don't get me wrong, there is nothing personal, this is purely philosophical. Myself I would never use such a server in production.


Jeremy Davis's picture

I like your resaoning and appreciate the thought put into your response.

And I think that your desires on using a different base (either architecture or OS) are in line with what Alon and Liraz are thinking. I recall discussions prior to them moving to a Debian Core was the original plan to provide both Debian and Ubuntu based appliances, unfortunately (or luckily depending on how you look at it) they realised that at this point that would just create a support nightmare for them (and probably me too as the forum mod).

I don't know exactly what the TKL devs have in mind but I think they plan to have some sort of open cloud based build infrastructure which anyone can use. So I think the plan is to make sure that the community have access to the exact same build infrastructure that they will be using. I guess we'll have to wait and see...

Sorry all very vague answers and don't completely address your concerns but that's the best I've got! Perhaps Alon or Liraz might drop in and give you a bit more...?

Dashamir Hoxha's picture

I have started working in creating a TKL-Core patch (assuming that the core developpers are too busy right now). I would like to install it on a chroot-ed precise system. The installation script can be something like this: https://github.com/dashohoxha/B-Translator/blob/master/TKL/install_tklcore_chroot.sh

What remains now is just to create the TKL-Core patch itself. I have already done the first step, finding the configuration differencies between a tklcore system and a freshly installed squeeze system. These are the results:

I think that it should not be very difficult to generate a tklpatch from these, it just needs some hard work and testing. Is anybody else (besides me) interested in this? Anybody willing to help? Also, any idea about improving the patch or the scripts is welcome.


Dashamir Hoxha's picture

I have already created an initial tklcore patch and installation scripts: https://github.com/dashohoxha/B-Translator/tree/master/TKL

Can somebody test it and help with improving it? For example I don't know how certssl is created on the original tklcore:

diff -rubB squeeze/etc/group core/etc/group
--- squeeze/etc/group   2013-03-27 15:49:52.917953990 +0100
+++ core/etc/group      2012-08-10 14:05:51.000000000 +0200
@@ -42,3 +42,4 @@
 ntp:x:103:
 ssh:x:104:
 shellinabox:x:105:
+certssl:x:1000:

diff -rubB squeeze/etc/gshadow core/etc/gshadow
--- squeeze/etc/gshadow 2013-03-27 15:49:53.017934368 +0100
+++ core/etc/gshadow    2012-08-10 14:05:51.000000000 +0200
@@ -42,3 +42,4 @@
 ntp:!::
 ssh:!::
 shellinabox:!::
+certssl:!::

Because of it, shellinabox fails to start.


Jeremy Davis's picture

Have a look at the first boot scripts on TKL Core for hints (I forget the exact path but IIRC its something like /usr/lib/inithooks/bin & /usr/lib/inithooks/firstboot)

Add new comment