TKLPatch for developing patches (idea)

I must admit I quite like it. It'd be sort of like a TKL Dev Workbench appliance! The idea of having apt-cacher integrated would be useful for any TKL dev work I reckon. Even if you have a fast connection with unlimited downloads, its still tons faster to install a deb that is locally cached rather than have to download it. It also means less bandwidth and load on the servers mirrorring the packages which has to be worth some OSS brownie points!?

But as you suggest, making apt-cacher work nicely in the patching process (without leaving broken ISOs) may be tricky. I'm fresh out of ideas how to do it!

but without affecting its normal functionality, here's my idea. How about getting TKLPatch to check to see if apt-cacher is installed on the host system, if so then call a script to work out the appropriate 01proxy config (ie IP address) and add it prior to applying the patch and remove it afterwards?

Not sure if that's the best way but my 2c :)

Please feel welcome to hack TKLPatch itself if you need to do that to enable certain functionality. We'll definitely consider including those patches in the next version of TKLPatch so long as they don't regress the usability of the default use case.

Regarding future features (e.g., webmin module for applying patches), I would recommend we keep it simple for now and then think long and hard about anything that would require writing substantial amounts of new code. If you don't design things right you're going to get things wrong and have to throw most of it away eventually. Measure twice, cut once.

I haven't look at the docs yet as I'm finishing the other appliances. Thanks Jed, this will make me go straight to what I'm looking for. And it resolves the main issue. It's going to be real simple like this. 

Nice! :)

You do not need to touch urls, source.list or any of that. All you need to do is add a /etc/apt/apt.conf.d/01proxy file with details of your apt-cacher machine (in the this case it'll be the IP of the base TKLPatch appliance). For example this would be the contents of the 01proxy file using apt-cacher-ng defaults when the IP is xxx.xxx.xxx.xxx:

Acquire::http { Proxy “http://xxx.xxx.xxx.xxx:3142″; };

It could either exist somewhere already and be copied in or be created on the fly. It can then be simply removed with rm when done.

This will still allow the chroot environment to still use non standard repos that the host machine doesn't have. Apt-cacher-ng keeps a track of all repos it is using.

Apt-cacher-ng is the prefered option (rather than the older more RAM intensive apt-cacher). It also has a nice web interface that can be found by browising to http://xxx.xxx.xxx.xxx:3142/acng-report.html

I've been thinking in the easiest and of course functional solution to this issue of using apt-cacher. As I have to change the repo's url in the files, this has to be done with care, so that the final system is built with the correct urls. The case is more complex if you use external repos. You should be able to cache them too. But I'll work first on a solution for ubuntu's official repos. 

Yes, I'll check TKLPatch code and try to do some hacking in a way that even Alon won't know that it's have been hacked :) So don't tell him anything please. 

as it makes more sense in this particular scenario. But I've just saw this and it might be a candidate for an appliance: 

http://www.linuxjournal.com/content/presenting-squid-deb-proxy-speed-your-update-downloads

I think it's very useful if you have a big network to have a deb cache in place. I'm going to setup one for home! 

Blazing fast package downloading!!! :D

Sorry, I just got exited with my TKLPatch for TKLPatch Dev prototype :) 

Do you people think that adding a squid to the mix could be of benefit? For those patches that download tar files... 

I made a small surgery to tklpatch-chroot to add and remove the proxy reference. So far is very transparent for the final user. Maybe the creators itself don't find it mwuahahahahaha. 

I'm still looking for a good way to clean the environment. Liraz suggested in an old thread that fuser -K could resolve this issue, but in my tests, I've been unsuccessful trying to release-and-delete the .rootfs directory after an explosion in the TKL Lab... So more ideas are welcome. I plan to make an extra command to clean the environment. I also was thinking on adding a parameter to tkl commands to look files on specific places if they are not present in the current directory. That way the appliance will have a turnkey-image-base dir (under /var/tklpatch/images?) with turnkey core by default. And an output dir for the process (the shared folder) by default. Say /srv/tklpatch/patched-isos/? Or better! a command to "publish" the result, (that internally copies it to the shared folder). 

MMMM some feedback please! Thanks... 

From the beginning I was thinking of a samba share, with the same justification you are explaining, windows users. So yes, that's in the plan. I've been delaying things for the TKLCore Client to work on this one, so that I can test it enough making the other patches.

Thanks for the feedback!

I think a simple squid proxy for tarballs is a great idea, not sure how that would interact with apt-cacher-ng though? I'm pretty sure I've read a bit about that somewhere (some tweaking req'd perhaps?) but can't recall where sorry (Ubuntu forums I suspect).

It sounds like your TKLPatch hacking is going nicely!

As for your cleanup issues, sorry no clever ideas here.

I really like your idea of having a default place for base and patched images, that makes lots of sense. This may be overkill but what about also having it as a Samba share or http access to these for Win users (I suspect that many TKL users use Win desktops)? Just an idea - and probably over complicating things but may be a nice added extra?

First things first, kudo's and cheers to this TKL team and it's leadership. I've been looking at JumpBox, Bitnami and TKL and I've been impressed with the tech and your warm community spirit. Keep up the awesome work gentlemen!

I was thinking how nice it would be to install TKLPatches from WebMin as a module and then I saw this thread. Some advantages I'm seeing:

1) Users can copy local VM images and only download TKLPatches

2) Lower the barrier to entry for testers of TKLPatches as they can just go to WebMin and try things out

3) Assist users in building combination TKL deployments similar to Bitnami stacks.

Thoughts?

I must tell you that I'm actively working in this project and you can see the advance here. Some of my thoughts on this:

- A Webmin module didn't seem right, as Webmin is mostly a system administration frontend, while the whole tklpatch subject is a "product". So I decided to make this a standalone web application instead. I already started development and this weekend I'll be posting a screencast with the preview. So keep watching the thread I mentioned earlier. 

- You got the main idea: lower the barrier for testers AND developers of TKLPatches. It's interesting how this appliance is perfectly targeted for beginners and advanced users alike. For beginners: easy patching for testing, easy development of new patches. For advanced users/developers: powerful tools and commands to make your work fast. An easy way to help beginners into testing the patches.