Jer's picture

I'm trying to build a simple ubuntu install without xorg. I don't want a netinstall but rather to move the squashed filesystem to disk after d-i sets up. The "live install" for ubuntu requires x. So neither install methods are working for me.

After 3 weeks of research and failed builds, I found the di-live script, and it appears to be exactly what I'm needing. In a different forum I found the creator of di-live suggested future questions be asked here. So here I am!

I'm able to debootstrap (or cdebootstrap) in a chrooted envorionment, squash it, create the iso, etc.. When I launch nothing comes up for d-i. If I use the di-live script, it installs the squashfs, but never launches d-i.

I've set the iso up like examples I've found in your repository (wheezy-core) and like a typical ubuntu install cd.. Nothings working.

I'd put this on the ubuntu forum, but questions like this don't get answered (that I've seen), so I'm turning here for help.

Does anyone know how I can get a base install transfered to disk after di sets up? I assume di-live will work, I just dont know what I'm missing - package or setup wise..

 

Forum: 
Jeremy Davis's picture

It's built to produce a headless Debian Wheezy install but Ubuntu is close and it's all open source so you can tweak to your heart's content! :)

I know that it's not directly answering your question, it's just that I don't know enough about di-live to be of any real help and the guys that would (Alon and Liraz) are pretty flat out ATM and probably won't have time to help you out anytime soon...

Jer's picture

Thank you for responding! I've seen posts from you on other sites in the past (don't remember where).. you seemed very helpful. I was surprised to see you here, and as such a regular poster - plus we share the same first name. :)

I figured you folks had your hands full.. but was hoping it was something simple everyone already knew except me, so the resolution wouldn't take to much of anyones time.

It seems like di-live is dead on - but I'm not using it correctly, or am missing some package during bootstrap/build. It installs the squashfs wonderfully and bypasses ubiquity - to which I'm very grateful to Alon for illustrating it can be done.. I was about to give up. It looks like getting the di frontend to fire up is the last hurdle.. so close, but I can't get it..

So, currently I'm trying to chase down errors with di-live.. it's easier to read / follow than the initrd..

I take a look at the TKLDev too! Thank you!

Jeremy Davis's picture

I'm not sure where else you might have seen me either; although I get about a bit...! This is my home though! :)

I'm guessing that you've probably already seen it but the di-live source is on GitHub here.

As I said I don't know that much from a technical perspective but I've had a bit more of a dig and I suspect that a missing piece of the puzzle for you is that we use a slightly modified busybox/busybox-initrmfs. It's a bit of a Frankenstein with Debian busybox; but the Ubuntu initrmfs creation script. Unfortunately we don't have the source publicly available anywhere as Alon modifies the current versions (from the respective OS) when we do a new release. However the (Debian) package is available from our repo (Wheezy version here). Also if you think it may be of value I can ask Alon for the source for you. He probably has it around somewhere...

Also AFAIK you'll need Casper - we forked it from Ubuntu but I'm not sure if it has been modified at all... (source here and Wheezy debs here).

Finally if you have a look through the 'boot' plan you can see what packages are installed for the boot process. FYI if you have a look through TKLDev & docs you'll see that that this file is essentially a list of packages to install which uses the C format as it is processed with CPP (C Pre-Processor).

Jer's picture

Yep, I found di-live-master at git about 3 days ago.

One of the ways I'm narrowing down issues is grepping (inverted) package lists between my build and ubuntu builds, then eventually your wheezy-core. I did notice that they use "busybox-static" and the wheezy build used just busybox.

This may help.. I noticed the wheezy build had a directory "base-install" when I was in /usr/lib. From the ubuntu perspective all I can find is "base-installer", "debian-installer" and the "live-installer", none of which are available via repos (have to install in jail via dpkg which leads me down a dependency chain that blows). None of that stuff is in ubuntu docs, guides or man pages though, making me think they build it into the init img when it's compiled on the host.

It's hard to tell if casper is only responsible for mounting the squashfs as live.. or if it also plays a part in moving the fs over. However I'm keeping it in till I get a working build, then will slim it down from there (I don't need a live environment).

The boot process seems to be going fine in so far as getting to a menu. It correctly kicks off options on the boot line, it's what's in the init (I think). There's examples for a net install, and examples for a full live install. The live install being closest to what I'm looking for (net doesn't look for squashfs), but once you get to the live init it's deeply intertwined with casper then x, then ubiquity. X and ubiquity obviously requires a ton of stuff.. So the idea of di-live intervening with a base live install init sounds like it would work.

That combo scrolls for max systemd child processes after moving the fs over, but never kicks off the di frontend.

I can tell a net install to point at the pool for packages, and slim the pool down, but with the squashed file system on the same disk, it's heavy and nothing in the init looks for the squashfs anyway, it builds via bootstrapping the target. So with the di packages, and the base+standard packages in the pool it's bloated.

I noticed in your boot plan upstart is called. That's a whole 'nother mystery. Systemd should work, however some current beta flavors do pull in upstart (some don't). I've tried with and without.

I'll keep poking at it.. and have the TKLDev to look at, thank you again!

 

Jeremy Davis's picture

Sorry I misunderstood. If you don't want to run live then I don't think that you need casper...

As for the 'plan's; like I said they are the lists of packages that TKLDev uses to build the OS and are processed by CPP. So the boot plan only installs upstart if the base OS is Ubuntu (the '#' in C isn't a comment...); if it's Debian (as it is always for us at the moment) it installs sysvinit (which we'll have to update to systemd for Jessie based release). TBH I haven't had much to do with Ubuntu since TKL switched to a Debian base so I'm not even sure whether Ubuntu still even uses it?! The references to Ubuntu stuff still in our code relates to back when we based on Ubuntu. The first TurnKey release was on Hardy; then v11.x was Lucid based; since v12.0 though TurnKey has been built on Debian (v12.x = Squeeze; v13.x = Wheezy; v14.x will be Jessie).

Also FWIW this is the what TKLDev (actually Fab specifically) uses to build our bootstrap. Despite the name, the bootstrap that we create/use actually won't boot as it has no kernel (it needs to install the boot plan first). The reasoning behind that the tools are built to be arch (and to a degree even OS) agnostic and also aimed at build targets that do not neccessarily need a kernel (e.g. OpenVZ and LXC containers).

Anyway although you are not building for Debian, personally I'd go the other way (to the way you're doing it). I'd start with building TKL Core (on TKLDev) and once you can follow the build logic built into TKLDev, then pare back the build process to what you want (Core is already fairly minimal, but I suspect that there is stuff in that you wouldn't want). Also I'm sure that you could bend it to build Ubuntu again (instead of Debian), although TBH since I swapped to Debian (I was an Ubuntu user for years) I haven't looked back. I now spend much more time using my system and less time fixing it! :)

Jer's picture

TKLDev: The result of TKLDev will effectivly mimic the bootstrap/iso creation script I made. Additionally I can't modify the bootstap from wheezy to ubuntu. Everything points to your git/repositories. I can send you the script I made which may show what I'm missing in my steps.

di-live: I can't figure out how to run this thing. :) I can't install it from your repo because I can't generate the key... It's not possible to install manually (that I can figure out). I don't remember the name of the di-live file, but it points to many locations that are not present on the bootstrap result, indicating I'm possibly missing initial packages.

di-live documentation makes it sound like d-i is kicked off via initrd (no di-live present), then with the boot param "di-live", di-live is started. When I do that it tells me "di-live main-menu not found".

Over the weekend I tried looking at different installers (not Debian's di). The prospects are bleak.

At any rate, it may help to better explain my goal...

"Create an iso that contains a text-mode installer and compressed filesystem. The installer should kick off and get user and partition info, and set up networking, then copy the compressed filesystem to disk applying the user supplied info to partion / account setup. The partitioning only needs to support ext4."

I don't need a live environment, as the compressed filesystem only contains cli utilties ('ubuntu-core' and eventually portions of 'ubuntu-standard' once I get the install figured out).

It sounds simple, and like what the purpose of the debian-installer is - but no version applies (d-i, net-install, ubiquity), and tweaking them has landed me nowhere.

So di-live; bypassing ubiquity requirements and installing squashfs, sounds correct even with the live portion which I can live with or eventually modify to suit - but I can't get it to start d-i / di-live. I missing something or some group of things, which I also can't figure out from following the logs, ugh.

It's not a lack of trying.. I've been putting in 10+ hours a day, weekends included for about a month. I'm stuck at home getting over a health issue, so I've had the time. You would think it would of worked out by now.. It has to eventually, given the infinite monkey theorem.. which is not a path I'd choose.

However I'm starting to feel bad for taking more of your time than I intended (I thought the solution might be simple) and I apologize for the long-winded post, but I thought it might help to explain. Again it may help to look at the script I wrote to automate the build, it may show what I'm missing for di-live (or d-i) if you still want to help (it's ok to not).

 

Jeremy Davis's picture

...Additionally I can't modify the bootstap from wheezy to ubuntu. Everything points to your git/repositories...

FWIW there should be no reason that I am aware of why you couldn't make an Ubuntu bootstrap. If you use the source code that I linked to above in TKLDev you should be able to build an Ubuntu bootstrap... Obviously TKLDev is built to build TKLDev but it shouldn't take much to tweak it to build any Debian derivative OS. OTTOMH I can't give you explicit directions on how to do it, but I have spent significant time playing with TKLDev (including building it from TKL Core and installing the software from source on top) and am sure it can be done relatively easily...

di-live: I can't figure out how to run this thing. :) I can't install it from your repo because I can't generate the key...

You mean the repo key (so Ubuntu doesn't complain about it being unsigned)? You should be able to force it to ignore that; or better still the key is on the Ubuntu keyring server and how to manually add it is in our docs (sorry don't have a link for you but you should be able to find it...)

...It's not possible to install manually (that I can figure out).

Not sure about how you'd do it without TKLDev but within TKLDev you can just 'fab-install' it into your base (chroot) root filesystem (or 'fab-chroot' into it and install with 'dpkg -i'). As the TKLDev tools leverage pre-existing Debian build tools I'm sure that you could do with without fab (or TKLDev) but I only have experience using the TKL tools.

TBH I just don't know enough about the guts of it all to probably be much more help... My understanding is that it initially boots to a busybox initrmfs which then launches di-live...

Anyway, it's all good. I'm sorry that I can't really give you anything more. Like I say I am unfamiliar with how all this installer stuff works under the hood and only have experience building TKL using the tools that Alon and Liraz have developed.

Like I said (or perhaps only hinted) if I were you I'd use TKLDev to build a Debian based version of what you want. Then once you can get that working, apply the same logic to port it all to an Ubuntu base. But I guess that I'm biased by the fact that I know (and love) the TKL tools... Good luck and when you figure it out it'd be great if you can post back. And/or feel free to post updates as you go. Especially on any progress you have. Who knows who might benefit from your endeavours!?

Jer's picture

I'll post back if I get anything figured out!

Add new comment