Dani M.'s picture

Sorry for the long post... I've summarized the proposals at the end (see SUMMARY OF PROPOSALS below), but I previously explain/analyze of the issues, and propose detailed fixes (so that they are easier to implement if "accepted").

I'm opening this new thread with different (and I think better) suggestions that the one I recently made in:

http://www.turnkeylinux.org/forum/general/20150203/why-not-vbox-default-...

There is a rationale there on the need of making the Virtualbox use case for novice users more "turnkey" (than it is now, apparently).

 

SUMMARY OF CURRENT ALTERNATIVES FOR TKLX ON VBOX:

Current alternatives for running TKLX if one has VBOX installed:

Alt. 0, ISO: Download the .ISO. Manually create a new VM following a tutorial, attach to it the .ISO, launch and install from .ISO (installing is an "extra step").

Alt. 1, default ZIP: Download the default ZIP  (containing .vmdk and .vmx) . Manually create a new VM following a tutorial, and attach to it the .vmdk extracted from the ZIP. This looks like the currently "recommended" for VBOX. Not "evident", and documentation al little bit outdated.

Alt. 2, OVF: Download the "OVF" ZIP ("OVF" link). VBOX assotiates with .ovf, so this is straighforward: extract the ZIP and double click on the .OVF.  VBOX opens a confirmation window with an "Import" button. This Window shows the characteristics of the VM that is going to be imported (name, OS type, RAM, etc), permitting adjusting if desired, prior to import. Import is in fact creating the VM folder and making a "clone" of the .vmdk (with a new and unique UUID, correctly reflected on the .vbox file that it creates). 

This Alt. 2 is currently IMO by far the simpler for the user (specially a novice, but also handy for everyone): it does not require to create a VM and configure it (need to ensure that PAE/NX enabled, change network settings to bridge, decide a RAM size, decide/know OS type and 32/64bit, decide a name for the VM...). Alt. 2 is simply more "turnkey".

... But alt.2 has currently an issue on 64 bit TKLX (current default architecture).

 

CURRENT ISSUES WITH 64bit OVF IMPORT IN VBOX  AND "MANUAL" WORKAROUNDS

My tests for alt. 2 of .ovf import have given the following results (tested with last VBOX branch, 4.3.x, on Mac OS X 64bit host, Win7 64bit host, and XP 32bit host):

- It works fine with the "to be deprecated" TKLX 32bit. Unzip, double click and go... Great!

- It is "broken" for "currently pushed option" TKLX 64bit (of course, this issue only applies on 64bit hosts). It imports apparently "fine", but when starting the imported machine, after Grub, it gets hanged on "Loading initial ramdisk" (without further error).

First solution/Workaround for 64bit: when importing from the OVF, the VM is "detected" as "Other/Unknown" (being in fact implicit that it is 32 bit). If one just changes the dropdown menu to "Other/Unkown (64-bit)" before importing (or after importing, in the VM configuration), things work fine.

Note:

For sure, the imported VM's also works if setting the OS Type to "Linux/Debian (64-bit)". However, this is not strictly needed (both for 32 and 64 bit): no need to specify that it is a Debian, nor even Linux. Current VBOX "only" seems needing "a hint on 64bit" (whatever it is).

An additional "minor" issue when importing (both in 32bit and 64bit) is that the proposed name on the Wirzard for the VM is "vm" instead of a more descriptive one (like e.g. TURNKEY LAMP seen on the .ovf file)... So the user clicking "next, next" will create a VM named "vm" in the VBOX interface. Moreover, a futur import of a different TKLX appliance will "clash" by default (same name), confusing a novice user.

This 2 issues (change OS type when importing 64bit TKLX that is currently pushed, and change name to a more significative one) can be easily documented. This gives globally by far a more easy procedure that following current suggestions of "starting a new VBOX VM" (alt 0 and alt 1) seen on tutorial http://www.turnkeylinux.org/docs/installation-appliances-virtualbox.

But it is still not as "turnkey" as it should (as it easily could).

 

FIXES FOR THE OVF ISSUES

I've been looking on fixes on this: fixing the OVF file, so that it proposes the correct stuff in the Import Wizard, so that the user only has to click "Import" (not typing, not configuring... turnkey, in a word).

These are my findings.

Here is the relevant extract from the current .OVF file generated for Turnkey Lamp (as an example of appliance):

[...]
​<VirtualSystem ovf:id="vm">
  <Info>A virtual machine</Info>
  <Name>TURNKEY LAMP</Name>
  <OperatingSystemSection ovf:id="1" vmw:osType="otherGuest">
[...]

Instead of:

ovf:id="vm"

it should better say something more explicit... For example, the same contents of the "Name" element below it. In this example, it could be:

ovf:id="TURNKEY LAMP"

so that a better name is detected and proposed by the Wizard.

And for 64 bit TKLX (currently default and pushed arch), instead of:

ovf:id="1"

it should say (for example):

ovf:id="102"

so that a correct (64 bit) OSType is detected and proposed.

Note:

This number comes is from the (obscure) "OVF" standard specification:

http://schemas.dmtf.org/wbem/cim-html/2/CIM_OperatingSystem.html

Searching "OSType", one sees that the value is an integer, currently between 0 and 116, and it shows there the corresponding "readable values". For "1" it is effectively "Other", and for "102" it is "Other 64-Bit".

BTW, "Debian" is 95, and "Debian 64-Bit" is 96. These could be a "better fit" for current 32bit and 64 bit TKLX, but are not strictly needed for what I see.

So the configuration could be for 64 bit something like (for 32 bit, the "102" should stay as "1"):

[...]
​<VirtualSystem ovf:id="TURNKEY LAMP">
  <Info>A virtual machine</Info>
  <Name>TURNKEY LAMP</Name>
  <OperatingSystemSection ovf:id="102" vmw:osType="otherGuest">
[...]

Note:

I suppose that this part besides the 102:

vmw:osType="otherGuest"

is a "directive" for VMWare products.

With these changes, I've tested that the VBOX import on 64bit proposes the good stuff, then when pressing "Import" and then launching the VM, everything goes fine without needing any adjust. And is makes easy the import of other TKLX appliances (since no generic name "vm" is used).

For fixing this on dev-side:

I see now in the TKLX documentation that the OVF ZIP is created "with VMWare's OVFtool.".

So I guess that it should be possible to fine tune this when generating the OVF... Sorry, but reading the OVFTool user guide, I don't see clearly how (maybe it is straighforward for the dev team). In this User guide, I see mentioned the "VM ID" stuff: "If you are creating a VM, you need to specify the id in the descriptor file.", but I don't unserstand how this is done (I don't see a command line option for specifying it, and I currently don't understand from where does it take it). They do not either provide hints on how the "OSType" is specified. But they mention this, so this is for sure "doable" in a straighforward way that I don't see now.

Surely, the best option is to find out it: so that the OVFtool just sets in source these values with a more correct value and then produces a more "correct" ovf output.

If the previous is not feasible/easy, a workaround could be to do an easy post-process (e.g. a simple sed command) of the output .ovf file, in order to adjust this stuff (also requires updating the sha1 in the .mf file)... Less elegant and more "risky"...

Only possible "problem" that I can foresee on "fixing" the .OVF for VBOX compatibility is if it may break other usages/users of the "OVF flavor". If I understand well, these usages are mainly the VMWare products (home and specially professional), but they should be compliant with OVF that follow the standard (and specially if generated by their own OVFTool). It should be possible to do this in a compatible way for "everyone".

 

SOME DOCUMENTATION ISSUES AND PROPOSED FIXES

I've found some documentation issues for the "straightforward case", using the "lens" of a completely novice user (that may just know Virtualbox, or may not even know anything on virtualization, but lands to Turnkey Linux).

- The turnkey home page only clearly suggests "Try the live demo!". One could see it as the lighter and simpler way to try TKLX... But with the live demo, the user does not really see the VM itself, nor can "deploy one", nor can he configure and use it (as far as I've seen, sorry if I'm wrong). The demo is oriented to show how multiple VM's management and monitoring looks like in the Hub, but not for testing and playing with a particular VM and "seeing" what Turnkey linux is really like... If the user wants to "play" further and see a particular Turnkey machine that is not "fake", he has to contract Amazon cloud for going further (account and billing info needed, even if "in principle" it is 0 cost at the beginning). This is a giant step for casual users that just want to try TKLX. It will put away a lot of potential users (young students for sure, but also a lot of grown-up, and specially if they come from an FOSS tradition, where only the locally executed FOSS software can be trusted).

I really miss a link in the home page after the "Try the live demo!" link...

Something like: "Or try it on your computer in a few simple steps". This link will provide a guide for the current simpler, more universal, more "minimal assumptions", aka "more turnkey" way of testing it: install Virtualbox and download the OVF link of a given appliance.

A third sentence could be "Or use your current virtualization product" (being a link to the doc/builds page).

 

- In the downloaded default .ZIP and the OVF .ZIP, the included README is only a link to:

http://www.turnkeylinux.org/docs/virtualization

This document, recommends the "default VM build" for Virtualbox users. It summarizes that the OVF build is "recommended for users of high-end VMWare products (ESX, vSphere)", while it is also useful for Virtualbox...

This should be fixed on "docs/virtualization" document: describe the OVF also as good for VBOX, and even recommending it for novices (even with the current 64bit import issue that I've described before).

Or maybe the README should directly point to "docs/builds" (currently more accurate, see below) instead of "docs/virtualization".

Or maybe "docs/virtualization" should just be merged/redirected to "docs/builds".

 

- On the Website, when going to a particular appliance page (from the turnkey home page, or landing from Google), the help for choosing a flavor is quite "hidden" IMO...

A little "?" after "Xen" (seems like a help only for the Xen option).

I think that the "?" link should be after the word "BUILDS", to indicate that it helps to decide what build one should download/use.

I would even change the "?" with the explicit word "help".

Or better, I would give in the same page of each appliance a little orientation for novice users (so they don't need to go through a further "?" link and they don't need to have read something that is in another place). A short, complet, explicit and straighforward guidance, like "For novice users: install Virtualbox and download from the OVF link".

- The "?" link goes to:

http://www.turnkeylinux.org/docs/builds

This one is much more accurate than the "docs/virtualization": in the second paragraph, it corrently points to novice users to download and use Virtualbox.

But then, the (novice) user has to read quite all the document to find the places where Virtualbox is mentionned, with some quite technical jargon (very related to Virtualbox and virtualization).

The document shows the three options, but really does not give a recommendation, and it is difficult for a novice to decide.

The document explains correctly that importing the OVF is a "step further" (cloning the vmdk), so may take longer and take more space (but being 250MB VM's, I think that the simplicity of the procedure currently really pays off). This may discourage using it...

It encourages in fact the other approach, since it gives a link for further info (the tutorial for VBOX on docs/installation-appliances-virtualbox).

I think that this document should better suggest the OVF approach, and also right at the beginning of the document (where it mentions VBOX as the easy way for novice users).

- The tutorial for Virtualbox:

http://www.turnkeylinux.org/docs/installation-appliances-virtualbox

At least the document "recognizes" that it is dated (I understand that there is no time to constantly fixing/updating it).

It is a pity that a newcomer currently tends to see this as the "suggested easiest way". It should at least include a mention to the "other possibility" of importing the OVF. And even recommend it.

While my previously proposed fix for the OVF generated file is implemented by a developper (or in case it is never implemented), I would just quickly document here the workaround: when importing, the user should choose as "OS Type" "Other/Unknown (64-Bit)" (if using the 64 bit OVF, that is currently the default architecture). And in the workaround, I would also recommend the user to change the VM name from "vm" to something more explicit (and to avoid future import clashes).

 

MAYBE MERGE "DEFAULT .ZIP" AND "OVF .ZIP"

I assume that the .vmdk on these two downloads are in fact really the same, or at least perfecly compatible... And they are the bigger part of the ZIP... Quite redundant!

Doesn't it make sense to put both contents on the same .ZIP?

E.g. just adding the "little" .vmx to the current OVF zip contents. A single ZIP download (storage savings for Turnkey developpers, less confusion for the user).

The user just extracts. Then he double clicks on the .vmx if he has VMWare Player, or double clicks on the .ovf if he has Virtualbox. Or he can also import the .ovf with any VMWare product, since they support the OVF format (according to VWWare knowledge base).

I'm sorry that I don't have now at least a VMWare Player at hand to test that this works fine... But I think it should work...

 

MAYBE PROVIDE .OVA FORMAT DOWNLOAD (EVEN SUBSTITUTE THE ZIP ONES)

In fact, .OVA may be a better alternative that providing current .OVF zip file.

.OVA is just a tar of the .ovf, the .vmdk and the .mf.

Also standarized like OVF, and it is precisely designed for VM's distribution/download.

It is supported by VBOX, and in facts avoids the extra step of unzipping, then importing: it directly imports from "inside" the OVA (less time, space and garbage than unzipping).

I see that the OVFTool generator also supports generating this format.

And this format is in principle also fully supported by VMWare products (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd...).

And surely there it also avoids there the extra step of unzipping.

I think that currently using zip for ovf distribution instead of this "tar" (.ova) makes little sense: the current .zip is not compressing further the .vmdk, and the TKLX .zip or .ova size is quite small (250MB... no need to split multi-giga file downloads).

Maybe I'm missing something, but in fact I would even merge current "default .ZIP" and "OVF .ZIP" in a single .OVA (tar of the contents of the current OVF zip, or directly generated by OVFTool). And forget the ".vmx" option.

Less storage, less (unneeded) choices.

If only an OVA is provided (instead of currently 2 separated zip), the only "minor issue" seems for "VMWare Player/Worksation" (desktop products): user needs to import the OVA instead of unzipping and then using directly the .vmx/.vmdk of the current "default ZIP"... But in fact, importing from the .OVA is quite equivalent to unzipping, in terms of time/space/garbage. And the usability of importing should be quite similar (if not better: maybe VMWare products also assotiate to .OVA in desktops, like VBOX does).

With the .OVA, the recommended approach for novices gets even more streamlined, quick and clean: install Virtualbox, download and open (double-click) the OVA file, then press the "Import" button.

No unzip step needed, no stuff left behind (except the downloaded .OVA, the same as the downloaded ZIP was).

 

SUMMARY OF PROPOSALS 

  1. Fix somehow the .OVF generation so that it works for 64bit on VBOX (OSType) and provides a more relevant name (instead of current "vm"). See before for technical details.
  2. If this fix is not possible/recommended, or in any case while it gets "fixed": documment briefly the workaround (in docs/builds, and/or in docs/installation-appliances-virtualbox): "fix" these values when importing. See before for details.
  3. For novices, on TKLX website, recommend this Virtualbox with OVF method: at the start of docs/builds and even at the home page.
  4. On every appliance page on the website, put the "?" help link after "BUILDS" word. Maybe also do an explicit recommendation for novices there. Or at least a "Advice for novice users" link, linking to the relevant paragraph in a central document.
  5. (Maybe) Merge the two ZIP currently made (default and OVF) into a single one. Or maybe even better substitute them by a single OVA.

I understand that the TKLX core team has to prioritize the suggestions between multiple different issues to be fixed (both in code and documentation). But I think that I describe here are issues worth fixing them.

This is because they are addressing the use case of the less savvy user that lands to TKLX.

This user is special, since it can be more easily frightened or overwhelmed if things are not really "turnkey" for him... If the first thing he finds is a choice of downloads (besides the choice of appliances), but not a simple (and correct) hint on what is the simpler initial option he has... He may (silently) run away (maybe looking for alternative options, maybe in the Bitnami style). How many times may have this (silently) happenned?

I think that "ideollogically" it is good to defend this user, the weakest... In fact, I know that Turnkey core team completely agrees on this (this is in fact the main original point of the TKLX project).

But besides this, I think that it is wise to do it this way, since it may help to increase the adoption of Turnkey Linux (being easy for everyone is its main "selling point"). Increase the adoption both for novices, and not-so novice users. For example I do not consider me so "novice", but I really appreciate when things that can be made easy, are made easy (kudos to the TKLX project for this). I know that there are other users to atract (not Virtualbox ones), but these suggestions should not be at all imcopatible (with some "regression testing").

In any caso, I'm not talking about marketing, just talking about making it simpler (since it can be made, in this case, with very minor difficulty)...

Again, I'm sorry if there is something obvious that I'm missing.

PS: I see that the TKLX documentation Wiki is not open, so I cannot try to help and improve it myself concerning the issues that I see and describe here... Besides, even if it was open, I surely wouldn't dare without prior consent/agreement.

 

Forum: 
Jeremy Davis's picture

I think that looking to produce an OVA to replace the OVF is probably not a bad suggestion. My only concern would be to make sure that the OVA works on VMware as it should. Perhaps initially we could look to implement it as an 'additional' format with a view for the OVF to be retired. I don't really think that retiring (or merging) the VMDK (zip) is probably an option. That is one of our most popular formats (although perhaps that might change if we reworked our docs?)

As for documentation, in case you haven't realised, the documentation is actually a wiki. So if you see something that is wrong or could be better then you should be able to change it (as a logged in user).

And I think the idea of presetting the OS to Debian/Debian64 would probably be the ideal. I know that you said that it doesn't really matter (so long as the arch is right) but seeing as TurnKey basically is Debian under the hood why not set it as Debian!?

If you wanted to you could post something on the TurnKey Issue Tracker. If you do, don't bother rewriting everything; just a quick summary and a link back here (to this thread)... Although perhaps it might make sense to split of some individual points into their own 'issues'?
Dani M.'s picture

Thanks! Regarding your concerns on the OVA on VMWare: "in principle" (according to their doc) importing OVA is supported as OVF. Besides, the OVA would be generated by their own tool (the same OVFtool used now for OVF). So this looks quite safe to me. Then, any "OVF-style user" (VBOX, VMWare, ...) in principle will benefit of OVA instead of zipped OVF (avoid unneeded extra unpacking).

I'm more "frightened" by the (minor) OVF fixes I have proposed for straight VBOX 64bit compatibility (but those are independent of the OVF being outside or inside an OVA tar).

Regarding the default ZIP, I understand that it is popular (moreover, being the recommended option for newcomers... this is also the power of defaults). Besides, now I see that my merging suggestion was "naive"... I've just tested that using the .ovf with the .vmdk of the default zip does not work on VBOX. So now I see that the .vmdk is internally quite different... and the risk of breaking things if subrogating the .vmdk of one case to the other.

Regarding doc "issues", strangely I did not have "edit" links even if I was logged in, but after your suggestion for editing, I have re-logged in, and it is now "fixed". Also good to know that community doc edition is encouraged.

Some doc proposals are not doable by me (e.g. the ones concerning the pages for each App: mainly concern was the place/visibiilty of the "?" link for choosing the build/deployent method). For the other ones, I'm doing this as soon as I can (glad to contribute!), although I will not dare making any "refactoring", but try "just enough improvements".

I will also post on the issue tracker (thanks for the info and the link!). Surely I'll split this on the different "issues", you are right. In any case, with some basic document improvements that I will make, this can "survive".

Thanks again!

Jeremy Davis's picture

I've also lodged issue #304.

There is probably another issue or 2 to be pulled out of your thread, but it's a start...

Jeremy Davis's picture

Hi, I've just been doing a little more research on this (prompted by someone else requesting that the 64 bit OVF be fixed here).

So I finally got around to putting up a feature request. I also logged the specific current OVF issue as a bug. I hope to fix the bug for the v13.1 release and was thinking I'll defer the feature request (OVA) for v14.0 release. However if you wanted to see this pushed ahead it'd be great if you (or someone else with your level of expertise and understanding) were interested in having a look. I will put as much relevant info up on the feature request.

Cheers :)

Add new comment