Converting a virtual disk image: VDI or VMDK to an ISO you can distribute
By Alon Swartz - 57 comments | Latest by MatthiasWhy would anyone in their right mind want to convert a VM into an ISO?
Good question, the answer for Conor Fox (who was the inspiration for this post - thanks Conor!) was to distribute his customized TurnKey PostgreSQL image so others could use it.
Distributing an ISO as opposed to a VM image allows it to be installed on any virtualization platform, as well as on bare metal, with the added bonus of running live.
I suppose that's a good enough reason, so lets get to it.
Convert VM disk to raw image and mount it
First we need to get qemu-img, a tool bundled with qemu (KVM's virtualization backend) to convert the VM disk to a raw image, and TKLPatch, the TurnKey customization mechanism to package the ISO.
If you are not using a TurnKey installation, see the TKLPatch installation notes.
apt-get install qemu apt-get install tklpatch
I'll show how to convert a VMWare VMDK image into raw disk format. If you are using a different virtualization platform such as Virtualbox, see this post on converting a VDI to a raw image.
qemu-img convert -f vmdk turnkey-core.vmdk -O raw turnkey-core.raw
Next, mount the raw disk as a loopback device.
mkdir turnkey-core.mount mount -o loop turnkey-core.raw turnkey-core.mount
GOTCHA 1: If your VM has partitions, it's a little tricker. You'll need to setup the loop device, partition mappings and finally mount the rootfs partition. You will need kpartx to setup the mappings.
loopdev=$(losetup -s -f turnkey-core.raw) apt-get install kpartx kpartx -a $loopdev # p1 refers to the first partition (rootfs) mkdir turnkey-core.mount mount /dev/mapper/$(basename $loopdev)p1 turnkey-core.mount
Extract root filesystem and tweak for ISO configuration
Now, make a copy of the root filesystem and unmount the loopback.
mkdir turnkey-core.rootfs rsync -a -t -r -S -I turnkey-core.mount/ turnkey-core.rootfs umount -d turnkey-core.mount # If your VM had partitions (GOTCHA 1): kpartx -d $loopdev losetup -d $loopdev
Because the VM is an installed system as opposed to the ISO, the file system table needs to be updated.
cat>turnkey-core.rootfs/etc/fstab<<EOF aufs / aufs rw 0 0 tmpfs /tmp tmpfs nosuid,nodev 0 0 EOF
GOTCHA 2: If your VM uses a kernel optimized for virtualization (like the one included in the TurnKey VM builds), you need to replace it with a generic kernel, and also remove vmware-tools if installed.
tklpatch-chroot turnkey-core.rootfs
# inside the chroot
apt-get update
apt-get install linux-image-generic
dpkg --purge $(dpkg-query --showformat='${Package}\n' -W 'vmware-tools*')
dpkg --purge $(dpkg-query --showformat='${Package}\n' -W '*-virtual')
exit
Generate the ISO
Finally, prepare the cdroot and generate the ISO.
tklpatch-prepare-cdroot turnkey-core.rootfs/ tklpatch-geniso turnkey-core.cdroot/
Thats it!
Bonus: By default the ISO will boot automatically. If you want to include the TurnKey bootsplash and bootmenu, extract the cdroot from a TurnKey ISO and tell tklpatch-prepare-cdroot to use it as a template.
tklpatch-extractiso turnkey-core.iso tklpatch-prepare-cdroot turnkey-core.rootfs/ turnkey-core.cdroot/ tklpatch-geniso turnkey-core.cdroot/
Ever needed to package a VM as a distributable ISO? Post a comment!
News and Security Announcements
Categories
Recent posts
-
1st Feb, 2012
-
16th Jan, 2012
-
30th Dec, 2011
-
29th Dec, 2011
-
6th Dec, 2011
- 1 of 23
- ››
Recent comments
Archive
- February 2012 (1)
- January 2012 (1)
- December 2011 (4)
- November 2011 (4)
- October 2011 (3)
- September 2011 (3)
- August 2011 (3)
- July 2011 (3)
- June 2011 (4)
- May 2011 (5)
- April 2011 (5)
- March 2011 (6)
Comments
Thanks!
Thats excellent Alon, thanks a million! It wasn't easy was it?! I'm snowed under right now but I'm really looking forward to trying this out once I'm back in "iso mode".
TKL just keeps on impressing me more all the time.
Every once in a while...
Looking forward to your feedback once you emerge from the snow...
Converting vmdk to iso
I am having a time with trying to get this to work. I have installed the core system, and all the programs, and continue to get the message (qemu-img: Could not open '/vtserver.icanreach.com/vtserver.icanreach.com-flat.vmdk'). I need some help.
Thanks.
Check your file path
and remember that Linux is case sensitive. Eg /somefolder/somefile is different to /SomeFolder/SomeFile. Also makesure the machine is not running.
Not Resolved Yet.
I made sure of the file and directory, and results are the same. I'm completely lost at what the problem is?
I'm all out of ideas sorry
I don't know enough about this process to be of any more help to you.
I'm in the midst of trying
I'm in the midst of trying this and I just now slavishly followed the "GOTCHA 2" suggestion of replacing the VM-optimal kernel with a standard kernel. However if the ISO is going to be used exclusively for VMs, would it make sense to skip this step?
Virtual kernel does not include squashfs
Also, an ISO can be installed to bare-metal, so it doesn't make much sense to include the optimized virtual kernel even if it was possible (to be complete, it is possible with a couple of workarounds but not worth the effort, and possible side-effects).
Another scenario
Hi,
Thanks so much for such interesting articles!
I propose another tipical scenario:
- You install a GNU/linux OS at secondary (non bootable) partition on your windows box.
- After try and setup a lot of things you decide to move your Linux instalation to a virtual machine....
The question is:
¿How to convert your secondary nonbootable linux partition to a virtual machine bootable disk???
Cheers,
Paco
Process should be very similar
I suppose (without thinking about it too much) that the process should be very similar, minus the VM conversion.
In general it should go something like this:
I would be interested to know if you try this, and what the outcome is. Keep us updated.
I had partial success with
I had partial success with this -- I was able to create an ISO that worked perfectly as a "Live CD". I was able to run my appliance directly off the ISO. The configuration console magically included a new "Install to Disk" option. However when I tried this it got pretty far into the installation -- including seemingly copying the rootfs to the disk -- but then failed while trying to set up grub.
But I'll leave that to another post -- I think even the creation of the "Live ISO" was a good start!
Heres another couple of "gotchas" I experienced:
(1) Alon mentioned that you may find you have multiple partitions on your source VMDK. In fact its probable that you will -- I certainly did on the Turnkey/Postgres-based VM I started with. However I was using another TKL-based VM as the "working machine" to run the commands on and I had trouble installing kpartx here. I switched to a Fedora host and was able to install kpartx ("yum install kpartx" iirc) and complete the rest of the procedure here.
(2) When I chrooted to apply the generic kernel to the rootfs, I found that I had no DNS (so apt-get couldn't find repositories). The same might happen to you depending on how your network is set up. Assuming the machine you're working on has working DNS, doing the following before you chroot should fix this:
(the desired destination of resolv.conf is likely different on a non-TKL-based rootfs).
Hope that helps! Heres the script I'm running to get as far as the chroot (its more-or-less as per Alon's directions except I've hardcoded the name of the partition in the loop device):
mount -o loop
mount -o loop turnkey-core.raw turnkey-core.mount
=
mount: you must specify the filesystem type
:-(
Raw image has partitions?
bootspash menu
This is the correct way to install the bootspash menu
tklpatch-extract-iso turnkey-core-2009.10-hardy-x86.iso
tklpatch-prepare-cdroot turnkey-core.rootfs/ turnkey-core-2009.10-hardy-x86.cdroot/
tklpatch-geniso turnkey-core-2009.10-hardy-x86.cdroot/
Remastersys?
Just throwing this out there. I used it on a server to create an ISO with all of the goodies I put on there. Never tried it on a virtual machine. Will try soon though.
I have never thought of it
This is a great idea, this way it much easier to be hypervisor independent. If you wanna go for VMware, Citrix XenServer, Hyper V, it doesn't matter. Thanks for sharing.
Acronis for basic windows users
I stumbled across this when searching to convert my VM image to iso.
Then thought of an easier way for basiic users
Take a backup using acronis (VMware is to mount tib files directly)
Have not been able to convert a tib to iso yet.
thanks
This process is quiet complex
This process is quiet complex and lengthy too. Is there any other easier method? I tried it and halfway through system got rebooted and lost all conversion I did. I had trouble mounting the secondary drive. And is there any preferable location to copy out the root file system. My operating system is windows 7 but the specification of my system is very low, running out of free space. Should I try again, is there any chance of crashing my system and losing all information?
Well, this is useful! I'm
Well, this is useful! I'm downloading a Chrome OS image (in vmdk format) and I need it on a DVD. 'Cuz I like to boot things on a physical machine. Not a virtual one.
convert a windows machine
i guess this procedure doesnt work for windows vmdk? Do you know any way to perform this on such a vm?
thx
jeff
Not that I'm aware of
The procedure detailed in the post is for Linux based VMDK's, particularly TurnKey 2000.10 VMDKs. I'm not aware of a similar approach for windows based VMDK's.
Note for future reference, the procedure will need tweaking on 11.1 VMDK's (soon to be released) as they use grub2 and LVM.
convert a windows machine
thx Alon for the clarification.
just for those who might google this page searching for the same solution as i am, there is a tool called "etboot" on the page reboot.pro.
this tool is able to convert a flat vmdk into iso (apparently no sparse vmdk).
jeff
Converting BackTrack
I accidentally downloaded the VMWare version of BackTrack. VMWare doesn't let u access the wireless card, so bad for me. anyway, this step doesn't work for me.. I've tried evrything... PLZ HELP!!!
qemu-img convert -f vmdk turnkey-core.vmdk -O raw turnkey-core.rawThe result is :-
I think its partitioned :-
Non-TurnKey VMDK's not tested.
The procedure detailed is known to work with TurnKey 2009.10 VMDKs. I'd recommend you either download the ISO or contact the folks at backtrack for help.
Thanks for the excellent
Thanks for the excellent article.
I tried this to convert .vmdk to .iso using qemu-img. My vmdk file is 1.7 GB and the resulting .raw is whoping 20 GB. Why is the raw file so big?
I suspect because .raw is flat/static image
And the TKL .vmdk is a sparse image. In other words the original HDD image contains only data (but can be up to 20GB) but the .raw file includes the empty space as well. Once you mount it and extract the filesystem you should be back to ~1.7GB.
.vmdk to .raw conversion
Maybe the .raw is 20GB because the .vmdk was created with 20GB virtual disk?
I created a VM with 20GB virtual disk using VMware fusion with no splitting option and installed linux core and base. I see that .vmdk is ~1.7GB and the following command produced linux.raw file which is 20GB.
What I'm trying to achieve is to install my application and the dependencies in the VM and create an ISO image and distribue it to my customers. It's possible to distribute the virtual appliance in .vmdk itself but if I can distribute an ISO customer can install it on any hardware or any virtualization technologies.
And what is the problem?
You only use the .raw file as an intermediate step so the fact that is 20GB shouldn't matter.
In your useage scenario you may be better off using TKLPatch, to create a patch that you can apply to the default ISO (to recreate your custom appliance).
Thanks. I have not explored
Thanks. I have not explored TKLPatch yet. Is it possible to convert my 20GB .raw file an .iso under 4GB?
As the tutorial says...
Once you have created the .raw file you then mount it (like you would an .iso) and copy out the contents (which should only be ~1.7GB - ie your original data size). Your resulting .iso should be around that size too. If not then I would assume there is something funny/wrong going on.
Thanks
Thanks
Version 11
It was mentioned that changes were required to do this with version 11 based appliances. What needs to be done to make this process work with those? Now that I have a working iFolder appliance I'd like to make a cd of it so I don't have to compile the packages again.
No sorry, got no idea.
Perhaps you could try RemasterSys in the meantime? Although TBH I haven't used it with a server (but it's been quite useful for desktop use).
PS Did you document your iFolder progress? It'd be great if you could share as I'd love to create an iFolder TKLPatch and then perhaps an official TKL iFolder appliance might be released.
iFolder
I posted a comment on the wiki page for the iFolder appliance about the steps i took to compile iFolder on version 11. If the compiled packages could be put in a repo somewhere then making a patch would be very easy. You would not have to wait for it to compile and you would not have to do all the cleanup (removing all the necessary packages, etc.).
Excellent, thanks.
I will investigate further. When I get a chance I will look at perhaps uploading it to a PPA. I'll post if/when I get to that.
Half way through!
Hello am at half way exactly on
cat>turnkey-core.rootfs/etc/fstab<<EOF
but I get the followin
Simple method!
I found the this command: VBoxManage filename.vdi newfile.iso --format RAW
Simple method
The command VBoxManage filename.vdi newfile.iso --format RAW would not work for me.
I think that the command would be VBoxManage clonehd filename.vdi newfile.iso --format RAW
I have not been able to test the iso file that resulted yet. The iso file is about 6 times the size of the vdi file.
The ISO file you created includes free space
That's why it's larger than your original VDI. And actually it's a RAW file (which is a raw harddrive image). If you follow the tutorial it is only halfway through the process. To complete it you need to mount the image you have created and copy out the files, then make an ISO of the files only (that doesn't include the free space). This should bring your ISO back down to a similar (or perhaps smaller) filesize as your original VDI.
The ISO file you created includes free space
Thanks Jeremy. I guess that just proves that when all else fails one should read the instructions. I appreciate your reply
basename: missing operand
Everything moves along swell up until the last command..
tklpatch-prepare-cdroot Mikrotik/
# preparing cdroot
basename: missing operand
Try `basename --help' for more information.
cp: omitting directory `Mikrotik//boot/'
Try this:
Just guessing but try:
(No trailing slash). The clue for me was the error message (unless it's just a typo):
Although must admit that the tutorial does include the trailing slash so not sure if that's it.
Nope, same problem.
Nope, the issue persists.
Do I have to prepare the image on a turn-key distrobution for this to work? I am trying to prepare the image from an Ubuntu 10.4 installation..
I'm trying to convert a
I'm trying to convert a virtual machine created with MikroTik Router OS from here
http://download.mikrotik.com/mikrotik-5.4.iso
Into a bootable ISO of the configured OS.
ISO --> Virtual Machine --> ISO of the VM
I've managed to properly mount all the partitions using some info gleaned from here: http://felinemenace.org/~andrewg/MikroTik_Router_Security_Analysis_Part1/
Basically have to mount the RAW image with and offset of 512 to id the filesystem (ext2) properly.
This will actually mount an image properly, so just modify it as the Gotcha above for partitions.
I'm just stuck now, I don't know how to properly add both partitions to the image nor will the script run on the rootfs gleaned from one of the partitions.
If anyone can help I would appreciate this.
Hmmm beyond me...
This tutorial works with TKL and I would imagine any Ubuntu or Ubuntu based distro. It may even also work on Debian - but perhaps not...
A quick scan of the second link you provided suggests that that particular version is a mishmash of 2 old versions of Debian but I couldn't quite work out what the go was with the new version. The fact that searching/scanning the website, forum and wiki resulted in no idea on what the OS is based on or what sort of licence it's under doesn't bode well (although perhaps I'm wrong). It seems quite closed and as such I'd suspect that the only way you will be able to do what you are hoping to is with the help and/or support of the makers/community. Have you posted on their forums?
But TBH unless there is a compelling reason, personally I'd be looking about for an opensource product that does the same thing thgat does support hacking etc. I haven't tested any but Wikipedia has a list of starters. No doubt there is probably more if you have a good google. My 2c anyway...
True regarding the opensource
True regarding the opensource firewalls distros.
Except I've been through ALL of THEM. :) Seriously, I'm running on Zentyal at my office at the moment, but I've been through Smoothwall, pfSense, ClarkConnect, IpCop, IPFire, etc etc etc.
//disclaimer - I am not affiliated with Mikrotik in any way whatsoever. In fact, they would probably look down on what I am trying to do by creating a LiveCD.
Mikrotik is absolutly the best of the best. Its a bit funky to get started with, ( I was actually planning on running pfSense before I read a single post from a random comment on the pfSense forums that said "Try Mikrotik, once you learn it, you will never go back." I spent last weekend learning it, and without a doubt, it frakking rocks.
Just to get started, a web interface and windows interface for the firewall.. Why would you need the windows interface? Well, for starters, it allows you to connect to the firewall to reconfigure it regardless of your machines IP settings. It connects via MAC address, TCP/IP4 or TCP/IP6 as well as autoscans the local network so a freshly installed firewall is accessible without even touching the console.
The 24 hour time limit on the demo is a bummer but you can backup your config, reformat , and reinstall your config to continue using it or learning it. As well as their licensing is confusing and a bit expensive for SOHO usage.. but considering you can buy some of the RouterBoards including the license for less than the licensing alone, I'm not sure where I stand on that.
I highly reccomend it if you do any networking at all, or if you just want something fun to play with.
I don't think this is going to work for you
AFAIK unless the original distro supports running live then this will not work for you to create a live distro.
I'm sure creating a live version of the router OS is possible, but not without some serious hacking. The TKL devs have hacked the default Ubuntu Server OS so it can run live so unlike default Ubuntu Server TKL can run live already. I imagine you would need to do similar hacking first to allow this distro to run live. How you would actually go about that is well beyond me and may also be outside the terms of the Mikrotik licence.
As you've possibly guessed from my previous posts I'm a strong advocate of the open source model. But I also try to be respectful of the relevant licencing. If you like Mikrotik RouterOS that much I suggest you consult with them on what your options are and what it will cost you to do what you want.
qemu taking a long time -- is it working?
I'm guessing that this conversion process could take a really long time. I'm waiting for the command "qemu-img convert -f vmdk Linux\ Mint.vmdk -O raw LinuxMint8Helena.raw" to finish executing. Should it really take so long?
obviously it depends on the size of the image
But if it's a big image, yes it will take a while. A good baseline timeframe to consider is how long would it take to copy the file? Then add a bit (depending on hardware). You can see if it's still doing something via a new terminal window. I don't remember the commands off the top of my head but if you google "check progress qemu-img convert commandline linux" or something similar. I haven't actually done it with qemu-img but I am assuming it would be similar to checking progress of dd.
failed to run command `dpkg-divert'
Thanks for the post... Everything was fine until the chroot command. I get the below error
$ tklpatch-chroot mrs-meego-1.1.rootfs/
chroot: failed to run command `dpkg-divert': No such file or directory
Any idea what is wrong here?
Thanks
Request
Can You Please Upload A Video For This
issue with Android-2.3-RC1 vmdk
hi, im trying to mount a vmdk with installed Android-x86 distro.
my goal is to modify system dir.
tried this :
chmod 777 android23.vmdk
losetup -s -f android23.vmdk
kpartx -a dev/loop0 , cause there is a partition
mount /dev/mapper/loop0p1 mydisk
mount was correct, but not access in delete system or any other dir
cheers
Gotcha #3?
error:
mount: you must specify the filesystem type
Neither 'msdos', 'fat32' or 'ntfs' seem to be recognized. (yes, I'm doing this with a windows 7 image).
Any suggestions? Thanks!
Don't think you'll have any joy with Win7
vfat should be a recognised format and depending on what distro you are using, you may need to instal ntfs drivers before it will recognise NTFS FS.
Regardless of that though, because of the nature of how Windows works, I don't think you'll have any joy using this method anyway. Even if you do manage to convert your VM to an ISO I imagine that you'll find that trying to run it on anything that doesn't closely resemble the virtual hardware of your VM will result in a BSOD.
I'm guessing you are trying to create a Win7 ISO with preinstalled apps and/or customised config? I don't know enough about Win7 to give you any really good suggestions, but if you are looking to create a preconfigured image then the MS tool sysprep is probably your best bet and then create an image of your syspreped system. If you hunt about online I'm sure you'll find tons of info detailing the best course of action.
Thank you Jeremy
Yeah, I wasn't sure if this would work for Windows, but was having trouble getting anything else to work so thought I'd give it a shot. I'll give sysprep a try! Thanks again for your quick and helpful response!
No worries
Another afterthought may be vLite. AFAIK it is the successor to nLite which was a quite cool way of creating a customised XP install CD (vLite is for Vista & 7 IIRC).
I have successfully been using a combination of sysprep and a FOG server to create and deploy Win7 images to multiple machines. It may be overkill (or simply not suitable) for your purposes but it works really well for mine! Possibly worth a look...?
Error
Everything works up to this point:
After this I get an error about syslinux not being found... Any suggestions?
Post new comment