Mike Levin SEO

Future-proof your technology-skills with Linux, Python, vim & git... and me!

QEMU Mac OS X… Success!

by Mike Levin SEO & Datamaster, 10/05/2012

It’s time for for me to create an up-to-date QEMU on Mac OS X. I had wonderful success yesterday getting QEMU compiled and installed on my Commodore 64x running Linux Ubuntu beta 12.10 yesterday. I’m working from home today on my Macbook Air laptop, so I’ll today, I’m doing it for Mac. My goal is to make the best portable Linux for USB drives that works across Mac, Windows and Linux.

I did this before in a proof-of-concept Levinux, cobbling together QEMU binaries from different sources, with inconsistent versions and unencapsulated dependencies. This time, I’m doing it in a cleaner and more organized way. This has been a stumbling block for me for quite some time, preventing me from “taking things to the next level” of which I see this as a critical first step. It turns out, compiling QEMU from source across all three platforms to my very specific requirements wasn’t as easy as I had hoped. But yesterday, I did it for Linux. What about Mac?

First, we look at the options for getting QEMU on a Mac, and inevitably you end up on the website for a project named “Q”, which has a lot of great characteristics in how it was made to look and work well with the Mac—except that it’s old, and the self-contained packages it creates are very difficult to follow and organize side-by-side with the files required for the other platforms. So, I’m going to take the identical approach to compiling QEMU on Mac OS X as I did with Linux.

It’s worth noting that there are other options to get modern QEMU’s installed on Mac. There are at least three free and open source (FOSS) package management systems and repositories for Mac: Fink, Homebrew and MacPorts. It would be possible to just use the QEMU repository from one of these and round up its dependencies and linked libraries as if I compiled it myself. That is in fact my backup plan if I can’t get the compile to happen correctly. But first, I’m going to try compiling.

First thing is what compiler I’m going to use. There is a gcc compiler that comes with the Mac’s native integrated development environment (IDE) called XCode, which is available in the App Store. Once it’s installed, you have to go to XCode / Preferences / Downloads and install the Command Line Tools. There are other choices to get the gcc compiler. The Homebrew community for instance managed to get Apple to unbundle gcc from XCode, and it is available now from the Apple Developer download page.

Once you’ve installed the Command Line Tools, you should be able to open a Terminal window and type gcc. It will say “i686-apple-darwin11-llvm-gcc-4.2: no input files”, which is nice, because now you know the compiler type and version.

Okay, so I downloaded the exact same QEMU 1.2 tarball file from the official site as I did for Linux, and unarchived it to the desktop. The Mac is a bit different from standard Linux/Unix file layout. What would be /home/username/Desktop on Ubuntu is:


…on Mac. So, I cd into the qemu-1.2.0 folder there, and run the exact same configure command as on Ubuntu:

./configure –target-list=i386-softmmu

…and I get the message:

Error: pkg-config binary ‘pkg-config’ not found

…and so begins dependency hell. This is the magic moment where you’re standing outside a rabbit hole on the precipice of Wonderland, wondering whether to chase the rabbit, or just move onto another project. Well, having traveled this wilderness before, I do in fact know that the choices are Fink, MacPorts and Homebrew. MacPorts is where the consensus wisdom was last go-around, but I remember half-hour install times with tangled webs of dependences that just did not look like they should be necessary for simple installs. Having researched further, the consensus wisdom seems to be shifting to the slightly less formal, but faster and more pragmatic Homebrew.

So, Homebrew it is. Their github page does a really good quick overview http://mxcl.github.com/homebrew/

You simply paste this into a terminal:

ruby -e “$(curl -fsSkL raw.github.com/mxcl/homebrew/go)”

…then enter your admin password, and voila, Homebrew is installed!

The first thing it tells you is to run:

brew doctor

…which tells me to upgrade my XCode from 4.4.1 to 4.5 and I started it from the App Store update, and it’s saying 2 hours. Ugh! I will let it continue downloading, but I don’t have the patience to wait.

I also get messages about unexpected dylibs files in various places that it says could cause problems and may need to be deleted. I will take note, but not actually delete anything yet. Who knows what I could be breaking. So, jumping right into it, I type:

brew install pkg-config

…okay, and it gets past pkg-config, but now I get the message:

glib-2.12 required to compile QEMU

This is the C standard library that I discussed regarding Linux Ubuntu in my last post, and here we have the same dependency trying to compile on OS X. So far, the only places compiling QEMU has required more downloads on the Mac side is getting a package management system (Ubuntu already has Debian’s dpkg system and all its simplification programs like apt-get, aptitude, and synaptic). On the Mac, I first had to get a Mac package management system (Homebrew), and a support program pkg-config for the ./configure step. Otherwise, I also had to download the compiler and C standard library for both Ubuntu and Mac. So, I type:

brew install glib

The last few steps of the Homebrew C library install was itself a compile.

==> ./configure –disable-maintainer-mode –disable-dtrace –prefix=/usr/local/C ==> make ==> make install /usr/local/Cellar/glib/2.32.4: 417 files, 15M, built in 5.1 minutes

I’ll tell you after having done the same thing recently with MacPorts, I’m glad I switched. 5.1 minutes felt long, but it was nothing compared to the seemingly endless MacPort installs.

And the “./configure –target-list=i386-softmmu” step just succeeded. Woo hoo! Now, we:

make make install

You might ask now that I know how to use DESTDIR on the make install step, why am I letting make plop the files into their actual locations, polluting my filesystem and obfuscating dependencies? Well, I want to make sure QEMU itself runs properly, controlling as many variables as possible. I frankly don’t know if QEMU looks for the files it needs relative to its own location (in which case it would run fine out of the DESTDIR location), or if it looks for things in absolute path locations. I’m not taking any chances. First I do a normal make install, test QEMU, then do ANOTHER make install using DESTDIR as a method of rounding up all the files for later encapsulation. Here’s the output of make install:

install -d -m 0755 “/usr/local/share/doc/qemu” install -c -m 0644 qemu-doc.html qemu-tech.html “/usr/local/share/doc/qemu” install -c -m 0644 QMP/qmp-commands.txt “/usr/local/share/doc/qemu” install -d -m 0755 “/usr/local/share/man/man1” install -c -m 0644 qemu.1 qemu-img.1 “/usr/local/share/man/man1” install -d -m 0755 “/usr/local/share/man/man8” install -c -m 0644 qemu-nbd.8 “/usr/local/share/man/man8” install -d -m 0755 “/usr/local/share/qemu” install -d -m 0755 “/usr/local/etc/qemu” install -c -m 0644 /Users/username/Desktop/qemu-1.2.0/sysconfigs/target/target-x86_64.conf “/usr/local/etc/qemu” install -c -m 0644 /Users/username/Desktop/qemu-1.2.0/sysconfigs/target/cpus-x86_64.conf “/usr/local/share/qemu” install -d -m 0755 “/usr/local/bin” install: chmod 755 /usr/local/bin: Operation not permitted install -c -m 0755 qemu-ga qemu-nbd qemu-img qemu-io “/usr/local/bin” set -e; for x in bios.bin sgabios.bin vgabios.bin vgabios-cirrus.bin vgabios-stdvga.bin vgabios-vmware.bin vgabios-qxl.bin ppc_rom.bin openbios-sparc32 openbios-sparc64 openbios-ppc pxe-e1000.rom pxe-eepro100.rom pxe-ne2k_pci.rom pxe-pcnet.rom pxe-rtl8139.rom pxe-virtio.rom qemu-icon.bmp bamboo.dtb petalogix-s3adsp1800.dtb petalogix-ml605.dtb multiboot.bin linuxboot.bin kvmvapic.bin s390-zipl.rom spapr-rtas.bin slof.bin palcode-clipper; do \ install -c -m 0644 /Users/username/Desktop/qemu-1.2.0/pc-bios/$x “/usr/local/share/qemu”; \ done install -d -m 0755 “/usr/local/share/qemu/keymaps” set -e; for x in da en-gb et fr fr-ch is lt modifiers no pt-br sv ar de en-us fi fr-be hr it lv nl pl ru th common de-ch es fo fr-ca hu ja mk nl-be pt sl tr bepo; do \ install -c -m 0644 /Users/username/Desktop/qemu-1.2.0/pc-bios/keymaps/$x “/usr/local/share/qemu/keymaps”; \ done for d in i386-softmmu; do \ make -C $d install   exit 1 ; \ done install -m 755 qemu-system-i386 “/usr/local/bin” strip “/usr/local/bin/qemu-system-i386”

So now, I cd into /usr/loca/bin and type qemu-system-i386 and a Mac requestor pops up asking me for a file. Yay, it compiled successfully! Now, I’m going to copy Core-current.iso from out of the Mac.app folder on my Dropbox (discussed yesterday) onto my desktop, so I can point the requestor to that (without risking messing up my Dropbox copy).

QEMU is reporting that its not a bootable disk. Well, it is actually a LiveCD image, and not a drive boot image, so I’m going to try to launch it from the command line:

qemu-system-i386 -cdrom ~/Desktop/Core-current.iso

…and, voila! Wow, this is going much more smoothly than any time in the past. This has been my symbolic hump I could not get over for the past year or so. Making this multi-platform USB Linux really is my first step of the next level of my career. It’s one of the unique things I have to offer that nobody else seems to be doing. Okay, so let’s make sure it’s working in the non-graphical “curses” mode, so that QEMU continues to “live” inside the Terminal it was run from:

qemu-system-i386 -curses -cdrom ~/Desktop/Core-current.iso

Wowsers, success again! This just keeps getting better and better. So, let’s round up all the files into one location that’s NOT superimposed on my Unix hierarchy. So, I type (knowing a little more than I did yesterday):

make install DESTDIR=~/Desktop

…and now I have a /usr folder on my desktop. I click into it and rename the single folder that lives inside it named “local” to be named “mac” (remember my naming convention decisions from yesterday?). And now I drag that mac folder into ~/Dropbox/Levinux/Mac.app… but wait! Remember how I said the Mac.app directory was going to become a dobule-click-to-run app? Well, it now can’t just be double-clicked to be opened like a folder anymore. I have to Control+Click and choose “Show Package Contents” which then opens it just like a folder. I have a placeholder mac folder in there, which I delete, and drag this new one into place.

Levinux is coming together.

The Levinux folder is now 36.1 MB on disk. Still not bad. There’s a bit more file duplication and bloat that I would like, due to so many of the QEMU resource files necessary for Linux and Mac probably being identical, particularly in the /usr/local/share/qemu folder, which is precisely the same 5,128,784 bytes in each location. I will solve these redundancy issues later in the symbolic links I use in the encapsulating script that launches the app on each platform.

Oh, did I mention the script that launches things on each platform? Well, Remember how Mac.app looks like a app but acts like a folder? Well, how do you imagine it knows what to do when double-clicked? Well, there is a standard way this works. A folder named “Contents” needs to be inside Mac.app, and inside Contents, you need an XML text file named Info.plist. And inside THAT, you can refer to your plain old Bash script. Here is the version I used on the proof-of-concept version of Levinux, which was originally generated by Q.

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>CFBundleExecutable</key> <string>RunOnMac.sh</string> <key>CFBundleGetInfoString</key> <string>© 2005-2006 Fabrice Bellard. All rights reserved.</string> <key>CFBundleIconFile</key> <string>q_icon.icns</string> <key>CFBundleIdentifier</key> <string>ch.kberg.qemu</string> <key>CFBundleInfoDictionaryVersion</key> <string>6.0</string> <key>CFBundleName</key> <string>Q</string> <key>CFBundlePackageType</key> <string>APPL</string> <key>CFBundleShortVersionString</key> <string>0.9.1</string> <key>CFBundleSignature</key> <string>QEMU</string> <key>CFBundleVersion</key> <string>0.9.1d118</string> <key>CSResourcesFileMapped</key> <false/> <key>NSUIElement</key> <integer>0</integer> </dict> </plist>

Okay, well the entry that says “RunOnMac.sh” is the one we’re interested in. But where is this running from? Well, it turns out you can locate it right in the same folder as Info.plist, so my set of files looks something like this:

Levinux/Linux.sh Levinux/Windows.bat Levinux/Mac.app/mac/ Levinux/Mac.app/linux/ Levinux/Mac.app/windows/ Levinux/Mac.app/Contents/Info.plist Levinux/Mac.app/Contents/RunOnMac.sh

The end result is just being able to see three clickable things from the Levinux folder. Windows and Linux will see two scripts and a folder. Macs will see one app and two scripts. But in any case, it will always be easy to figure out which one you should double-click. I’ll probably put a README.txt in there just to clarify things.

So now, it’s time to take things as far on the Mac platform as I did yesterday on Linux, and so I actually make the Contents and MacOS directories, and copy in the Info.plist into location from the proof-of-concept, and finally, make a RunOnMac.sh file that launches the same command more-or-less as I’m using for Linux.

Okay, we encounter the “run silently” problem. This technique actually HIDES the new Terminal window, which I presume is desirable behavior as it prevents the perpetual annoyance on Windows that .bat files can’t run silently by default with a double-click, and you had to resort to a VBScript launcher script. We have the opposite problem here on Mac using these .app packages. If your Info.plist points to a bash script, it runs silently unless you simply USE that as a wrapper to launch a visible window, which you have to use AppleScript to do:

osascript -e ‘tell app “Terminal” do script “echo hello” end tell’

Okay, it’s not as pretty as I’d like, but it’s going to work. Okay, it getting it to find the actual ISO image, I ran into the age-old dilemma that script-launchers are rarely smart enough to set your current directory to the directory it was launched from. So this enhanced version of the a above script takes care of a number of problems, such as setting the Terminal size, making sure the background is black, and launching QEMU with the correct path to the ISO image.

osascript -e ‘tell app “Terminal” tell application “Finder” set myFolder to parent of (path to me) as alias end set myPosixFolder to quoted form of POSIX path of myFolder do script “resize -s 25 80 && qemu-system-i386 -curses -cdrom “ & myPosixFolder & “Mac.app/Core-current.iso” tell application “Terminal” to set background color of window 1 to {0,0,0,0} end tell’

Now, everything is working nicely. I struggled with questions like whether AppleScript’s “do script” is a one-shot deal or not. There are mixed opinions, and I could not stack multiple commands successfully, so I’m appending them on the same line. This may become burdensome as my QEMU command grows longer, but I will cross that bridge when I come to it.

One last thing that bothers me is all the extra stuff that Info.plist has in it. It seems to me, I should be able to strip everything out except for the key/value pair that tells it what to run: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>CFBundleExecutable</key> <string>RunOnMac.sh</string> </dict> </plist>

…and so it does. I have the working parts not for a compiled and running QEMU on both Linux and Mac desktop. I feel like touting the virtues of what I’ve accomplished here, and the path that I’m on, but I will do that in my next post.