Everyone hates it when an application crashes (I can’t tell you how many times my code editor has crashed on me… GRR!), but changes in Songbird and Mozilla should make crashes a little easier to tolerate than before.

Recent nightlies (including the last blessed build) have quietly enabled support for the Mozilla Crash Reporter using Google’s Breakpad library. Now every time Songbird crashes you will have the option to submit a small crash report to our server for analysis. Check out the typical data included in a report here on our Socorro server.

These reports are extremely helpful to us as they tell us exactly what went wrong when the application crashed, enabling us to fix our code and deliver a much more stable product in the future. Report submission is optional but you are strongly encouraged to help us improve the quality of Songbird by electing to share that valuable data. You’ll be glad that you did!

And, of course, many thanks to the wonderful folks at Mozilla and Google for making this great open-source technology and for all their help getting it running.

The Mozilla build system is big, complicated, and sparsely documented. There are only a few folks that are actively working on it to my knowledge. As bsmedberg pointed out, however, the build system is also quite powerful and has had a ton of bugs worked out of it over the years. Unfortunately for most XULRunner developers it’s also off limits unless you feel like jumping in and hacking it yourself. It builds Firefox, Thunderbird, XULRunner, and other Mozilla-hosted apps and extensions happily, but what about XULMine? Other XULRunner apps? Songbird?

Well, the build system underwent a substantial change last week. (more…)

I’m happy to say that I landed some of the work to make the build system cooperate with XULRunner apps in bug 380846. I’m going to continue some smaller tasks next week. As some other guys have already started making changes to their products’ builds I figured I should at least blog about the changes that have gone in until I have time to make a doc on MDC.

The major changes are:

1. Support for arbitrary applications through ‘–enable-application’ in mozconfig files.
   • Adding ‘–enable-application=myapp’ to your ‘mozconfig’ file instructs the build system to check the folder ‘mozilla/myapp/’ for a ‘build.mk’ file. This file, in turn, tells the build system where to start the build process. See the Minimo ‘build.mk’ for the simplest example in the tree. Note that if you forget to make a ‘build.mk’ the build system will stop dead in its tracks and yell at you.
   • The folder ‘mozilla/myapp/’ will be checked for a ‘confvars.sh’ file to influence the behavior of the configure script. See the XULRunner ‘confvars.sh’ for an idea of what to do there. At a minimum you should set ‘MOZ_APP_NAME=myapp’, ‘MOZ_APP_DISPLAYNAME=My Amazing App’, and ‘MOZ_XUL_APP=1′. Add anything else you want. This file is completely optional, however.
   • The folder ‘mozilla/myapp/’ will also be checked for a ‘makefiles.sh’ file that will be called to tell the build system which makefiles the app requires. That script should call the ‘add_makefiles’ function. See the Suite ‘makefiles.sh’ for a shiny new example. This file is optional, but it doesn’t really make sense to build a new app without adding at least one Makefile…
2. Support for arbitrary extensions through ‘–enable-extensions’ in mozconfig files.
   • Setting ‘–enable-extensions=myext’ in your mozconfig file instructs the build system to look in the ‘mozilla/extensions/myext/’ folder for a ‘makefiles.sh’ script. That script should call ‘add_makefiles’ as above to tell the build system which makefiles are required to build the extension.
   • Note that extensions cannot influence the configure script through a ‘confvars.sh’ script, nor can they influence the starting point of the build system through a ‘build.mk’ file.
3. Support for building extensions without building another app first.
   • For lightweight extensions it’s a pain to build something like XULRunner just to get your JAR built, so you can now specify ‘–enable-application=extensions’ followed by ‘–enable-extensions=myext’ in a mozconfig file.
   • Note that you probably won’t have much luck trying to compile binary components without building XULRunner first… This option is really only for extensions that have JS/XUL/CSS files (unless you have a libXUL SDK – see below).

So, what to do for XULRunner apps that don’t want to rebuild XULRunner each time? Here’s the plan:

1. Make a folder for your app in the ‘mozilla/’ directory. Add ‘build.mk’, ‘confvars.sh’, and ‘makefiles.sh’ files as detailed above.
2. Add ‘Makefile.in’ files to your new folder to build your application. See the documentation on MDC for details on how to write makefiles. A good place to start is probably Plasticmillion’s extension tutorial.
3. Use a mozconfig file similar to this for whenever you want to build or rebuild your XULRunner SDK.
4. Use a mozconfig similar to this to rebuild just your application.
5. Run ‘make -f client.mk’ as usual!

This week (or next) I’ll be looking at reducing the number of makefiles required to build an app or extension with a libxul-sdk. I’ll also be looking at making a checkout target for just the build system so that you don’t have to pull the whole mozilla tree every time.

In the meantime, if anyone is interested in yanking makefiles out of the hugely bloated ‘allmakefiles.sh’ and moving them into an app-specific ‘makefiles.sh’ please feel free to ping me.

How does this work? We don’t hack out the encryption keys or anything illegal. Songbird supports multiple playback cores so we simply use Apple’s and Microsoft’s own playback engines to do the decoding for us. We use VLC for playback of most file types, but now whenever you play a protected WMA or M4P file we swap in the Windows Media Player or QuickTime core. Sounds easy, right?

Well, no. Not really. The world of DRM is a little (cough) unfriendly, so I figured that I should share some of the war stories and a few tricks for anyone interested in making DRM playback work in their own apps.

It seems both Apple and Microsoft are a little paranoid when it comes to debuggers. I fault Apple a little more than Microsoft in this case for reasons I’ll hit in a second. But be forewarned: using DRM software under a debugger may not work correctly (or at all), and documentation may be totally misleading. What happens? Well…

Applications that use QuickTime APIs on OS X will appear to crash the *instant* a FairPlay track is loaded if the app has a debugger attached. Seriously. The entire app dies and prints “Program exited with code 055″ to the console. What, you don’t know what code 055 is? No one else seemed to, either, except for Google. I was led to an excellent blog post that helped me unravel this mystery. In a nutshell QuickTime aborts the app if it sees an attached debugger and prints that cryptic error message.

For an honest developer (me) trying to use Apple’s public API this is unacceptable. Killing the entire app is entirely unnecessary. Why not just make the load function fail? The QuickTime docs don’t mention this limitation as far as I have been able to find, and that is equally ridiculous. Try searching developer.apple.com for “quicktime 055″ and see how many hits you come up with. I’d even be happier if it printed a simple “Not allowed with an attached debugger” message. As it is I spent quite a bit of time examining my code for changes that could have caused a “crash” before admitting defeat and searching Google.

Fortunately there’s a really simple way around that mess. As detailed in the blog post above, the abort is triggered from the ptrace() function, so using gdb you can do an early return and keep your app alive. I added a gdb script to my machine that automatically skips ptrace() and I haven’t looked back. Except with bitterness.

Now, before I move on to Microsoft I should point out that once the debugging issue was solved I have been rather pleased with the way QuickTime works with protected content. It even works on Windows. The APIs are a little, uh, tough to get used to if you haven’t done much Carbon programming, but with enough time and patience I got an acceptable result (and I learned enough to know that I made some mistakes – I can’t wait to fix some of the code for 0.3).

Microsoft is just as paranoid about debuggers and DRM but they win the “developer friendly” award by a hair. Accessing protected WMA files with Windows Media Player seems just like opening any other file. Your app continues to run (amazing, no hard abort!) except that the file refuses to play under a debugger.

The play() function succeeds as it does for every other file, and let me tell you, it’s really frustrating to receive no error code when you know that something has obviously failed. I ended up adding a bunch of error logging code that examines the IWMPErrorItem associated with a media item and found that Windows Media Player was actually returning an error sometime after my play() call supposedly succeeded. This leads me to believe that play() does nothing more than add the internal play command to a queue somewhere (but good luck finding that in the documentation).

The error code returned was 0xC00D2767, and after a little searching I found that code was defined as NS_E_DRM_DEBUGGING_NOT_ALLOWED: “Running this process under a debugger while using DRM content is not allowed”. Of course I wasted several hours trying to figure out why play() was failing yet claiming success, so I was still royally upset when I realized I had been bitten by the same kind of “bug” that I had fought with QuickTime. But at least Microsoft was nice enough not to kill my app and to actually return an informative error code (albeit in a way that was difficult for me to figure out).

A week ago I had never used the Windows Media Player APIs, and I’m brand new to the QuickTime APIs as well. I’m sure a lot of my frustrations were just the normal headaches associated with learning new APIs. I’m also willing to bet that a bunch of the “bugs” I encountered were caused by me using those APIs incorrectly. I’m used to debugging XULRunner, and whenever I receive an error code I’m unfamiliar with or something fails for no reason I can go look at the code to figure out what’s going on. In contrast QuickTime and Windows Media Player are black boxes, and without good documentation it’s really tough to figure out what’s going on… Add DRM into the mix and the black box turns into a black hole.

Both Apple and Microsoft are trying to protect their DRM schemes (which isn’t a bad idea), but in doing so they’ve made it much more difficult to use their products. Maybe I would feel less bitter if they’d simply stamped a big red “WARNING” banner on their documentation.

After working on Songbird for a year you would think I could have written at least a dozen blog entries and dazzled the world with my wit and expertise. The joke would be on you, of course, because I haven’t written a single one. Zilch. I’m lazy, I guess. Or shy. Or something.

So here we go, my first blog post… Ever.

I’ve recently been working to prepare Songbird for the fabled leap to Gecko 1.9 (aka “trunk”). Songbird 0.2 was based on Gecko 1.8 (two other apps you may be familiar with that were also built on Gecko 1.8 were Firefox 1.5 and Firefox 2.0). See, Gecko 1.8 has been around for a long time now. It has been thoroughly tested and crashes only sometimes. Why move to 1.9 then? Well, check out this feature list to get some idea.

The trick is that all interfaces were frozen on the 1.8 branch some time ago. Not so on trunk. So I knew I was in for a little pain trying to update all of Songbird’s interface usage. But hey, not that much has changed from branch to trunk. I thought it would be easy.

But it turns out that there was something else I had to do first. Something I had tried to ignore for as long as possible. Something I knew would bite me eventually.

You see, Songbird has been cheating for a year now. With Gecko 1.8 all XULRunner apps were given three choices when it came to using Mozilla components. Components could use two flavors of “frozen” linkage or go for “internal” linkage. At the time the frozen libs didn’t provide all the love we thought we needed, and rather than spend the time to expand the libs we decided to coast on through with internal linkage. It’s like Mozilla threw wide their doors and said, “Please, take anything you’d like! No strings attached!” It was fast, it was easy, and it was great.

We were warned, of course, that such open access would disappear in the future. Well, the future is now, as they say, and in Gecko 1.9 it is no longer possible to use internal linkage. Luckily, bsmedberg took pity on us cheaters and created a basic migration guide to help XULRunner apps move to using the frozen linkage. He has also added to the glue libraries substantially in the last year so that using frozen linkage isn’t really bad at all.

So for the last few days I have been working on porting all of our code to use Gecko 1.9 as well as frozen linkage. I got more than I bargained for, to be sure, but I’m pleased to say that Songbird builds and runs almost perfectly on Gecko 1.9 after about four days of work. On Windows, at least. The Linux and Mac boxes are my next targets.

So what kept me busy? I had to move the nsAutoLock
classes into the glue, and I also had to add comparison operators to the frozen String API in order to make our STL classes happy. Oh, and there’s a nice bug in this function that took about a day to track down (50 points if you can spot the problem without checking my patch!). Aside from those few problems I must say that everything else migrated nicely.

So, beyond our architecture plans we’re also going to be experiencing some platform love in the future. Look for hardware-accelerated graphics, airbag integration, transparency, SVG widgets, tighter OS integration, and, of course, the acid2 smiley face before too long. Exciting times, indeed.

So I went ahead and landed the majority of the changes needed for 1.9 builds to work tonight. The only hitch is that I only got a chance to check in the debug Mozilla SDK for windows… So mac and linux builds are totally hosed at the moment. Fun! I would highly recommend that eager-beaver types wait a bit before pulling the latest source – It’s going to be a little messy for a while.

Oh, and our public svn server seems to be running a bit behind… Ausman/Mattman to the rescue?