I spent the Spring working with Microsoft’s Windows Portable Device API along with others here at Songbird to get MTP support in Songbird. WPD is the API Microsoft provides to allow applications to talk to MTP compliant devices. At first glance it looks simple enough. You have objects that have properties. So you figure you get an object and then ask it for its properties or other objects under it, similar to many other API’s I’ve come across. Well conceptually that’s true, but the interfaces are different. The big advantage of the way the API’s are laid out is you do not have COM object instances representing objects on the device. So when you enumerate a large list of objects you don’t have to keep a lot of objects alive. When you call EnumObjects you get a collection of ID’s not objects. You don’t end up with a ton of objects alive when you don’t really need them. One can only hope there’s some caching built into the API to reduce object creation and destruction.

MSDN does well enough at documenting the interfaces. A quick run down of the main interfaces:

There’s a handful of other interfaces that do things such as hold collections of values or named value pairs, but these are the main ones.

All in all it’s a pretty efficient API. What makes things more interesting is that the stability and consistency of the implementation is primarily up to the device driver developers. A poorly written driver can really create headaches. If the device is popular and you want to support it, you’re left specializing your code for that device, adding to the maintenance cost of your application. That’s pretty much true of any driver architecture. The relatively new MTP standard definitely has issues due to its relatively young age. Also there isn’t a lot of code out in the wild to learn from.

Built in Race conditions

One of the things that I found out late in the game is that the WPD isn’t really what I’d call thread safe, well for that matter multi-process safe. Rather than handling concurrency with blocking they’ve seem to have taken the stance that if two processes or threads hit the same WPD object at the same time they just return an error, “object in use” 0×800700aa. I guess that’s the easy way out and prevents possible deadlocks. But what it ends up doing is requiring the user of the API to either special case the error and bail or add a retry loop. We opted for the latter where the problem was showing up.

I’ve found that having Explorer open while running an application accessing MTP can really increase the chance for this error. That’s what kind of turned me onto this issue in the first place. I was noticing very sporadic failures, and just figured with the debugging and such the device was getting into a bad state.

Take it Slow

One of the annoying things we ran into near the end of the Dokken release was that if you manipulated objects too fast you’d get errors. Even waiting and retrying didn’t seem to work. Once you ran and tripped you just couldn’t seem to get back up, you were stuck. We ended up putting in some sleep statements after making changes to objects and that seemed to do the trick. Man how I hate that type of solution. I think some issues revolve around the lifespan of the IPortableDeviceProperties interface. Seems to be a good idea to release it after making a set of changes.

It’s a Transaction Based System?

One of the features I thought I’d like was the transactional nature of the API. This would be nice, if you needed to bail just abort. So when we went to implement use cancel of a copy I figure this will be nice, I can just call the Revert method of the IStream. Well that seemed to work. When you looked the object wasn’t there and all was good. Well until you unplugged and replugged the device in. Then what I call ghost objects appear. Apparently the Revert didn’t commit, but it also didn’t quite revert all the way. This seems to be a problem with only certain devices, though.

While investigating this I realized there was a Cancel method on the IPortableDeviceDataStream interface. Ah ha! Apparently whoever designed the interface decided that Revert was just a bad name and they really liked Cancel better. Well what could be going on here, though I haven’t proved it, is the Revert is for solely the stream part, and the Cancel is for the MTP object part and maybe the stream. In any case, using Cancel didn’t change the behavior, we still ended up with ghost objects.

The only way we found to deal with the ghost objects was to try and delete them the next time the device was mounted. The content length appeared to be zero in this case. That turned out to not always be true, it sometimes was wild values. And then we ended up with problems where some devices didn’t report a content length and we erroneously deleted them.

One way to deal with this may be just to always commit. If you’ve canceled I believe it will error out since you haven’t written the entire content. Then you could just attempt the delete in case you did the cancel near the end and had written the entire content.

Just Need to Unplug

One thing I did notice is that sometimes you just have to unplug the device. Debugging and stopping in the middle of some operations will leave objects on the device marked in use, even when the process is long gone. And sometimes you just have to reboot the operating system.

A curious behavior on my Vista system is if I startup with the device connected I have to unplug and plug it back in twice before even Explorer will recognize it.

Properties

Properties was probably the biggest weakness. The sample app, WpdApiSample that comes with the SDK is ok and a good source of how to go about things. It’s well worth browsing the code. One header file in particular is excellent, the portabledevice.h header. This has all the properties and what types the values should be.

What is lacking is what to expect. One good source is the device driver kit. This basically documents what the device driver authors should be doing. So it’s worth browsing that MSDN area and even installing the device driver kit. The install contains a nice utility for inspecting objects, better than the WpdApiSample.

Playlists

These proved to be a little problematic. We create an empty playlist intially and that seems to create problems. WPD will do it, but often when you go to retrieve it, you’ll get an error 0×80004005 (Operation failed). Also we found that if you go to change values in the playlist it just recreates the playlist. We were saving of the persistent ID and assuming modifications would keep the ID the same, but unfortunately at least for some devices it doesn’t. So we just basically recreated the playlist ourselves and had to update the persistent ID we were storing.

Bulk API’s

I also recently got to work with the bulk property API’s. These looked to be a real savings. I could query properties for objects across the entire device. No more walking the folders. And it would have been a great performance boost if it actually worked. My experience was that query properties for more than one object only reliably worked when query just a single folder. I got the infamous “Object in use” error 0×800700aa when ever I called the API’s at the device level or even for a single storage item. Like so many other issues, this could be a poor driver implementation by the device manufacturer. It is also called out pretty plainly that not all drivers support these bulk API’s as well.

My approach was to abstract out the behavior of iterating the device. Since I couldn’t just query at the top level I had to go to each folder this made the use of bulk API’s closer to the use of the normal API’s. So now the bulk of our code doesn’t care which API we happen to be using. The decision is made up front by QueryInterface’ing the IPortableDeviceProperties instance to IPortableDevicePropertiesBulk. If that succeeds then we create our bulk property retriever else we create a simple property retriever.

Hopefully once the MTP interface and devices mature the quirks of the various API’s will go away and the least common denominator we currently have to support will be closer to what the full API’s specify.