It looks like "upgrading" our installers to work on Windows Server 2008 wasn’t as painful as I expected, at least not once I figured out some of the core issues.  This post summarizes some of the lessons learned.

Install IIS 6.0 Management Capability Feature

Out of the box, WiX can’t talk to IIS 7, so any extensions you use for IIS-related tasks won’t work.  You can get around the issue by installing a compatibility layer over IIS 7.  Details here.

Built-in Accounts Have Changed

In Windows Server 2003, the Internet guest account is "IUSR_[MachineName]".  All anonymous activity in IIS runs as this user.  In Windows Server 2008, this user has been removed and is instead replaced by the group "IIS_IUSRS".  This means you have to do some trickery with your installation if you want to grant the Internet guest account access to your installed web application in a cross-platform manner.  Details here.

In addition, the ASPNET user, which was unused by still there in Windows Server 2003, has been completely removed in Windows Server 2008.  If your installer must work with even older versions of Windows, you will need to use a similar condition-based technique to find the correct user based on the installation environment.

Double-check your conditions!

Sadly I spent a couple of hours trying to figure out why a particular feature was always disabled in Windows Server 2008.  It turns out that the feature had the condition "VersionNT = 502", so it was only installable on Windows Server 2003.  A quick change to "VersionNT >= 502" fixed that.

When it all goes wrong, msiexec is your friend.

I ran into a lot of problems while trying to get these installers working, but fortunately msiexec can generate some fairly useful logging information.  To do so, run your installer using "msiexec /Log log.txt /i YourInstaller.msi".  If anything goes wrong, the log will give you details.  It isn’t always the most straight-forward thing to debug, but it is much better than flying in the dark with the "helpful" errors MSI gives you by default.  In particular, I recommend looking at the "ADDLOCAL" property to verify that the features you think are being installed are actually being installed.  The log also contains the values of all properties and entries indicating which custom actions executed.