That's the thing about compatibility, though. It's all well and good to talk about the efficiency gains if one were to drop aspects of legacy support in the OS or x86 architecture, but as soon as you come across an old favorite game or ancient-but-useful application that just refuses to work, I feel like you'll be singing a different tune. It seems to me that it would be a relatively simple matter for Microsoft to develop some sort of environment that would maintain old code compatibilities while simultaneously stripping them out of the main OS codebase, which would let you have your cake and eat it too. As it stands now, until the point when someone manages to develop a fully-functioning emulation of your standard Windows 95 box, if what you're trying to run can't be handled by DOSbox, you're out of luck.
To be honest, though, what irks me more than the OS side of incompatibility is that which is prompted by graphics manufacturers, or even the software developers themselves. As an example, I have an old EA-developed NASCAR game from 2000 lying around that's flat-out incompatible with XP at its core, no matter what sorts of compatibility combinations one applies. No big loss, and perhaps no big surprise considering the publisher, but still completely unnecessary. Far more frustrating than this were my attempts to get one of my brother's favorite games, Medieval Total War, to run on his new laptop...only to find that it's almost completely incompatible with just about any nVidia product released over the past two years. The only solution I've seen is to install a two-year-old driver version...which does him absolutely no good when he has an OS that didn't even exist at that time. And all the while, the developer (which still exists and is fully active, mind you) all but ignores all customer complaints, passing the buck to nVidia. Brilliant.