Actually, DirectX 9 was the first DirectX with true programmability. PS2.0 is the first decent pixel shader that actually lets you do stuff without using texture lookup for everything, and there's definitions for PS2.x and VS2.x (which stand for version 2 extended, if you wonder), and the definitions for VS3.0 and PS3.0, which means there will be at least three more generations of graphics hardware that will do only DirectX 9. Of course, after VS and PS 3.0 are implemented on every platform, and blending operations are made available for floating-point framebuffers, all that's needed is just to wait for everyone to get VS/PS3.0 hardware, 'cause that has everything - dynamic branching, booleans, texture lookups at the vertex shader, and whatnot.
Basically, that level of hardware could have a compiler that can compile any Renderman script for it (Renderman is a language used for shaders in some commercial 3d graphics programs), even if it may be multi-pass.
What I meant to say is, that while FS2 uses DirectX 8, DirectX 9 IS a significant upgrade, even if there don't seem to be any "new" features, like the original introduction of shaders in DirectX 8, or Fixed-Function TnL in DirectX 7.