Also, keep in mind that XP x64 is based on Server 03's codebase, so it's not going to handle exactly like XP will. That might be a good thing though. XP x64 really showed me no real improvement except for a slightly more responsive system. You WILL need twice the RAM though. It's like 98 -> xp, where in 98 you could get away with 256 mb of RAM, and in XP, 512 is a bare minimum. 1GB will soon be the absolute bare minimum soon, and 2GB will probably be the standard "comfort zone" for most users.
What turned me away from it honestly though was that my firewall didn't have x64 support, and ran like crap in 32bit mode. Outpost now supports 64 bit, but if I were going to run anything in 64, I'd go Linux or Vista, and keep XP 32bit.