I'd go for the CPU. The 5770 is a solid card that should be able to handle most things.
I was thinking that the nVidia cards with their CUDA and stuff might compensate for my proccy in games? I am not knowledgeable on this subject though.
Both are quite lacking in performance. I'd save the money.
Save the money for a future upgrade?
Considering that what you have is significantly better than my Athlon II x4 620, 4 Gb of Ram and s 4870, and I consider mine a reasonably capable midrange machine 3 years into its lifetime, I'd wait till the spring to do any upgrades. Primarily because right now, any reasonable CPU upgrade for your processor will likely require a new motherboard as well. And that 5770 is still a pretty solid midrange card
Even my machine is 3~3.5 years in service, and my mobo has an AM3 socket, so isn't that compatible with those hexa-core proccys (the x6 series, i believe)
Aarrgh, its so confusing
Yes, AM3 is compatible with Phenom II x6 processors. And yeah, that would be an upgrade, but if you're looking for more gaming ability those extra cores are still of negligeable benefit. Despite the fact that many games are making use of more cores, very few are capable of using more than 4, and a large number of programs are still using 1 or 2. In those cases, the slightly lower clock speed of most of the reasonably priced hexacore Phenoms would actually be a hindrance.
Personally, I would grab a decent CPU cooler and, since IIRC the 945 is a Black Edition processor, overclock it a bit. They're well known for being fairly easy to overclock, and that would give you a performance boost without the full cost of a CPU.
As for the GPU... There is very little reason, gaming-wise, to pick Nvidia over AMD or vice versa, with the possible exception of drivers and some issues with openGL on said AMD drivers. The 5000 series will also be supported under Windows 8, if that matters at all to you. My 4870 technically isn't, but runs fine with the 7 drivers installed using compatibility mode on the installer. Nvidia does support their older cards much better, so you might get a bit more longevity out of a newer Nvidia card, however.