Well where's the bottleneck? The Task Manager should be able to show you whether RAM and/or CPU are nearing critical levels, and if they're not, it's probably the GPU. (Keep in mind that, depending on the amount of cores in your current pc, critical CPU levels may be 50% or 25% for single-thread applications.)
.... Did you just ask one of SCP's lead developers if he knows where the bottlenecks in his engine are? One of the lead developers on the one team that did more than any other one to profile the engine and figure out where the bottlenecks are?
There are several reasons why debug builds are generally slower. Logging code that has to be executed, extra checks that have to be performed on everything, compiler optimizations that can't be present (compiler-optimized code being essentially unusable for debugging purposes), using the debug CRT....
All in all, FSO is almost always CPU-bound, with very few exceptions.