No, you're misunderstanding. The frequency of alleles in the human population as a whole is changing more rapidly than it has in the past (not merely 'mixing about'). This is the definition of evolution. Ergo, our species is evolving faster than it ever has.
The selection issue is a dubious one and I've seen no data that really studies it. The idea that poor people would somehow 'evolve faster' than the 1st World rich suggests a belief that 'evolving' is a good thing that makes people more attractive and intelligent. If anything, selection pressure in those countries would favor resistance towards disease and the ability to survive on a restricted diet. We're not likely to see Homo Superior suddenly popping up in Somalia or whatnot. Evolution is adaptive, not directional.