EG say you have two threads going, each reading from a different place in memory. The processor has to take the time to switch between tasks, including different memory or hard drive positions if the thread needs info from both. Kind of like trying to alternate between solving math problems and reading every 30 seconds or so, as opposed to just doing math and then doing the reading.
I imagine there are algorithms in place to try and switch at the most efficient spots possible, but it's just not going to be as efficient as solving tasks one by one in order.
With a dual core or CPU, the processor can assign a thread to each of the processors/cores and have them work on each separately. Using the earlier example, it's like if you had a twin who did the math while you did the reading.