Moving the US Army into any muslim / arab country is effectively painting a bullseye upon them. It doesn't really matter who is ousted to do so; in a sense it's almost a diversionary tactic - they're shifting every mobile terrorist in the Arab world to Iraq rather than, say, them flying off to a western nation and blowing up stuff there.
(of course, there's a difference between Arab regional terrorists and 'international' terrorists anyways)
Saddam was simply not a worthwhile target for any terrorists; he set himself up nicely by increasing the Islamic influence upon secular Iraq (IIRC he was introducing Shariah law, for example) to try and win Muslim nations support, and was also an enemy of the US (so anyone trying to oust him ran the risk of looking like they were helping the US).