Apparently fructose is the real problem, though when eaten in natural fruits it's less dangerous because some of the uptake is blocked by the fruit's fiber content. Right now nobody has anything definitive, even though the debate has been going on for a small eon, but there's enough tendentious evidence to make some fairly level-headed people leery.
Certainly having a ton of sugar in your diet is much worse for you than having comparatively less sugar - whether because of the old 'empty calories' argument or because sugar, particularly fructose, actually is a metabolic poison. I don't feel like I know enough about this to speak on the topic with any more confidence than 'some scientists seem to have good reason to be concerned'.
