Actually, if you fuse tritium and deuterium, you do indeed get a neutron out of the reaction, in addition to energy released as gamma/röntgen radiation and kinetic energy of helium and neutron... And if you fuse two tritium nuclei, you get even two neutrons per reaction.
In fact, the modern fusion bombs rely on neutron flux to produce their tritium. Their fuel of choice is lithium deuteride, or rather, lithium-6 deuteride. When lithium is exposed to neutron bombardment, the reaction produces tritium. The preferrance of solid fuel instead of liquified deuterium or tritium is obvious - solid fuel is easier to keep solid. Anyway, the reaction goes like this:
Li-6 + n => 2 H-3 + He-4
H-2 + H-3 => He-4 + n + ENERGY
The resulting tritium then reacts with deuterium that was in the lithium deuteride. The reaction can be triggered by any means that produce sufficient pressure and temperature, but in any case, some neutrons are needed to create some tritium to make the fusion reaction possible. This makes it quite clear that a small fission device is best suited to fire up the lithium deuteride.
And speaking of annihilation - that is a different thing altogether. It is true that the slow neutrons will affect the uranium particles from the fission detonator so that most of it goes through a fission reaction, but that doesn't really annihilate the radioactive particles (ie. transmute them into pure, massless energy in form of electromagnetic radiation). Instead, the released energy (kinetic, or thermal, whatever you prefer) blows the radioactive daughter nuclei (like Strontium, Krypton, Barium and Xenon) on a wide area. They form the most of radioactive fallout - and when they decay, the reaulting nuclei, of course.
Besides, most of the neutrons emanating from the fusion reaction are too fast to trigger fission reaction in the spare uranium nuclei that still are around. The Uranium-235 -fissionable requires slow neutrons to change the nuclei into Uranium-236, which is spontaneously fissionable. That triggers the fission chain reaction. U-238 is in the mixture to make the fisson material stable enough not to blow up spontaneously from one stray neutron...
Actually, the neutron flux is probably going to play a big role in future fusion reactors as means of producing tritium. It's simple - just cover the insides of the reactor torus with lithium.