The ninja were created from an order of Monks who had learned and perfected the art of stealth, because they were often times set upon by bandits or local provincial daimyos for taxes/shakedown/protection money/whatever they wanted. They were so good at it in fact that some of the monks from this order were drawn away by a provincial daimyo a few provinces over in order to act as his spies. They were so good at it, they were eventually given a family name and were always "advisors" to that daimyo, their true purpouse was unknown. Thats how it started anyway.
A Ninja kills his target by sneaking into their bedroom at night, and killing them swiftly and quietly. Then sneaking out. usually with a dagger. Or preferably, the samurai who is sleeping next door's Wakizashi (the short sword that is paired with the Katana, the Wakizashi symbolizes the samurai's honor...which is why it is used in the Seppukku ritual).
Ninja rarely even carried their swords. They would get in the way. When they did, they carried Ninja-to, which were compleatly strait, shorter and thinner than a katana...and made of some of the crapiest steel the weapon smith could get his hands on. Because they were often THROWN AWAY at pursuers instead of used to fight.
Shuriken were compleatly useless as weapons. If I threw a shuriken at you as hard as I could, from TWO feet away....I would probably not do much in the way of actual life threataning damage. If you hit someone in the head with a shuriken, it would scratch them and bounce off, because of the rotational energy. If you hit them somewhere soft and squishy...like the gut for example...there's plenty of padding around there that is is an inconvience and not life threatening. They were used as distraction tools. Not weapons.
So far, in the 2 most comonly "Ninja" items...none of them are weapons. Because the weapon they actually used to kill people with, was their dagger or poison.
They question of "who would win in a fight" is rather moot, as most ninjas...if they were seen...would just run away. If they were trapped, they'd just slit their throat or stick themselves in the belly with their daggers. Why you ask?
Because the samurai were much better fighters. A samurai would generally try to cut off the ninja's arm, or a leg, then knock him out. Drag him to the local monk or priest, try to have him saved...then have some peasants torture and interogate the ninja until he talked.
Face it. Samurai lived their entire life preparing to kill. They lived every single day of their life waiting to die in a bloody, gruesom, and very painful sword fight. Their entire Life was devoted to the skills required to kill someone. Just because a ninja fights dirty, doesn't even give him a chance. Whats he going to do? Sneak around behind you when your looking RIGHT AT HIM? Throw dirt in your eyes? Who would just stand still and watch as a ninja reached for something then threw something towards your face. No one. That would be stupid.
In case your wondering how this all came about, I had a friend a few years ago who was at the intermediate-advanced level of training and skill in Ninjutsu. I had another friend who had just finished the "beginer" levels of Kenjutsu (Art of the Sword). We even gave the "Ninja" a bokken (wooden katana), no armor. He had all his ninja gear. We went down stairs, and they had a 1:1 fight.
out of 3 fights, the ninja used his powder (the kind they throw in people's eyes), all his shurikens, and tried to use every single dirty trick that he knew how to do.
at the end of the fight, (And mind you, I never said this was a "light" fight. They were fighting to win. You don't know my friends....) the score was Ninja: 0 (With a broken nose, broken hand, and a mild concusion) "Samurai": 3. With some blood on his Hakima from when he grabed the ninja's head and broke his nose with his knee.
And this was a below average "Samurai" against an average "Ninja".