In regards to AI, let's be honest - they are stupid. I'm not sure how the algorithm goes, but here are what I know based on what I have observed through my multiple playthroughs.
Engage All and Ignore My TargetBy default, the AIs are in C-3-9 (Engage All). In this case, they will divide equally among all enemy present on the screen, and I believe also based on their closeness (they won't run to engage someone that is 9000 miles away, if you got enemies 1000 miles near you). However, the big con is exactly what you're describing in mission 3 : they tend to suicide into larger vessels.
In order to counter that, you need to tell your wingmen to Ignore the target (C-3-6). Usually, whenever a new cap ship jumps in, I immediately go to the fleet menu (F2 or F3, can't remember exactly), assign a F-something to all the cap ships one by one, and then, one by one I target them then C-3-6.
Engage All vs Destroy My TargetNow for the C-3-9 vs C-3-1.. Let's suppose you have a fleet two times larger than the enemy's. If you C-3-1 all the way, by the time you destroy the enemy fleet, the enemy will have destroyed at least 2 or 3 ships of yours. That's because when you order your wingmen to focus someone, they will focus him until they are hit by someone else - but when they are hit by someone else it's already too late to retaliate. This does not happen with a C-3-9, since you have a 2v1 on each fight, which almost guarantees that the 1 will be destroyed without damaging any of the 2.
However, for a C-3-9 to work best, you need to have all your fleet facing all the enemy fleet, and not scattered around. Indeed, if they are scattered, some of your mates will arrive faster than the others, so you won't have as much firepower in the "first strike" than you would if you had everyone gathered.
My tacticUsually, what I do is that between each breathing I C-3-7 (or C-3-8 depending on my mood), and when the enemy arrives, I face them, then 2 to 3 seconds later I C-3-9. This is, of course, after having ignored the cap ships one by one. The result is usually deadly. You can later go back to the cap ships by C-3-1.
This is assuming you got superiority in battle in terms of numbers and/or ships. Which is the case of most of the survive-the-waves missions (the real challenge is to keep your mates alive in these missions - but you outnumber them at each encounter if you are careful enough)
If you are outnumberedNow, if you are outnumbered, C-3-9 is definitely not the way to go. It will only contribute to scatter your team mates.. You need to have a bit of micro management. The best way is to C-3-1 a vessel, while you target another one. Whenever your mates have destroyed their vessel, switch to another one.
If you can, you can also try to dive in first so that the enemy focuses on you ; once they are on you, you C-3-1 each one of your assaillants. If you are say in a 3v6, it can be a way to get out of this situation (if you had just C-3-1ed or C-3-9ed, your mates would have died leaving you to a 1v3 or something like that..). But you need to outrun your mates, or put some distance between you and them, so that you really are the first to engage.
This was for general missions - fighters vs fighters. Now for protection missions, I need to discuss C-3-5.
Protect My TargetWhat your mates do when C-3-5 :
1) If no threat is nearby, they circle the vessel to protect. I find it very pretty - it's really impressive when you order all your wingmen to protect a single entity in the big missions..
2) If a threat is nearby (within 2000m i think ?), they will start attacking it, by order of priority: the bombs, then the bombers, then the fighters. (by the way, regarding my previous point: a good way to attract fire on you is to play a vessel labelled as bomber, and approach a capital ship. Guaranteed success ;p)
This is a great strategy when the enemy is outnumbered. But when there's not enough fighters to defend.. It's complete chaos. Besides, in some situations, you are better of ordering your men to attack the target before it comes close, so that you have more time before it starts launching the bombs (cap ships got turrets to take care of the bombs most of the time).
So when it comes to defending, I would say that C-3-5 is good when you're not here (when you have two sides to defend for example - you can't manage two battlefields at the same time), but you are better of ordering your men with C-3-1 and perhaps even C-3-9 (but beware that they will prefer fighters if you C-3-9, so you would need to take out the bombers yourself. Which is a good strategy if you want to minimize the loss of team mates).
Careful about non-retail missionsBe careful though - in some mods (Blue Planet for example), missions become unwinnable if you start ordering your mates here and there, because they already have really precise orders defined by the mission designer, and if you start messing up with it you will only decrease their efficiency (what they were meant to do). This is not the case for the retail missions as far as i know..
Only exception : if you see that your mates have the order "Attack Wing ..". This is a great one, it's a kind of C-3-9 but only for a wing, so that they won't scatter until the entire wing is destroyed. This is my favourite order of the game, and I wish it was implemented as an assignable one.. (I would love to tell say Squadron Beta to engage Squadron Psi while Squadron Alpha engages Squadron Mu)
When strategy does not work.. Take a heavy fighter and gun your way out until you achieve numerical superiority. The proper Alpha 1 way I would say

but more importantly is the "J" button: Target Target's Target. If a new bomber wing comes in, select a bomber, press J, and send a few wingman to cover that craft. It will make your life a lot easier: Bombers have a tendency to double down on the same target. Bombers with the same wing designation will always attack the same target.
This is a great advice, and this is what saved my life on that Blue Planet 1 mission where you got tons of bombers and need to protect all your capital ships that are numerous and spread out, with 4 wings at your disposal..
Incidentally, I could never beat the last FS1 mission on Insane while staying alive. I beat it, once, and died in the process. That's when I discovered that you can win the FS1 campaign despite dying just like you can win FS2 despite dying. 
I could not beat the last mission of FS1 at all. Because I didn't have enough team mates to even got the slightest chance of hit those reactors..
I cheated in some way. I restarted the previous mission, but instead of killing everything, I just told my mates to protect the GTD Bastion while I went to the node. I was hit, hull at 50%, but it was worth it. All of my team mates being alive, the next mission was a piece of cake.
This makes me want to play Freespace again

Especially since I didn't try this Freespace Blue yet..