Okay, this isn't quite as drastic as it sounds but I had a weird idea.
What if we changed the code that so that whenever the player decides to host a multiplayer game, instead of simply hosting on the machine as it currently does, the game instead started up a new standalone instance instead?
Basically the game would detect that the player wants to host a new multiplayer game and fire off a second fs2_open process with the -standalone switch. The player would then connect to this automatically and be set as the host, same as if they'd connected to a standalone on a separate computer. When the player wanted to quit, the game would send a message to the standalone telling it to shutdown.
The advantages are as follows :-
1) The multiplayer code instantly becomes more manageable. Currently we have several different possible options. The process we're running may be a client or a server. If the process is running the server there may be a player on it or it might be a standalone. If there is a player on it, they are the host. If the process is a client it also may or may not be the host depending on whether the server is or isn't a standalone.
The code frequently has to check which one we're dealing with and we have had lots of bugs caused by people not realising that things can be different when a game is on a standalone.
With this change the server never has a player on it, it's always a standalone, so we can remove any code that deals with a host on the server, making the server-side code much easier to deal with (We don't actually need to remove it immediately, but with this change it would be unreachable).
The client-side code already has to deal with a client being a host since that's exactly what happens when someone connects to a standalone for the first time, so we wouldn't need to make any client side changes.
2) Multiplayer testing of new features becomes much easier. Even when you play a multiplayer co-op on your own you'd now be playing as a client. So testing features for multiplayer would become as simple as switching the mission in FRED from singleplayer to co-op and replaying it. We instantly remove that whole class of bugs where something works on the server but doesn't work on the client because no one ever plays on the server any more. Admittedly we now have "Works in SP but not in Multi" issues but we've always had that, and now they're easier to reproduce.
Of course there are some possible downsides
1) I'm worried it might make it harder to debug the server side of things. Since the standalone is being launched by the code itself, we'd need some way to attach a debugger to it. Is it possible to have a second process do this automatically?
2) The standalone code is currently buggier than the standard multiplayer code. I'm thinking of adding this feature using a commandline switch for now so that people can still play multiplayer normally without having this issue. Eventually we'd want to remove the switch or we'd lose the benefits of a neater code.
3) The standalone only works on Windows currently. This solution would effectively lock Linux and Mac OS X out of hosting multiplayer games until they also have a standalone. I believe some work has been carried out on a wxWidgets port though. A daemonised standalone might be a much better solution though.
If anyone has any thoughts, Especially about problem 1) I'd like to hear them.