Hard Light Productions Forums
Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: ksotar on October 23, 2018, 04:54:45 pm
-
It is common knowledge, that multiplayer (MP) in FSO is not in ideal state. For some mods and standalones it is plane unplayable (SWOT, Diaspora). The main problem is, that you can hardly score a hit. I will refer to this problem as Network Miss (NM). Other problems may also persist, but we will address them later.
Now, there is a small team of willing and, the most important, able people to deal with that problem. This thread will describe the current state of events and problems we face. Ideas and offers of assistance are welcomed.
Please note that posts about how multiplayer is not needed and that we should put our efforts to something else will be considered offtopic here and I’ll kindly ask moderators to remove those.
I. What we know so far.
To solve a problem one need to comprehend it first. And here just some empirical notion like “it seems hard to hit anyone in MP” is not enough. We need some hard evidences that are documented and are measurable. And we have some, and will obtain more. Their proofs and techniques we use to document them are gathered here (https://www.hard-light.net/forums/index.php?topic=95198.msg1875478#msg1875478). This section can be extended with your help.
A) What can be seen ingame:
1) There is a significant offset of a player’s ship position as it seen on the server and on the client itself, both during the flight and in a static situation (latter is proven in Diaspora, but not on FS).
2) On the client it may seem like hits being scored: projectiles hit enemy ships, damage sfx are in use, enemy could even “catch smoke and fire”. But on the target’s monitor it is seen that hull percent only flashes while "damaged", but don’t go down.
3) In the end the “primaries hit” percent number (accuracy) is lower, and disagrees with what player saw on his client.
4) With certain mods NM problem gets worse, and accuracy against enemy fighter may go as low as to 0-1%, with only few hits actually score. General rule - the further mod is from original FS (in terms of ships, their speed as well as projectiles speed), the worse is the problem.
5) There are lags and jumps of the ships even if ‘Object update’ setting in Multiplayer options is set to LAN. In fact, jumps can become even more noticeable with this setting.
B) What we know from code:
1) Game uses Bezier splines to approximate client’s ship trajectory. Those are not even pass through the points that client reports to the server. This could possibly be the root of position problem.
2) ‘Object update’ determines the rate with which client state is synchronised with the server.
II. What is the main hypothesis.
For now I think that the main problem is NM. If it will be fixed, MP will be mostly playable, especially in Diaspora and other non-FS based mods. And it is very likely, that we can leave for now the network layer itself and how packets do actually fly, because nowadays network throughput is very forgiving. So we can try not to touch network layer (hard part) and to fix the math (easy part) first.
Others can correct me if I’m wrong on this.
III. What we do.
1) First, we need some tools, that could help us to illustrate and debug the problem. This likely includes:
Mod and missions, which are specially prepared to debug the problem (for now we have one).
2) Automated tool (https://www.hard-light.net/forums/index.php?topic=95198.msg1875479#msg1875479) that will allow us to test MP quality in terms of NM problem (80% is ready, I’ll post later).
3) Tools that are using Debug console to show various network parameters (for example local and server positions of the client’s ship, etc), allows network Debug (without server disconnection). (10% ready - as there were some usefull commands in Debug console already).
4) And finally, we need to fix NM problem.
Also, if someone sees some wierd English senteses here, please, let me know - I'm trying to improve my English.
-
Documented problems.
Dynamic.
In case someone missed. Here is the video, that pretty much illustrates all aspects of NM. Youtube has a nice feature - frame-by-frame skipping. You can use "<" and ">" buttons for it.
To see this difference yourself you can switch view on server ship to "From" client ship.
Static.
I found this problem in Diaspora. It is MP mission Asteroid dogfight. After some flying client's ship position has notable offset. Here is the picture from a non-moving client: aimpont is at true server ship location (set by viewing from server and in that positions hits by client are registered). And where client thinks server ship is, you can see yourself.
(http://ovalhalla.ru/my/img/Freespace/Diaspora_offset.png)
Autoaim is not active. If you start carefully flying straight ahead to server ship, there will be no offset: you need to maneuver a bit beforehand. I know that it is position, not direction vector, since client can collide with "server ship", but from server's view it is like if client's ship hit an invisible wall, and bounced off.
AndI was unable to reproduce the situation with FS itself: whatever maneuvers led to no notable static offset.
-
Automated test for MP quality estimation.
The idea is to have primary shots hit % as a metric, how close MP experience is to SP (single player).
Obviously, we need large amount of flight test here, and therefore it is better to automate the process. For SP we have a convenient SEXP called 'player_use_ai'. We only need to make it work for MP. And I almost had it done, for now I use some dirty hacks in the code to make it work, but I'll refine that later.
And we need to make AI to fly like 50 missions in a row, without human interference. This also could be done from inside the engine, but I didn't want too much unnecessary changes. So I picked up Sikuli Script (http://www.sikuli.org) automation tool, it is Python-like language to automate all button clicks for repetitive tasks (see attachments for script example).
Also I made changes to the code, that stores mission stats to .csv file to analyze later.
I use a mission with unarmed AI ships for our player ship (under control of another AI) to hunt. I attached it, but if someone interested to use it too, I'd better share a mod for this metter (it uses BP AI, and there maybe will be some other changes as well).
Here what we got for now:
| Takes | Time | Prim.fired | Prim % | Min % | Max % |
| SP | 50 | 01:27 | 166 | 52% | 43% | 60% |
| MP Server | 49 | 03:35 | 263 | 31% | 20% | 43% |
Original data is in this google sheet (https://docs.google.com/spreadsheets/d/1Lu_GPfpv2PPplbxWhHh-QNjlqomr4A8N1CopHf5V26Q).
And here is the distribution (not exactly useful now, but who knows):
(http://ovalhalla.ru/my/img/Freespace/Stats1.png)
(http://ovalhalla.ru/my/img/Freespace/Stats2.png)
[attachment stolen by Russian hackers]
-
Reserved 3
-
I don't know if this is helpful, but I have two quick ideas about what is causing things to be worse on TC's.
Do the TC's rely a lot on aim assist? (I can't remember the real term but it has to do with adjusting the angle of player shots) If the client side is doing aim assist using its approximations of positions, then it would explain why you can't hit anything at all. The assist would all be based on bogus data from a poor approximation. I am pretty sure that Diaspora uses it, not at all sure about FotG.
My second, probably less helpful idea is that perhaps there is something about the math that relies on the relative speeds native to normal FSO, without looking at the code, I could not point to anything specific.
-
Sorry, what is TC?
Yes, I should've mention it. But it depends on difficulty level, for example in Diaspora autoaim is active only on Very Easy. But I have an impression, that it don't make situation any worse. That could be logical enough - both the player and autoaimm aiming at non-existant position. We could find out more, then I'll have autotests run in full power.
As for speeds - yes that could be one of the reasons, as well as ship sizes. But I have very bad results or static situations as well, so it maybe not necessarily the case.
-
I have updated the automated test (https://www.hard-light.net/forums/index.php?topic=95198.msg1875479#msg1875479) section.
And there we have a problem:
when testing with MP server it has lower performance right away, while network issues shouldn't be in effect yet, since we're running server only for now, there is no clients. Does someone have ideas about that?
-
I don't have anything useful to contribute, but I just want to say that working co-op would be amazing.
-
And there we have a problem:
when testing with MP server it has lower performance right away, while network issues shouldn't be in effect yet, since we're running server only for now, there is no clients. Does someone have ideas about that?
This is a FPS slowdown or "network" choppiness?
-
I just want to say that working co-op would be amazing.
Thanks, that's good to know that there are people who are interested.
This is a FPS slowdown or "network" choppiness?
No, it is not slowdown of any kind, but I'd say it is like AI getting "dumber".
-
This has been needed (IMHO / In My Humble Opinion) for a long time. Single player is great, but communities grow better with multiplayer, I think. Regardless, it's one more feature to attract new players, whether the community grows a lot or only a modest amount. So, cheers!
Also, TC stands for Total Conversion, which means a mod that does not require the original game to function (aka / also known as a Stand Alone game / mod). Diaspora, FotG, and TBP (The Babylong Project), and Starfox are TCs, it does not require assets from original FreeSpace to function.
Not sure if I can help much, but if you need a player to try something and I happen to be available, I can try to set up.
You're on the HLP Discord, right ksotar? I'll message you there.
-
Thanks for explanation, just found you on Discord.
-
As far as I know FotG is the only TC using any aim assist, and isn't released yet, so all real world experience with TCs and multi would be from those using standard FS2 aiming. That means every firepoint goes straight ahead, always. At some point I had just concluded FotG could never have multiplayer with the aim assist, if at all, because it would only make the situation even worse, as you wouldn't even be able to attempt to lead the target and manually compensate for the lag, since the autoaim would just put the shots behind where you know you would want to be shooting.
-
As far as I know FotG is the only TC using any aim assist, and isn't released yet, so all real world experience with TCs and multi would be from those using standard FS2 aiming. That means every firepoint goes straight ahead, always. At some point I had just concluded FotG could never have multiplayer with the aim assist, if at all, because it would only make the situation even worse, as you wouldn't even be able to attempt to lead the target and manually compensate for the lag, since the autoaim would just put the shots behind where you know you would want to be shooting.
What bout Diaspora, I thought that had aim assist?
-
As far as I know FotG is the only TC using any aim assist, and isn't released yet, so all real world experience with TCs and multi would be from those using standard FS2 aiming. That means every firepoint goes straight ahead, always. At some point I had just concluded FotG could never have multiplayer with the aim assist, if at all, because it would only make the situation even worse, as you wouldn't even be able to attempt to lead the target and manually compensate for the lag, since the autoaim would just put the shots behind where you know you would want to be shooting.
What bout Diaspora, I thought that had aim assist?
It has autoaim for Very Easy and Easy.
-
WC Saga's Excalibur fighter uses it also.
-
So, no versions on why AI acts worse in MP?
Updated post with static position offset description (https://www.hard-light.net/forums/index.php?topic=95198.msg1875478#msg1875478). It would be interesting if someone could reproduce this. Also your thoughts on this are welcomed.
UPD looks like some pics are not showing, I'll try to fix it later.
UPD2 pics are back.
-
Any new updates? Not sure if the conversation moved to discord mostly at this point. It looked like you guys were on the right track. Very exciting to think of mp in these great conversions. Wish I had the skill set to assist. GL
-
Well, I haven't seen ksotar around lately, but since finally getting into FSO's code, I'll admit that I've been working on this problem. On the client side, we've committed a timing fix that improves the accuracy of the interpolation somewhat. This was *just* committed in the past week or so.
I also have a few other interpolation upgrades, but they only help the client calculate more accurately, and they require a multi version bump.
We have the other problem that if latency is too high, the client can't update the server fast enough so that once the server knows to fire, the target that it used to be shooting at is long gone.