Author Topic: Multiplayer issues in mods  (Read 2692 times)

0 Members and 1 Guest are viewing this topic.

Offline ksotar

  • 26
Multiplayer issues in mods
We have a plan to run some multiplayer sessions for 12 players in SWOT mod.

First tests with FS2 Open (3.7.2-3.8.1, whatever) showed that there are indeed issues for everyone except player who is a server to score a hit with primary weapons. Sessions were 4 players max for now.

And it looks somehow different from the issues that I read about on forums. My observation are following:
  • Looks like it is not big deal to score first hit: it clearly has effect, enemy ship starts some burning, etc.
  • Following projectiles seem to hit as well - all due SFX are performed. But it seems to have no effect on actual enemy health. And we could never bring even a single ship down.
  • If we managed to survive the mission, stats page confirm exactly that: first hit into a ship counts, but non other do.
  • We tried FSO with MediaVPs 2014 only and looks like problem is gone, at least we are able to kill enemies, not sure if with the same ease as the server player.

This looks somehow weird for me, since mods do not alter the code or multiplayer functionality - what can be the reason of this?

Is multiplayer playable for 10+ pilots with just FSO without mods?

What can be done about that? We have at least two willing C++ developers who could try to fix it somehow, but they need someone at least to point them in right direction.

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: Multiplayer issues in mods
Only the first hit counts, even on non-moving ships?   Wow, that might actually help us narrow down a possible cause.  I would definitely like that to be investigated further.
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 

Offline ksotar

  • 26
Re: Multiplayer issues in mods
Only the first hit counts, even on non-moving ships?   Wow, that might actually help us narrow down a possible cause.  I would definitely like that to be investigated further.
Well, non-moving ships I didn't guess to check, will definitely do that.

Need to say that my observations were for AI ships only, since we were playing coop missions.

 

Offline m!m

  • 211
Re: Multiplayer issues in mods
One thing which would probably improve the user experience would be to use client side hit detection. As you mentioned, the later shots cause the visual and audio effects of a hit but no damage was being dealt. If instead the client sent the server a message that said "I just hit object A with weapon B at position C" the server could use that information for updating the health of the object without relying on server side hit detection.

That would also solve the lag issue where you could never be sure if you actually hit the object even though the visual effects of the game indicate that you did hit it.

 

Offline jg18

  • A very happy zod
  • 210
  • can do more than spellcheck
Re: Multiplayer issues in mods
Is multiplayer playable for 10+ pilots with just FSO without mods?
Unfortunately, at the moment it isn't even playable for two without mods .


What can be done about that? We have at least two willing C++ developers who could try to fix it somehow, but they need someone at least to point them in right direction.
Cool that you know people who are interested in helping out. :) Right now we have no idea what the cause is.

At this point I've spent upwards of 100+ (maybe 200+?) hours studying the netcode and related resources and know a fair bit about how it works, even if I'm admittedly a little rusty on it. Nonetheless I'm happy to be a resource.

How familiar are the C++ devs you know with netcode in general and with real-time action multiplayer game networking in particular? Prior knowledge of the latter is obviously helpful but not strictly required.

Only the first hit counts, even on non-moving ships?   Wow, that might actually help us narrow down a possible cause.  I would definitely like that to be investigated further.
I agree, I think this is an interesting result. Good find. :yes:

m!m: I'm not sure client-side hit detection is the answer or even a key part of the answer, at least for FS2 retail. I think something more fundamentally wrong is goign on, no idea yet what. Client side hit detection would also introduce questions of, for example, which client has authority over the health of another ship. EDIT 2: Consider the case of multiple ships firing at a single ship, or two ships firing at one another. If the clients are allowed to have some degree of authority (is partial/distrbuted authority even possible?) it might make it unclear how much health a given ship has.

The only way to really know what's going on is to use a packet capture tool like Wireshark and study the results. FSO has some built-in network profiling functionality that may also help.

As a next test, I'd try a PvP mission like dogfight. Try sitting still and shooting at each other and see whether health consistently drops. The server has authority over all client's health and sends clients updates about their own health, albeit not often, only 3x/second IIRC. One of the experiments I wanted to try was increasing it to 10x/second with some changes to ignore out of order "client update" packets, but I haven't gotten to it yet.

EDIT: I have a software engineer friend who, while not familiar with C++, has an interest in performance analysis. I've been meaning to work with him and see if we can make any headway in figuring out what's going on.
« Last Edit: April 14, 2018, 12:14:29 am by jg18 »

 

Offline ksotar

  • 26
Re: Multiplayer issues in mods
Thank you all for your answers and suggestions.

Non of those guys has netcode experience, but anyways they want to try.
Maybe discord channel on that matter?

Below I will be talking about 3.7.2 only, since my notebook that I used as second machine (mainly server) has no OpenGl 3.2 support (damit, Intel!).

I have created mission with non-moving AI ships. At first try I found out that first hit always counts, but not all consecutive hits are ignored, every now and then I was able to hit again, while some large sequence of shots (10+) had no effect on health. But then I decided to switch server and client, and since then I couldn't reproduce the problem. Every hit count from now on, I tried many times switching which machine acts as server, that doesn't matter.

Then I went back to moving targets. I can say that "First hit always scores" seems busted. I guess I had this impression since first hit in SWOT causes TIE fighter to start burning, loose some hull parts and play hit sound, moreover, hp% in small target preview started to flash, as it happens when you actually hit, but mainly, hp% doesn't go down. So, my mistake :(

But I managed to record synchronous video from the client and server, while server was dead and in observer mode, then you can switch view to show client cockpit. So we can see and analyse how client info appears to be on server side. Maybe it will give us some hints, I'll try to upload it soon.

 

Offline ksotar

  • 26
Re: Multiplayer issues in mods
Here is the video. I hope it could provide some insights.

I had to rise brightness for TIE fighters to be seen clear, hence gray Space.

On the server HUD is missing - it happens from time to time then I use multiplayer, IDK why (all HUD settings just switche to off), so we don't have even a crosshair, but cockpit seems to be enough to judge object positions.


Fullscreen 1080p view is recommended.
https://www.youtube.com/watch?v=tinl9RIEBdY
You can use "<" and ">" on Youtube to skip frame-by-frame. Video is 25 fps, hence 40ms per frame.

I have made some series of screenshots with this video to illustrate some of my conclusions, I'll post them later.

PS I used "Alliance - Beat The Odds (Coop)" mission from SWOT, just disabled weapons on enemy ships, and had to kill the server by myself for it to become observer.
« Last Edit: April 15, 2018, 12:24:15 pm by ksotar »

 

Offline ksotar

  • 26
Re: Multiplayer issues in mods
BTW I've lost one of the "willing developers". He looked at the code, and ran streaming in horror into the night.

But he passed me a clever idea. If we could somehow (perhaps with some cheat) activate AI to fly client ship, we could use it for automated testing.
AI flies vs enemy with disabled weapons as a server first, then we have a statistics of  its hits, it is considered an absolute goal for client side.
After that we ran it for the client and can judge buy its improvement if we fixed something for good.

Am I making myself clear here? :) What do you think?

 

Offline jg18

  • A very happy zod
  • 210
  • can do more than spellcheck
Re: Multiplayer issues in mods
That's too bad about your dev friend. Unfortunately I don't understand his idea; could you explain it in more detail?

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: Multiplayer issues in mods
I think it's something like this:  instead of all AI ships being controlled by the server (and therefore all there shots originating on the server, the same as if it were the player on the server), we have a setup where some AI ships were actually controlled by a client, and the shots originate on the client, directed to another ship (either one controlled by server or client).  This would be a bit of code work I think, to have clients get involved in controlling AI ships.  There might be an easier way to cheat though, and that is to directly control the client player's own ship via SEXPs or something, instead of client-controlled AI ships.  We could probably still achieve the intended goal of automating the behavior, and not require any changes to the engine.
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 

Offline AdmiralRalwood

  • 211
  • The Cthulhu programmer himself!
    • Skype
    • Steam
    • Twitter
Re: Multiplayer issues in mods
It read more to me as a troubleshooting step: have "players" controlled by AI to see how many hits are registered, and using that as some sort of performance metric. I'm just not clear on why, exactly, AI needs to be involved.
Ph'nglui mglw'nafh Codethulhu GitHub wgah'nagl fhtagn.

schrödinbug (noun) - a bug that manifests itself in running software after a programmer notices that the code should never have worked in the first place.

When you gaze long into BMPMAN, BMPMAN also gazes into you.

"I am one of the best FREDders on Earth" -General Battuta

<Aesaar> literary criticism is vladimir putin

<MageKing17> "There's probably a reason the code is the way it is" is a very dangerous line of thought. :P
<MageKing17> Because the "reason" often turns out to be "nobody noticed it was wrong".
(the very next day)
<MageKing17> this ****ing code did it to me again
<MageKing17> "That doesn't really make sense to me, but I'll assume it was being done for a reason."
<MageKing17> **** ME
<MageKing17> THE REASON IS PEOPLE ARE STUPID
<MageKing17> ESPECIALLY ME

<MageKing17> God damn, I do not understand how this is breaking.
<MageKing17> Everything points to "this should work fine", and yet it's clearly not working.
<MjnMixael> 2 hours later... "God damn, how did this ever work at all?!"
(...)
<MageKing17> so
<MageKing17> more than two hours
<MageKing17> but once again we have reached the inevitable conclusion
<MageKing17> How did this code ever work in the first place!?

<@The_E> Welcome to OpenGL, where standards compliance is optional, and error reporting inconsistent

<MageKing17> It was all working perfectly until I actually tried it on an actual mission.

<IronWorks> I am useful for FSO stuff again. This is a red-letter day!
* z64555 erases "Thursday" and rewrites it in red ink

<MageKing17> TIL the entire homing code is held up by shoestrings and duct tape, basically.

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: Multiplayer issues in mods
That's the automated part, guess it seemed like it would be easier to have AI ships involved than player ships.  Maybe you could have more than two ships involved that way as well, or use it with just one user on a client connected to a standalone server, and still have multiple ships involved in the test.
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 

Offline ksotar

  • 26
Re: Multiplayer issues in mods
That's too bad about your dev friend.

Well, eventually  he agreed "to do some experiments", so who knows, there is hope :)

Guys here captured his idea with AI quite right, I think.

Quote
using that as some sort of performance metric. I'm just not clear on why, exactly, AI needs to be involved.

Well, I have flown  like 5 hours in this single mission - quite tiresome and repetitive task, so I think, some automation can be handy here.

Quote
That's the automated part, guess it seemed like it would be easier to have AI ships involved than player ships.  Maybe you could have more than two ships involved that way as well, or use it with just one user on a client connected to a standalone server, and still have multiple ships involved in the test.

You mean, some kind of wingmen for each client player? I thought that they'll be controlled by server anyways?

 

Offline ksotar

  • 26
Re: Multiplayer issues in mods
So, we had pretty thorough test session with pure FS2 (MediaVPs only) this Sunday.

There were 4 players including myself, 3 on LAN with ping <10ms average (while FS2 reported 24-40ms) and one was connected via Internet from remote location. Server was the 5th ship (I haven't find a way to run Windows standalone server) and it was not played, since we wanted to test client side.

I need to say, that it was pretty playable. My hits % with primary went from 1% (SWOT mod) to 13% (I had more or less same result if I played on server side). I almost had none of that frustration feeling "I certainly should hit him, but it doesn't score".

I'm quite sure that there were some hits missed due to client/server desynchronization (I made a second video on it) but it seems minuscule compared to SWOT mod.

Our friend with external connection could sometimes jump around (especially when your projectile hits him - I suppose that's the moment when the server understands that coordinates of his ship on our clients went really wrong and fixes it). But other then that - nothing.


So, what would you say, what can be so different with mod that it goes from "pretty much playable" to "it almost impossible to score a hit"?


One of the problem we faced all the time - often we started mission with no HUD. Then we were to go to settings and reset it (all features appeared "off" somehow). Should I report it as a bug on git? Problem is that we can't run 3.8.0 and higher builds - there is a bunch of problems with hardware compatibility with those.
« Last Edit: April 23, 2018, 11:34:21 am by ksotar »