Hard Light Productions Forums

Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: Cobra on October 14, 2016, 10:56:09 pm

Title: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: Cobra on October 14, 2016, 10:56:09 pm
This is more or less a thread to find out how widespread this issue is. Trivial Psychic said it best in the Derelict thread:

Rather than opening a new thread, I thought I'd simply add onto this one.the screen suddenly started whipping around as though someone was doing rapid circles with my stick's hat and it was going rapidly from one view to another.  I managed to clear this by tapping the afterburners, but when the screen settled, the Primary Objective Complete message was up and it stayed up indefinitely.  Then, nearly all of my directives disappeared and slowly started returning, one by one, over the next several minutes.  Also, I could no longer fire primary weapons or my non-Treb bank of secondaries.  I then proceeded to target and destroy every subsystem, and over time, eventually I regained fire control over the other secondary bank (armed with Tornadoes) and my primary banks.  Things appeared to be back to normal (except for the Objective Complete message), and I was getting the Orion's hull pretty low, when it happened again.  I continued my attack until at last the Auriga's hull failed and she exploded.  I had moved some distance out, but was still buffeted by the shock-wave.  I decided that it was time to warp out, and it would begin the power-up sound, but never opened the vortex.  I then realized that I no longer had control over my direction.  I could turn the ship completely around at any angle, but was always traveling in the same direction.  I even tried my afterburners, but they would only accelerate me in the direction I was traveling, not how I was facing.  Eventually, I gave up and abandoned the mission, and restarted Freespace.  I got to the same point and the same screen jumble triggered again.  I aborted again at that point and restarted, deciding I wouldn't try to completely disarm the Auriga, but instead would just create a safe zone from where I could attack in safety, but the same effect triggered again.  I decided to just try to warp out, and it worked.  There was no weirdness in the debriefing, and I moved on to the next mission, but I was still not sure if this was a code anomaly, or some kind of elaborate FRED technique to prevent the player from taking out a campaign-critical enemy target.  It should be noted that I am using the new Leviathan and I was using an Win64 build from this month, though I may have also tried it with a build from Sept. 30th.

This bug happens with varying intensity. I literally encountered this not five minutes ago while playing Blue Planet. Everything described happened here. In fact, it can be progressive. Most times I can mostly continue as normal and was lucky enough for the bug to happen after the mission was completed and be able to jump out, but during the (final) Blue Planet mission it got to the point where it was if I was stuck in constant glide mode, and nothing I did could pull me out of it.

Further symptoms of the bug include the inability of most ships to fire primaries (I witnessed one or two fighters firing primaries out of the twenty or so on the field), capital ships take no damage, and events sometimes just stop triggering. It also occurs in Win32 builds and has been present prior to the 9/30 build, but how far back it goes I can't tell you. And I still can't quite discern the cause of it. For me, sometimes it happens after I crank up the time compression; sometimes it happens when I'm buffeted by a shockwave; sometimes it happens for seemingly no reason at all.

For the hundredth time, no, this is not caused by TrackIR being used, because the TrackIR .dll is not present. It could be something with the TrackIR code, but if that was the case the events wouldn't be cycling through the five views like I was waggling the HAT like a madman. Also, if the only thing you have to say is "your install is possessed/haunted/whatever", **** off, because you are not helping.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: niffiwan on October 15, 2016, 12:37:37 am
I have not experienced this issue. So I guess we need to try to narrow down the possible causes with lots of boring testing.

What builds have you had the experience with? In your controls, is there anything bound to glide? Can you see if messing with the various glide keybinds triggers the issue? Which Blue Planet mission were you playing?

What mission in Derelict did Trivial Psychic have the issue? Maybe this one?
Quote
Firstly, in the mission DL2-10 "Descent" when the Auriga makes its first appearance,
I'm not familiar enough with the campaign to know what it is from the description. That might be a good one to test with since it seemed to occur twice in the same place?
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: m!m on October 15, 2016, 04:09:32 am
I think I had the same issue when I was playing the BtA demo. Is the issue still present when you try a nightly from before 12 August 2016 (http://www.hard-light.net/forums/index.php?topic=92383.0)? There were a few changes to the view code which may have introduced this issue.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: tomimaki on October 15, 2016, 06:27:56 am
I've encountered this bug recently in final STR mission but when replayed it didn't happen again.  :sigh:
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: niffiwan on October 16, 2016, 07:14:09 am
Aha, I've just hit the bug in a FotG mission. I was playing in debug so I've got a log... which is mostly useless (see below after I removed 1,516,104 lines of OpenGL Debug messages from it (no, I'm not kidding))

Code: [Select]
CSG => Saving complete!
vm_forward_interpolate: Bad rotation
vm_forward_interpolate: Bad rotation
vm_forward_interpolate: Bad rotation
vm_forward_interpolate: Bad rotation
nothing ok to fire
vm_forward_interpolate: Bad rotation
vm_forward_interpolate: Bad rotation
59651 frames executed in  30.001 seconds, 1988.302 frames per second.
nothing ok to fire
vm_forward_interpolate: Bad rotation
Warning: Too many triangles in shield hit.
nothing ok to fire
nothing ok to fire
Warning: Too many triangles in shield hit.
vm_forward_interpolate: Bad rotation
Warning: Too many triangles in shield hit.
Got event GS_EVENT_MISSION_LOG_SCROLLBACK (22) in state GS_STATE_GAME_PLAY (2)
...

While my HUD had the shakes, I could fire secondaries but not primaries. It recovered after a while (maybe when I triggered my "burners"), but still couldn't fire primaries. I also lost most of the "centre" of my HUD, everything except the reticle itself. And just prior to the issue occurring, I lost a custom part of the HUD, I don't know the mission well but I think it's specific to the mission (although I can't see how it's done from a quick scan through the mission)

Lastly, my mission log seemed to stop around 9:41 when a turret was destroyed, but my HUD mission clock (after everything recovered and a bit more flying around) said 20+ mins. Which seems super odd as I recall the turret being destroyed not much before the issue started, and I'd swear the problem & my flying around after the recovery didn't last anywhere near 10 mins...  I wonder if the issue is time related; can anyone else comment on whether they saw the issue around 10 mins into a mission?

The FSO version is... around Oct 11.

edit: primaries starting working again eventually... except now after a collision I'm flying away from everything slowly, and I can't stop it. Makes no sense; it's like I'm gliding, but FotG has disabled the gliding keys and no ships have glide... so HTF was it applied to my ship?

I also have the screens-cam script running, it's broken as well, when activated the cam-ship isn't moving anywhere near as fast as it normally does.

edit2: WAIT A MINUTE. The turret I destroyed just before the bug triggered was in the log with the time 9:34:58.  The other entry was a turret I destroyed only with secondaries after the shakes stopped... it's log time is 9:41:41, but I'm pretty sure the HUD mission clock at that time was around 20 mins!?!

edit3: Yep, something timing related is completely borked. I managed to fly back towards my target, what I thought was glide wasn't, it was more that my velocity was changing excruciatingly slowly. When I got back in range, I destroyed another turret... log time says 1:51:03 (i.e. time ~has~ gone backwards), while the HUD clock says 31:25.  A bunch more turrets destroyed and the log clock has 6:19:35, in the same time the HUD clock has reached 33:22. Log time is travelling 3x faster than HUD time... and it's wrapping around at about the 10 min mark?  And I just noticed that the log clock wrapped at least once before the shakes occurred, so maybe the shakes were triggered around the 20 min mark of the mission, not the 10 min mark?

(http://i.imgur.com/XYtw1iO.png)
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: Cobra on October 16, 2016, 08:46:32 am
Wow. That's definitely some weird **** happening right there. Especially when I thought it was merely timing to blame.

It occurred to me that nearly every instance of this bug happened after ten minutes of in mission time, so there's commonality there. The only time it didn't was the very first time I remember encountering it, which was the first mission of Age of Aquarius in the first few seconds. I thought it was a weird physics bug and thought nothing of it. It didn't happen again until apparently months later.

Your wraparound time is something I haven't seen, though, but you were going more in-depth to find it. I'm not really sure what to say about it, except for the fact that I'll need to replay Delenda Est from Blue Planet again to see if it's 20 minutes long. Most of the missions didn't exactly last that long.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: mjn.mixael on October 16, 2016, 08:47:21 am
 :eek2:
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: Mongoose on October 16, 2016, 10:23:05 am
All hail the dread lord Codethulhu!
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: m!m on October 16, 2016, 12:33:15 pm
I think I can reliably reproduce this. If I go to x64 time compression and then use the controls (I used the numpad) I can sometimes reproduce this bug. I am using a very simple test mission that just has another Leviathan in it. It doesn't happen always but if I keep rotating the ship is happens pretty much all the time and I can stop the weirdness by using the afterburner so this is probably the same bug everyone is experiencing.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: Cobra on October 16, 2016, 07:41:49 pm
Okay, bug reproduced in same spot of Delenda Est, at approximately 18 minutes, 12 seconds. I also found the time in the event log to have wrapped around at least three times. The event log's "current time" does not match the HUD time nor the times listed in the log.

...did the log viewer always have 0:00:00? :wtf: [EDIT] Okay, no. For some reason I thought the log time was showing hours instead of minutes.

Hypothesis: The timer resetting multiple times is the cause. The desynchronization and constant restarts cause a buildup of some sort, and with sufficient buildup the engine freaks out because it has no idea where it's supposed to be at the present state in time and everything just freezes.

I know nothing of coding but this is what seems to be happening from my point of view.

[EDIT 2]
Time in mission:
(https://dl.dropboxusercontent.com/u/71111768/screen0353.png)
Times in log:
(https://dl.dropboxusercontent.com/u/71111768/screen0352.png)
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: AdmiralRalwood on October 16, 2016, 08:55:03 pm
It seems less likely that the timer is wrapping around and more likely that the 10s digit for minutes is simply not visible (if there were a wraparound, it shouldn't happen at a decimal value); however, this could still be an internal timer-related issue.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: niffiwan on October 16, 2016, 09:13:09 pm
From my experience I think that (after the issue occurred) the timers were not incrementing at the same rate, which points to some sort of timer issue.  Probably need a better test mission, with (e.g.) ships set to arrive every second to provide a steady stream of log entries to match vs reported mission time.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: Jackho on October 17, 2016, 12:20:49 am
I had this bug too. Don't have anything more to say to describe it that hasn't been already stated in previous posts. I began to have this bug when trying build 778cb6f from 4 october for the first time. I first thought it was some kind of regression introduced in it. Or maybe I was only experiencing weird things on my computer. So I tried different builds until bd5fb38 from october 14 then I stopped trying. I still use this version today. Thing is, this bug happens pretty regularly, in every mission I tried to reproduce it and it happened the same way.
It triggers at 00:18:15 modulo 1 or 2 seconds depending I was careful enough (looking at display clock)
And it doesn't happen at all in 3.7.4 final so I use it instead when I need to play 20mins+ long missions.
Oh, and also using time compression did break a mission in BP WiH but I can't tell which one at the moment because I'm not home. But it didn't happen on any other one so it may not be related to this bug.

Quote from: niffiwan
...ships set to arrive every second ...
That would be 1095+ ships on screen  :nod:  :banghead:


My 2 cents

Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: m!m on October 17, 2016, 02:14:00 am
There is definitely something wrong with the timers. When I reproduced the bug my secondary weapons HUD gauge showed a cooldown although I didn't fire it so the timer probably resets for some reason. I am reasonably sure that it's not a data corruption bug since it happens in debug and release and it's highly unlikely that a write to a random address always hits the timestamp counter. The timestamp code has a check that resets the counter if it gets too big so that is the first place I am going to check.

EDIT: ****! I think I found the issue. When I converted the internal timestamp ticker from milliseconds to microseconds I didn't adjust the overflow control so instead of wrapping around when 12 days have gone by it now wraps around when 17.8 minutes have gone by. That matches pretty much exactly what Jackho reported so I'm reasonably certain that this is the cause of this bug.

EDIT2: Fix submitted. If it gets merged quickly it will show up in the nightly from today.

EDIT3: Fix has been merged. It will be present in the next nightly.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: Cobra on October 17, 2016, 03:03:52 am
Fantastic. This was one of those things where people needed to be brought together to find the error, and I really hope that was the cause. This is an annoying-as-hell bug that gets in the way of everything.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: chief1983 on October 17, 2016, 09:15:44 am
How long back are nightlies affected by this then?
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: m!m on October 17, 2016, 11:30:06 am
The PR that added the bug was #708 which was merged on September 1st so all nightlies after that should have the bug.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: Cobra on October 17, 2016, 02:37:21 pm
Well, either through bug-fixing or by luck, the Delenda Est mission went without a hitch. Whenever I have some time to play extensively I'll do that to make sure it's gone.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: FrikgFeek on October 25, 2016, 03:59:41 pm
I just hit this bug with the Oct 24 nightly, win x64.
(http://i.imgur.com/db4YjLC.png)

In-game timer was around 24 minutes. Log timer shows a wraparound and claims the mission time is 6:24.

Also, the log timer was switched from H:MM:SS to M:SS:CS some time after 3.7.4 which is a bit weird. The centisecond timer also seems to loop around at 60 instead of 100 which causes times to get desynced. And the minute timer loops around at 10 minutes, something which would be a bit more logical with an hour timer.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: Cobra on October 25, 2016, 04:58:11 pm
Wait, are you saying the view freakout happened again? Or is it just the timer craziness?
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: FrikgFeek on October 25, 2016, 05:04:25 pm
Just timer. Can't confirm any other effects but it really can't be good to have that timer so de synced.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: Cobra on October 25, 2016, 05:17:31 pm
I suppose the timer being screwed up could have side-effects like changing the timing of SEXPs, but the "game breaking bug" appears to have been fixed, since I haven't experienced it since the 14th nightly.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: mjn.mixael on October 25, 2016, 10:06:06 pm
The bug did somehow stop that mission from proceeding.. so something's not quite right. He played that mission again on 3.7.4 and it worked just fine.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: Cobra on October 26, 2016, 02:46:13 am
How is it that I can play a 20 minute long mission and not have an issue? :wtf: Especially after the 14th nightly allowed me to progress past a point where the engine broke consistently.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: FrikgFeek on October 26, 2016, 07:20:15 am
As I said, I can't 100% confirm that the timing issue is at fault here without more testing, but It might be throwing off some time-based SEXPs.
And even if it's not and SEXPs work just fine, it would still be nice to have the log show accurate mission times rather than counting a second for every 0.6 seconds.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: Bryan See on November 15, 2016, 11:21:37 am
I'm the only one who got many game-breaking bugs, including "not responding" and CTDs. Especially during the mission loading screen (can be resolved by some streaming mechanism), rendering of large numbers of models and effects (both of which can be resolved by instancing).
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: chief1983 on November 15, 2016, 12:43:41 pm
Bryan, can you elaborate on your mission load crash?  I have seen one on Macs in recent builds that happens in the audio loading code, after starting to load beep_1.wav.  What streaming mechanism are you referring to?
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: Bryan See on November 16, 2016, 12:54:29 am
The mission loading part. During this, I've made Alt+Tab to switch to another window in Windows 10. A crash, but a hang.

EDIT: I've also encountered this during rendering of beam effects at some point.
Title: Re: Very much game-breaking bug present in current builds. Who's experienced it?
Post by: AdmiralRalwood on November 16, 2016, 01:29:26 am
The mission loading part. During this, I've made Alt+Tab to switch to another window in Windows 10. A crash, but a hang.
...Yeah, don't do that. FSO isn't actually hanging, but it has stopped responding to operating system events, so Windows thinks it has. (This was fixed in 3.7.5, but that obviously won't help anyone still using 3.7.4.)