Hard Light Productions Forums

Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: Nightmare on August 17, 2018, 01:10:10 am

Title: Shaking Skyboxes
Post by: Nightmare on August 17, 2018, 01:10:10 am
I have noticed that the skyboxes start to shake or vibrate when large/hi-poly ships (not entirely sure but I think it's the latter) are located at positions which are greater than ~150km from the 0,0,0 point of a level. This occurs whenever the player ship moves, either around or while maneuvering. Once the ship stands still the vibrating ceases. This might not effect that many people :D , but it's a bug so I thought I report it.
Title: Re: Shaking Skyboxes
Post by: The E on August 17, 2018, 04:38:52 am
It's not a bug, exactly. What you are seeing is a result of the game engine becoming unstable due to floating point inaccuracy.
Title: Re: Shaking Skyboxes
Post by: Nightmare on August 17, 2018, 12:52:20 pm
So that probably means there's nothing that could be done about it. :(
Title: Re: Shaking Skyboxes
Post by: m!m on August 18, 2018, 07:59:58 am
At least for the skybox it should be easy to fix this since the actual rendering position of the skybox does not matter.
Title: Re: Shaking Skyboxes
Post by: The E on August 18, 2018, 09:23:38 am
At least for the skybox it should be easy to fix this since the actual rendering position of the skybox does not matter.

I am not sure if the camera orientation stays stable enough for that.
Title: Re: Shaking Skyboxes
Post by: Nightmare on August 18, 2018, 10:47:07 am
Actually while I pushed the engine pretty far, with several hundred (about 220) objects being more than 150km from the mission center, some as far as 2200km away, the issue with the skybox was the only problem I've noticed.

Out of curiousity, do the rendering issues on very big ships also result from this "floating point inaccuracy"?
Title: Re: Shaking Skyboxes
Post by: Cyborg17 on August 18, 2018, 12:27:21 pm
Could you be more specific?  Are they rendering issues that happen at large distances, and what exactly happens?
Title: Re: Shaking Skyboxes
Post by: Nightmare on August 18, 2018, 12:32:29 pm
The model starts to flicker, some parts are rendered, some not. At first it happens to submodels (turrets etc/at about 100km size), then the main model.

It's a known error though, the first time I've seen it was with the planet models in INFR1.
Title: Re: Shaking Skyboxes
Post by: m!m on August 18, 2018, 12:39:35 pm
I am not sure if the camera orientation stays stable enough for that.
Is the camera orientation dependent on the camera position? Intuitively I would assume that it wouldn't be but I can't check the code right now.
Title: Re: Shaking Skyboxes
Post by: Zipper730 on September 22, 2019, 03:39:26 pm
Would making all space using floating points work around the problem?  I've heard the idea mentioned once on a Discord chat -- I have no idea if it'd work.
Title: Re: Shaking Skyboxes
Post by: The E on September 23, 2019, 03:21:27 am
Would making all space using floating points work around the problem?  I've heard the idea mentioned once on a Discord chat -- I have no idea if it'd work.

Not exactly. This issue exists precisely because we are using floating point variables to store position and orientation data. There are only two real fixes available here:
1. Use double-precision floating point math. This does not fix the underlying issue as much as it papers over it; this also has unfortunate sideeffects as many data structures in the engine now take up literally twice the space. This has performance implications as well: We gain a lot of speed using SSE instructions on some of our math operations, which require that they be done using single-precision floats. Plus, there is rendering performance to consider: While modern GPUs are incredibly fast even in double-precision workloads, they are still much slower than they are when doing single-precision calculations.
2. Always treat the player or camera as the origin point of the global coordinate system. This is a technique that is used by modern streaming game engines like Unreal; since every important action will happen within the zone of highest precision, float inaccuracy will never be an issue. This would require extensive redesign of the physics system to make work in FSO.

TL;DR: Yes, there are ways around these issues. However, implementing them requires a large amount of effort, and so it is unlikely to ever get done.
Title: Re: Shaking Skyboxes
Post by: Nightmare on September 23, 2019, 03:34:26 am
At appears that the skybox issue could be fixed though. When I set a mission in subspace (which- to my limited understanding - acts basically like a skybox as well) I'm not having any problems even when I put my ship 500km off the mission center.
Title: Re: Shaking Skyboxes
Post by: General Battuta on September 23, 2019, 07:36:25 am
Does anybody actually need this fixed? What’s the use case here?
Title: Re: Shaking Skyboxes
Post by: Nightmare on September 23, 2019, 10:20:48 am
Well I run into it from time to time. It's not like it's going to be used in every campaign from then on, but it's easier to built, say, multi stage missions the more space is available. If it'd take a coder many hours to fix it, the time would surely be invested better elsewhere; but if a viable solution already exists in the code (of which I have no idea), why not have one issue less? The skybox thing is the first (and most obvious) thing that will go wrong with distance related issues - having that done could bump usable space from ~70 to about 100km.
Title: Re: Shaking Skyboxes
Post by: 0rph3u5 on September 23, 2019, 10:51:52 am
(this was wirtten before Nightmare's post)

Considering that LUA script have the potential to normalize in-mission-jumps, checkpoints as well as the normalisation of hull repair in newer mods, increased spatial requirements were bound to come up. Esspecially if you were going for missions where the player would launch from a port and do sub-missions by jumping back and forth, as well as requiring the "port area" to be in a constitent state (so you wouldn't just ship-vanish and ship-create it over and over again).

However I think, this doesn't require the so much to alter the overall mission space avalible but to add tools to effect the illusion of distance. As distance is percieved through reference points (that how you do optical illusions and in-camera effects to shrink or grow objects in film) that wouldn't take much in FS_Open as per se the mission space doesn't have reference points from the player perspective that aren't placed by the FREDer. The only obstacle to effect illusion of distance there would be the distance measure in the HUD, if the any far away objects could still be targeted that is.

Using a custom HUD configuration you can already use $Length Unit Multiplier: (https://wiki.hard-light.net/index.php/Hud_gauges.tbl#.24Length_Unit_Multiplier:) and $Speed Unit Multiplier (https://wiki.hard-light.net/index.php/Hud_gauges.tbl#.24Speed_Unit_Multiplier:) to affect distance measure in the HUD globally.
To more control an illusion of distance here then, could be affected by on a case by case basis or globally by on a "if distance >{any value} then multiply all distance on the HUD by {any factor}" or through "add {value} to distance displayed" ... that way you could actually shrink the absolute distance between "areas" in a multiple in-mission jumps mission without blowing up the spatial requirements.
Title: Re: Shaking Skyboxes
Post by: General Battuta on September 23, 2019, 10:58:37 am
I hear you on the multi-stage mission thing, but in practice, I’m having a hard time imagining ACTUALLY needing more space. Universal Truth 2 seems to run without any noticeable skybox issues, and it’s tough for me to imagine a mission ever requiring more stages and more random crap than it does.

Orpheus, you don’t really need more space to do in-mission jumps, even with a persistent ‘hub’ where you want stuff to keep happening. Everything can happen in the same overlapping space. There should be enough area to move your ‘hub’ far off, past the point of visibility; and lots of tricks to hide it if not.
Title: Re: Shaking Skyboxes
Post by: General Battuta on September 23, 2019, 11:00:56 am
Worth noting that not even Elite Dangerous, a game which builds entire solar systems at full scale, actually uses large playable spaces - just a TON of instancing. Even EVE Online only creates physically simulated areas where players exist. There’s no attempt to keep the whole available solar system ‘on grid.’
Title: Re: Shaking Skyboxes
Post by: 0rph3u5 on September 23, 2019, 11:12:36 am
Orpheus, you don’t really need more space to do in-mission jumps, even with a persistent ‘hub’ where you want stuff to keep happening. Everything can happen in the same overlapping space. There should be enough area to move your ‘hub’ far off, past the point of visibility; and lots of tricks to hide it if not.

I know, but there is also no compelling reason not to do what I described. :)
Title: Re: Shaking Skyboxes
Post by: X3N0-Life-Form on September 23, 2019, 12:03:23 pm
Except maybe shaking skyboxes'n'stuff.
Title: Re: Shaking Skyboxes
Post by: Nightmare on September 23, 2019, 12:11:37 pm
Well it was just an idea, not a request; in the end it's the up to the coders if they feel like looking at that part and whether the use/needed time is good or if they're interested at all. :)
Title: Re: Shaking Skyboxes
Post by: mjn.mixael on September 23, 2019, 01:30:18 pm
BtA has a handful of missions that jump all over the place and we never ran in to these issues. You don't actually need to move the player that far for a new sub-mission space. It's complex, but with sexps you can hide small ships entirely from targeting/radar. at 10-15k out from the action, it's highly unlikely the player will stumble upon it. With larger ships, you can move them even further and then bring them back. You could also use the shipsave script for easier removal/re-add to the mission space. These are just a couple of the tricks I used. I'm sure Battuta and Axem have used even more advanced tricks. Simulating these kinds of missions are very possible with clever FREDing.
Title: Re: Shaking Skyboxes
Post by: DefCynodont119 on September 23, 2019, 02:03:58 pm
I'm pretty sure you can also make things invisible too, there is no need to taunt the deep space kraken with floating point issues if you can reuse the same patch of space over again with clever Fredding.
Title: Re: Shaking Skyboxes
Post by: General Battuta on September 23, 2019, 02:31:34 pm
BtA has a handful of missions that jump all over the place and we never ran in to these issues. You don't actually need to move the player that far for a new sub-mission space. It's complex, but with sexps you can hide small ships entirely from targeting/radar. at 10-15k out from the action, it's highly unlikely the player will stumble upon it. With larger ships, you can move them even further and then bring them back. You could also use the shipsave script for easier removal/re-add to the mission space. These are just a couple of the tricks I used. I'm sure Battuta and Axem have used even more advanced tricks. Simulating these kinds of missions are very possible with clever FREDing.

You could use SOMEBODY'S cloaking to make them invisible!

Well it was just an idea, not a request; in the end it's the up to the coders if they feel like looking at that part and whether the use/needed time is good or if they're interested at all. :)

It's up to the designers first to figure out if it's a feature anyone actually needs.
Title: Re: Shaking Skyboxes
Post by: General Battuta on September 23, 2019, 02:33:21 pm
I know, but there is also no compelling reason not to do what I described. :)

Well, except that...it doesn't work, right? That's why we have this thread, because you can't move stuff too far from the origin without hitting bugs. Or am I misunderstanding...?
Title: Re: Shaking Skyboxes
Post by: mjn.mixael on September 23, 2019, 03:27:42 pm
I always forget about the cloaking feature. Yes, that works too! Lots of options already in the engine to handle this sort of mission. :)
Title: Re: Shaking Skyboxes
Post by: Nightmare on September 23, 2019, 03:41:41 pm
Well it was just an idea, not a request; in the end it's the up to the coders if they feel like looking at that part and whether the use/needed time is good or if they're interested at all. :)

It's up to the designers first to figure out if it's a feature anyone actually needs.

No. Sometimes/often, coders simply made new features, and table editors/mission designers took stuff and used it in many different ways. Roughly 1 year ago I received a feature for something I worked on at Scroll (an edit to the fireball.tbl). It didn't even thought it would be necessary and could be handled differently, and I didn't knew I could have much use for, but in the end it turned out to be one of the most useful things I've seen in years.
Title: Re: Shaking Skyboxes
Post by: General Battuta on September 23, 2019, 03:50:00 pm
Yeah, actually, crosstalk is necessary. Coders need designer input or their features don’t always work the way they intend (or meet needs which don’t exist). See: AI classes, which did not function correctly for about...10 years after V implemented them?

You need to have that conversation or features don’t get written for the game as it’s actually used. Case in point: this thread.
Title: Re: Shaking Skyboxes
Post by: Nightmare on September 23, 2019, 04:51:55 pm
If you want something specific, sure. :nod: Lots of things in FRED are unintuitive, despite quite many coders being FREDers.
Title: Re: Shaking Skyboxes
Post by: Zipper730 on September 23, 2019, 04:57:19 pm
Quote from: The E
1. Use double-precision floating point math. This does not fix the underlying issue as much as it papers over it; this also has unfortunate sideeffects as many data structures in the engine now take up literally twice the space. This has performance implications as well: We gain a lot of speed using SSE instructions on some of our math operations, which require that they be done using single-precision floats. Plus, there is rendering performance to consider: While modern GPUs are incredibly fast even in double-precision workloads, they are still much slower than they are when doing single-precision calculations.
What kind of system requirements would be needed to run a game like this smoothly?
Quote
Always treat the player or camera as the origin point of the global coordinate system. This is a technique that is used by modern streaming game engines like Unreal; since every important action will happen within the zone of highest precision, float inaccuracy will never be an issue. This would require extensive redesign of the physics system to make work in FSO.
I know this might be silly, and probably a great deal simplistic, but how?

Quote from: 0rph3u5
Considering that LUA script have the potential to normalize in-mission-jumps, checkpoints as well as the normalisation of hull repair in newer mods
What do you mean by "normalize", "in-mission jumps", and "checkpoints".  If I was to guess in mission jumps, I'd assume like Wing Commander's "Autopilot" function.
Quote
However I think, this doesn't require the so much to alter the overall mission space avalible but to add tools to effect the illusion of distance. . . . To more control an illusion of distance here then, could be affected by on a case by case basis or globally by on a "if distance >{any value} then multiply all distance on the HUD by {any factor}" or through "add {value} to distance displayed"
So, this would basically allow you to set speed-gauge and distance gauge to read erroneously long distances, or extremely high speeds on a case-by-case basis?

Quote from: General Battuta
Worth noting that not even Elite Dangerous, a game which builds entire solar systems at full scale, actually uses large playable spaces - just a TON of instancing.
How large, and what does "instancing" mean?
Title: Re: Shaking Skyboxes
Post by: Nightmare on September 23, 2019, 06:09:37 pm
Quote
What do you mean by "normalize", "in-mission jumps", and "checkpoints".  If I was to guess in mission jumps, I'd assume like Wing Commander's "Autopilot" function.

Checkpoints aren't supported by FS by default, you have to built in everything yourself. Normally, things go this way: mission start. stuff happens. player jumps out. next mission (or restart from beginning). With SEXPs/LUA you can fake the player jumping around inside the mission without actually leaving.
Title: Re: Shaking Skyboxes
Post by: General Battuta on September 23, 2019, 06:49:05 pm
Zipper my dude just play the game a bit, the campaign’s fantastic and seeing how missions are laid out will answer a bunch of your questions.

Checkpoints are mid-mission save points so you don’t have to restart the whole thing if you die/fail.

In mission jumps are basically like autopilot waypoints yeah.

Instancing means creating a chunk of the world for the player to inhabit, rather than simulating the entire world at all times.
Title: Re: Shaking Skyboxes
Post by: Zipper730 on September 23, 2019, 08:50:53 pm
Instancing means creating a chunk of the world for the player to inhabit, rather than simulating the entire world at all times.
Oh, okay: I think that's actually a good idea.

I think this is kind of what I was going for...
Title: Re: Shaking Skyboxes
Post by: General Battuta on September 23, 2019, 11:45:33 pm
Well it’s not something you really do in FSO. The closest analogy is reusing the same mission space with different ships/objects/backgrounds to suggest to the player they’ve moved somewhere else.

Elite does this because it’s an open world game where you can go anywhere in the galaxy whenever you want. Freespace is linear and mission-driven; you load a mission file and do stuff in it, but you’re not going to be dynamically moving between mission files the way you would locations in Elite or Freelancer or what have you.
Title: Re: Shaking Skyboxes
Post by: General Battuta on September 23, 2019, 11:46:00 pm
I have no idea if I’m making any sense
Title: Re: Shaking Skyboxes
Post by: The E on September 24, 2019, 01:01:16 am
What kind of system requirements would be needed to run a game like this smoothly?

I don't know. To the best of my knowledge, there are no real game engines out there that try to actually do this (StarEngine does not, for the purposes of this, count as a real engine).

Quote
Quote
Always treat the player or camera as the origin point of the global coordinate system. This is a technique that is used by modern streaming game engines like Unreal; since every important action will happen within the zone of highest precision, float inaccuracy will never be an issue. This would require extensive redesign of the physics system to make work in FSO.
I know this might be silly, and probably a great deal simplistic, but how?

It's reasonably simple when you look at the math. Basically, what's happening is that instead of the player avatar moving through 3D space, the universe is moved around it. However, converting an engine built on one paradigm to working based on the other is hard: There are so many built-in assumptions that arise from choosing one or the other that rooting them all out and making sure that everything still works after conversion is a hard task.
Title: Re: Shaking Skyboxes
Post by: 0rph3u5 on September 24, 2019, 01:10:19 am
I know, but there is also no compelling reason not to do what I described. :)

Well, except that...it doesn't work, right? That's why we have this thread, because you can't move stuff too far from the origin without hitting bugs. Or am I misunderstanding...?

Yes, you do.

The limitations to that approach are not a compelling reason not to do it, they are just another factor to keep in mind and work with. That's why I suggested means to effect the illusion of greater distance.

Quote from: 0rph3u5
Considering that LUA script have the potential to normalize in-mission-jumps, checkpoints as well as the normalisation of hull repair in newer mods
What do you mean by "normalize", "in-mission jumps", and "checkpoints".  If I was to guess in mission jumps, I'd assume like Wing Commander's "Autopilot" function.

"in mission jumps" would be when the player seems to engage the jump drive but instead of ending the mission as per standard in FS2 doesn't end the mission but is moved elsewhere. Many modern campaigns from Blue Planet: War in Heaven to Vassago's Dirge do exactly that.

"checkpoints"; there are at least one LUA script out there that can take a snapshot of the a mission's current state and save it to or load it from an external file, allowing for a mission to be replayed from a certain point rather than having to do it "from the top" each time. It can be seen in use in -among others- Between the Ashes and Scroll of Atankharzim.

"normalize"; to make normal, commonplace or an accepted standard.
Title: Re: Shaking Skyboxes
Post by: Zipper730 on September 24, 2019, 08:58:27 pm
Well it’s not something you really do in FSO.
Oh well, it was worth a shot!  :lol:

That said, is it possible to stitch a set of coordinate centers like this?

https://ibb.co/b6qzrDZ
(https://ibb.co/b6qzrDZ)

I posted the link and the URL because the image won't load right for some reason.

Quote
Elite does this because it’s an open world game where you can go anywhere in the galaxy whenever you want.
Does elite have an open source game engine?

It's reasonably simple when you look at the math. Basically, what's happening is that instead of the player avatar moving through 3D space, the universe is moved around it.
But since the game engine is based on the universe being stationary and the ship moving through it, is basically the reverse: You'd have to essentially reprogram nearly everything?
Quote
StarEngine does not, for the purposes of this, count as a real engine
Why, if I may ask?
Title: Re: Shaking Skyboxes
Post by: General Battuta on September 24, 2019, 09:36:50 pm
What are you actually trying to do?
Title: Re: Shaking Skyboxes
Post by: Zipper730 on September 24, 2019, 11:26:32 pm
What are you actually trying to do?
I kind of had an idea for a WC Fan Project.  But the idea is that it would allow for a degree of freedom of movement at really high speeds like how the novels allowed.  The idea I had basically was kind of like this...
It's kind of a wishlist, but I figure that you'd need to know what can be done before you can really even attempt to get started.
Title: Re: Shaking Skyboxes
Post by: The E on September 25, 2019, 02:24:39 am
It's reasonably simple when you look at the math. Basically, what's happening is that instead of the player avatar moving through 3D space, the universe is moved around it.
But since the game engine is based on the universe being stationary and the ship moving through it, is basically the reverse: You'd have to essentially reprogram nearly everything?

In an ideal world, no, you wouldn't have to reprogram everything except for a relatively contained portion of the physics engine, but we are not living in an ideal world.
The problem here is that once you choose a paradigm for your engine to follow, programmers will make assumptions about how certain systems are working. If you then change that paradigm, then you invalidate all of those assumptions, which then leads to bugs. It's not so much a matter of having to reprogram everything, it's a matter of having to check everything to see if any such assumptions have been made and correct them.

Quote
Quote
StarEngine does not, for the purposes of this, count as a real engine
Why, if I may ask?

Because CIG is making very stupid decisions based around Chris Roberts' ego rather than any technical necessity. Elite Dangerous proves that you can make a game with literally universal scale with much less effort if the people implementing it are free to make technical decisions that are best for performance and maintainability rather than one man's desire to stick to silly talking points.
Title: Re: Shaking Skyboxes
Post by: Zipper730 on September 26, 2019, 04:56:18 am
Because CIG is making very stupid decisions based around Chris Roberts' ego rather than any technical necessity.
Can you explain in more detail?
Title: Re: Shaking Skyboxes
Post by: The E on September 26, 2019, 08:41:01 am
Let's take one of croberts early mandates as an example. In the early presentations, Roberts was very clear that there would be no cheating the physics in SC. If a ship wants to move, it needs actual thrusters doing actual work to do so. While this sounds interesting, it's a giant headache in reality, both in terms of implementation and game balance.
In implementation terms, doing a physically accurate simulation of forces acting on a rigid body is possible, but complicated to set up and maintain, which is why actual games use more simplified models that are accurate enough for game use.
Another early mandate was that there be no instancing ever, every player was supposed to be present in a single global game instance. This, again, sounds neat in theory but is incredibly complex to set up; that the cryengine netcode this was all based on was built around simple multi-player matches doesn't help matters.

Basically, CIG have a history of announcing features that sound cool but (to people who know what is and isn't feasible in a real time game) are easily recognizable as pipe dreams.
Compare and contrast this with Frontier and Elite Dangerous: that game has delivered a large chunk of what CIG wants to do, on a much smaller budget and in less time, and is successful enough to continue to operate.
Title: Re: Shaking Skyboxes
Post by: Nightmare on September 26, 2019, 10:09:47 am
Any thoughts on this?

At appears that the skybox issue could be fixed though. When I set a mission in subspace (which- to my limited understanding - acts basically like a skybox as well) I'm not having any problems even when I put my ship 500km off the mission center.
Title: Re: Shaking Skyboxes
Post by: Zipper730 on September 27, 2019, 11:40:35 pm
Unreal has the ship/camera as the center of the universe with things only rendered in area of highest percentage? 
What about Unity?
Title: Re: Shaking Skyboxes
Post by: The E on September 28, 2019, 01:24:28 am
Unreal has the ship/camera as the center of the universe with things only rendered in area of highest percentage? 
What about Unity?


That's the sort of question you should find the answer to by yourself by reading the relevant documentation.
Title: Re: Shaking Skyboxes
Post by: Zipper730 on September 28, 2019, 11:31:39 am
That's the sort of question you should find the answer to by yourself by reading the relevant documentation.
I've already started sorting all that stuff out...
Title: Re: Shaking Skyboxes
Post by: Zipper730 on September 28, 2019, 09:47:59 pm
Seems that it be yes
Title: Re: Shaking Skyboxes
Post by: chief1983 on October 01, 2019, 09:49:58 am
Seeing how much breaks from switching to stationary player/moving universe could actually be a fun way to start.  It's always possible that everything is abstracted well enough that it just works with making the updates to the core calculations.  Only things that _change_ an object's position relative to whatever 0,0 is should need to be updated.  Once positions are determined each frame, nothing else should be any different, I can't think of any assumptions after that point that would matter.

So you start at 0,0 (2D for simplicity's sake) and everyone else starts at some x,y.  Let's say you accelerate along the y-axis and another ship was moving along the x-axis.  You would have moved 5 in the course of a frame along that axis, and the other ship moved 10 (arbitrary units).  Right now you have a calc that says ok, your new position is 0,0+5.  The ship at x,y at the start of the frame is now at x+10, y.  Now, you would instead keep your location as the 0,0 reference.  So the other ship (to maintain the same relative position to you as before) would not only move the distance along the x vector it was before, it would move along the y vector in the opposite direction, or x+10, y-5.  If jumping a large distance in mission that should be accomplishable the same way, whatever vector would have been applied to the player to jump far away, apply that to all the other objects instead, accomplishing the same goal.

TL;DR:  Formula should just be to add the opposite player movement vector to all objects in mission and leave the player at 0,0.

Ok, I do see one gotcha.  Any code that references absolute coordinates after the mission has started would somehow need to be updated every frame or something, or a vector containing how the player has moved since the mission began would have to be maintained so that any time absolute coordinates are used, that vector would applied, or probably many other solutions that I'm not aware of for a problem like that.  But they could probably all be updated every frame just like all object positions are updated, although updating many things that weren't changing before might have a performance impact.
Title: Re: Shaking Skyboxes
Post by: Nightmare on October 01, 2019, 10:01:49 am
As just for Skyboxes
At appears that the skybox issue could be fixed though. When I set a mission in subspace (which- to my limited understanding - acts basically like a skybox as well) I'm not having any problems even when I put my ship 500km off the mission center.