[chief1983]

It's known that the engine isn't perfectly compartmentalized yet, but short of rewriting it completely, could work be done to fix those areas where it's not, as it pertains to the graphics code?

[headdie]

I am going to show total ignorance of the technology involved here but is a stage by stage upgrade available for example

3.7.1
Upgrade render order

3.7.2
upgrade texture handling

e.t.c.

[Sushi]

You do have to wonder when conversation along these lines goes from "upgrading FS2_Open" to "coding a brand-new space sim engine."

Well, that's exactly what I'm afraid it would take. But like I said, I'd be happy to be proven wrong.

[portej05]

Have a look through modelcollide for some truly horrible coding. Allow me to elaborate:

Code: [Select]

// Textured Poly
// +0      int         id
// +4      int         size
// +8      vec3d      normal
// +20     vec3d      normal_point
// +32     int         tmp = 0
// +36     int         nverts
// +40     int         tmap_num
// +44     nverts*(model_tmap_vert) vertlist (n,u,v)
void model_collide_tmappoly(ubyte * p)

They didn't use a struct for 'p' - they've used offsets within 'p'....
And they've done it enough to probably require a fair amount of work to fix it - I think this might be worth looking at in Antipodes (it shouldn't be behaviour changing, but if a mistake is made, it might require some sharp eyes to spot errors).

[Nuke]

i was contemplating the idea of support for the physx api to handle collision detection. i ha planned to implement it in my own game engine first. but it hasnt gone anywhere.

[portej05]

Changing the physics behaviour would probably screw up the AI in some way or other

I'd really like to see a lot of the code that handles the models be cleaned up, made more robust and modular. (think supporting new formats or something like that)

BTW, where are Goober or karajorma or taylor? I'd have expected them to weigh in at this point (although, kara said he's going to be away for a while)

[Nuke]

doing the actual physics with the physx api might screw up the feel of the game, but using it for collision detection might speed things up a lot.

[Sushi]

Yeah, collision detection is the biggest framerate-sucker right now (next to rendering). It's definitely a good target for improving game speed.

[DaBrain]

Yeah, some tests revealed that the collision detection costs a lot(!) of CPU time. Considering that the engine doesn't take advantage of multiple cores, this is really annoying.

I think the collision detection is the main reason for all slow-downs in SoL.

[Fury]

To be honest, I believe it is time to give up on retail compatibility and create a branch of fs2_open which focuses on optimal code instead of maintaining backwards compatibility. That said, it is still possible to port FS1/FS2 data and material to be compatible with the new engine when it has matured to a stable point. It's just that the original as-is wouldn't work. The port wouldn't be 100% accurate original FS experience, but you'd still get the play the original campaigns in brand new engine.

And remember, the current branch wouldn't disappear anywhere. So it is not like you lose what you have now. Only that new development would be focused on the branch, instead of this nightmare of a code and inefficiency we have now.

SCP and FSU have both outdone themselves by maintaining backwards compatibility and yet providing such amazing engine that is fs2_open. But it has its limitations which cannot be removed without removing backwards compatibility. Instead of being stuck with what was coded back in the 90's, SCP should be given the chance to bring us a modern engine that plays like a dream on modern PC's.

I'm all for maintaining backwards compatibility, but when it becomes more trouble than it is worth, it's time for a change. Port the data to be compatible with new engine.

[Commander Zane]

About time someone admitted to wanting to take FreeSpace one step further.

[Rodo]

It was never an issue about the comunity not wanting to make a new engine from scrach, the issue is that it's too much work for just one or a few... so who's gonna take the heavy burden of doing all that work?

I might help but I'm not an expert coder and honestly every day that passes by I get less time to fiddle around with FS :S

[karajorma]

To be honest, I believe it is time to give up on retail compatibility and create a branch of fs2_open which focuses on optimal code instead of maintaining backwards compatibility.

UGH!

This one comes up once in a while and it's a very bad idea.

• We're basically talking about a fork in the code here. We're not talking about a branch cause at no point will this new code every be merged back into trunk. And any coder on the SCP who isn't getting chills at the idea of maintaining two increasingly disparate codebases with the skeleton crew of coders we currently have really needs to question their sanity. There is a very good chance one codebase or the other would quickly end up being discarded and forgotten about. This is exactly what happened with the HEAD vs Stable branch and the result is that all kinds of code went into HEAD and after the switch to SVN no one has any idea what most of it was or how to get it back out without a line by line review of the code. And to be honest given the reasons I'm going to go into below, I suspect it will actually be the new one that goes. That's exactly what happened with HEAD and it will happen here for much the same reason + Several new ones.
• The issue of how major the changes would is a major one. If the changes are minor then some (but not all) of my objections vanish. However, Portej05 is already talking about changing the way pofs work. That basically invalidates every single pof in the world. Not to mention making PCS II worthless for working on this branch until it is also changed. And since the changes are also going to accumulate there we might even be talking about a fork there too. At which point Kazan is going to turn up and throw rocks at you since he hates forks even more than I do.
• Okay, you've invalidated all the game data. How the hell are you going to test the engine works now? We've already seen that major bugs can creep into the codebase in between releases because people don't play the entire campaign and point them out all the time. So now we're talking about not being able to test the whole campaign works for somewhere between several months and a year or two while this process is going on. I'm getting chills thinking how badly this is going to affect stability by the time the new fork is ready for serious testing.
• Furthermore you've basically stuck the existing mods and TCs in a position where they will be using the original fork and not this one. No sane project is going to put up with the kind of disruption this would cause to their work. They'll just wait until you're done and use it then. Except when they try they're going to find a lot of their data doesn't work either. Meaning you've added several months of dev work to the mods and TCs that do use this fork.
  
• They're going to want to add code for new features too. Which is definitely going to break on the new fork quite often.
• Once you break compatibility the first time you've lost the main argument against doing it again. So the cycle starts again when some coder finds a major problem that needs a compatibility break. And we will undoubtedly find them. So either we make the same arguments against a second compatibly break I've just made or we say no and leave the broken code in. Which means real pressure to catch every bug in the first break and then means that all we've done is simply reset which compatibility issues we're stuck with, sure some of the ones will be gone but I'm not arrogant enough to assume we're not going to make some howlers of our own.
• So if this goes ahead we're not going to see any progress in FSUP or any other project which uses this engine for a very long time. Yeah you still have the old version, but either FSUP keep improving that (and making themselves more and more loathe to swap) or they keep using the new engine and don't make much progress for a very long time

So my point of view is that if we could magically break compatibility and automatically switch all the mods over to the new system, I'd say go for it. But my main objection to the idea is that we'd probably end up with an engine that was in development for a very long time with no data to use on it.

[portej05]

To put this in context:

Code: [Select]

// Flat Poly
// +0      int         id
// +4      int         size
// +8      vec3d      normal
// +20     vec3d      center
// +32     float       radius
// +36     int         nverts
// +40     byte        red
// +41     byte        green
// +42     byte        blue
// +43     byte        pad
// +44     nverts*int  vertlist
void model_collide_flatpoly(ubyte * p) {
...
mc_check_sphereline_face(nv, points, vp(p+20), fl(p+32), vp(p+8), NULL, -1, p);
...

modelcollide.cpp is a mess, and one that is not going to be fixable with the way things are currently set up (think compiler structure alignments for starters - if everything is 4 byte aligned, not so bad, but this doesn't appear to be the case everywhere - not to mention the single letter macros all over the place).

Additionally, in modelread.cpp in read_model_file, we find:

Code: [Select]

cfread(pm->submodel[n].bsp_data,1,pm->submodel[n].bsp_data_size,fp);

i.e. bsp_data is stored within the POF file itself (if this is wrong, someone correct me). Which is additional badness because the structure of bsp_data cannot under any circumstances be changed without breaking everything.

I'd love to see an updated roadmap posted  (even if it's only Kazans internal roadmap), based on what we've currently got, 3.6.11 doesn't look remotely like the roadmap currently.

[chief1983]

Yeah the current roadmap, well, sucks.  It's not even really guidelines at this point, at least not the way the numbers line up.  Coders that proposed many of those ideas are no longer working on them and they will likely never see the light of day.  However, an engine rewrite would not have to mean elimination of backwards compatibility.  If the code were better structured, support for all legacy formats could be maintained while opening up support for new ways of doing things.  D2X-XL was rewritten from scratch and it fully supports D2 _and_ D1 assets.  There's no reason FS2 couldn't achieve the same with FS2.  However, for the time being, incremental rewrites are still possible and could wring a lot out of the engine.  In the end it comes down to what the coders are motivated to do.  If people want to keep tweaking, that's what will happen.  We can't just order everyone to stop work on this engine and start rewriting a new one, but if there's motivation to that end, then it's always a possibility.

[Sushi]

Basically, if we are going to change the rendering engine, we first need to compartmentalize so that we have:

Artwork Files --> Rendering Engine --> Rest of Game

all cleanly separated and working, WITHOUT changing the rendering engine.

Then, you'd have to figure out how to replace the rendering engine without changing any of the forward or backward dependencies.

In other words, let's make the rendering engine truly modular before we even talk about replacing it. The only way it's ever going to work is if we can truly treat the rendering engine as separate from the rest of the game (which we currently can't). Otherwise, we have the nasty scenario Kara outlined...

[karajorma]

Yep. I really can't see why we need to lose retail compatibility in order to upgrade the engine anyway. The issues with retail can simply be dealt with by flagging the newer data in some way. Yes, that increases the size of the code base and makes it more work to add features, but it's also the only sensible way I can see of doing things.

[chief1983]

And if it's properly modularized, it should be easier to work with than it is now.

[Zacam]

Or, don't bother flagging the newer data, but flag internally the old data.

For missions, we already have the in-engine list so that they give the Icon. For POF's, this would be a bit tricky, and I'm sure "new" pof's can be flagged someway, maybe by changing the extension so that it can load compatibility data (and ignore additional data) if used in a situation where compatability is needed.

[chief1983]

We can't internally flag all old data.  That would include all third party assets, campaigns, models, etc.  We don't know what exists for certain, so I don't think that's very feasible.  Simply marking stuff as "v.2" in its header for all the new stuff shouldn't be that big of a deal.