[portej05]

Dear all coders (potential and on team), modders and artists,

Looking ahead to future versions of FreeSpace Open, it has been identified that the rendering engine is in need of a significant overhaul.
There was recently a post on SCP coding internal by DaBrain which has basically set in motion a lot of thinking about the rendering engine and where we'd like to go with it in the future. It has also highlighted a number of deficiencies and inefficiencies with the current engine.
To this end, we're (members of the SCP Coding Team) putting together a series of consultations and design 'meetings' with artists, coders and modders to develop a project team to work on a new design.
There have been a number of efforts to redo the graphics engine, including the conversion to OpenGL and a large number of performance improvements and feature enhancements. We've also had a number of recent improvements in the performance of the engine. However the FSO rendering architecture is far from modern and fairly difficult to work with (and is blamed for scaring away a number of potential coders). It has also been described by Vasudan Admiral as ``it looks something like what you'd get if an asteroid made of spaghetti hit a shoelace factory and the firemen all used sillystring to put out the blaze''.

We believe that a project like this could drive interest in the FreeSpace Open engine and the FreeSpace game and associated mods. We also believe that allowing early input into the design of a major engine component will encourage a number of coders out of the woodwork, and may even give previous coders some incentive to get involved again.

We're also under no illusion as to how massive a project this could become. It may drain significant resources from other areas of the engine, however we'd like to make it's impact as little as possible for as long as possible. There is also a good chance that this effort may not succeed, however it will be driven hard for as long as we can. It will be done separately to the current SCP codebase for reasons of licencing and ease of development, and will be incorporated in the future.

So what we're looking for at the moment is basically a show of hands of who is interested in participating in this kind of project. We'd also like to know if you have friends who have particular expertise in graphics programming or 'technical art'. We'd also like to know if you have these skills.

This project has the support of the SCP Coding team.


portej05
(on behalf of the SCP Coding team)

[Dragon]

I'm very interested in it, though I can't do much with the code itself, count on me with testing the new, experimental builds.

[Raven2001]

Unfortunately, I have no acquaintances with any kind of programmer\coder

Define "technical art". Also, what would be needed from the artists?
From my part, I am proficient in modelling, digital sculpting, animation and rigging basics, texturing, drawing\painting and design in general. I'm an adept at game design as well, so I have some knowledge of how things work in the gaming industry in terms of development, although I have no clue about the programming part.

Finally, I'd like to give my input on all this (I assume you also posted in order to know what people think of this).
I believe too much emphasis is being placed on the graphical engine. True, it was needed before, and it sure added a lot to the game. However, I think that at the state the game is now (visually competitive with professional products, bar 1 or 2 features), more emphasis should be given to other areas: gameplay, physics, interface, etc.
You guys did all the wonders we have available in terms of Freespace modding, and all for free, with passion. You could expand you area of activity\interest, in order to not only upgrade Freespace like you have so far, but also make an excellent, flexible space sim engine. An application that anyone can easily use to make their own space game, be it one of the likes of Freespace, or the likes of Privateer\Freelancer, or even I-War.

Personally I'd value that far more than dedicating resources to graphical updates again.

And of course, there's always the matter that too many resources are dedicated to this overhaul, leaving the other areas stagnant, but you guys seem to already be conscious of that


Ultimately, any way you decide, let me know where I can help!

[The E]

In my opinion, since the graphics engine is the one part of the engine that is in greatest need of an overhaul, it should be done as soon as possible. Once that is under way, the coders can start redesigning other parts of the engine as well.

[Dragon]

Does it means 3.7?

[Iss Mneur]

Does it means 3.7?

I think it would warrant a 4.0 myself.

I believe too much emphasis is being placed on the graphical engine. True, it was needed before, and it sure added a lot to the game. However, I think that at the state the game is now (visually competitive with professional products, bar 1 or 2 features), more emphasis should be given to other areas: gameplay, physics, interface, etc.
You guys did all the wonders we have available in terms of Freespace modding, and all for free, with passion. You could expand you area of activity\interest, in order to not only upgrade Freespace like you have so far, but also make an excellent, flexible space sim engine. An application that anyone can easily use to make their own space game, be it one of the likes of Freespace, or the likes of Privateer\Freelancer, or even I-War.

Personally I'd value that far more than dedicating resources to graphical updates again.

And of course, there's always the matter that too many resources are dedicated to this overhaul, leaving the other areas stagnant, but you guys seem to already be conscious of that

What is not clear in portej05's post, and as The_E mentioned, is that after this overhaul is done, the game itself will not really look better than it does now from the outside.  That is, this rewrite is more about preserving the coders sanity (and possibly scaring less coders away because of the mess) and less about making the engine more pretty, though I am sure there will be some improvements in the looks of the graphics as well.

[Bobboau]

you should consider using Ogre, it would forever end all discussion about DirectX Vs OGL (not that much is happening right now). the big advantage is the whole 'materials system' issue would be solved as well because that's how Ogre works from the getgo.

[headdie]

I for one think that sorting the graphics code out would be for the better.  I am assuming though initially there will be little difference in visual quality it would work more efficiently and improvements would be less complicated (thus faster) to implement.

if all that is the case then taking time out now make improvements come through faster for us and easier for the SCP ppl gets my vote.

[karajorma]

you should consider using Ogre, it would forever end all discussion about DirectX Vs OGL (not that much is happening right now). the big advantage is the whole 'materials system' issue would be solved as well because that's how Ogre works from the getgo.

That would have the effect of completely removing responsibility for maintaining the rendering engine from the SCP. Definitely a plus.

In my opinion, since the graphics engine is the one part of the engine that is in greatest need of an overhaul

It's not. Collision detection is.

That code has really needed an update since day one.

[Solatar]

I'd be willing to test buggy code on my machine, but that's all I can offer.

History majors aren't very good at coding.

[FreeSpaceFreak]

So what is needed besides coders? Can one help without being able to code?

[ChronoReverse]

you should consider using Ogre

This brings up a good point.  If the current (hodgepodge) graphics engine is to be scrapped (for the sake of argument) and replaced with something completely new, why not start with a pre-existing one?

What would be the technical, legal or philosophical reasons not to do this?

[blackhole]

While this is my area of expertise, I can't assist the SCP due to time constraints (as I have mentioned before on several occasions). Unless of course one of you guys can pay me $60000 a year I will, however, do my best to point you guys in a few right directions:

Don't use Ogre. You will regret it, I promise, and this is from personal experience. Ogre is too bloated to be of any use to this project. In general from my analysis of open source generic graphics engines, there are very few out there with the kind of stuff that freespace 2 needs. What might be worth looking into is the treasure trove of open source space simulators out there that have their own graphics engines. These can be used as jumping off points for design decisions and graphical techniques.

In general, I would strongly argue against using a pre-existing graphics engine simply because it will utterly fail at providing the functionality that Freespace 2 requires, and you will spend so much time trying to integrate it into a functional C based API and getting it to render things the way you want to you'd be better off just overhauling the thing using another engine as a working off point. Pre-existing graphics engines will not work. If you don't believe me then by all means, ignore what I say and find out the hard way.

Another thing: Scrap DirectX completely. If you already have figured out how to make OpenGL work then there is no point in supporting DirectX in a project like this. It only serves to complicate the crap out of matters, and unless you guys start using Shader Model 3.0 with geometry shaders, you probably won't run into too many issues with OpenGL. DirectX is better for games in the long run, but if you've already built an entire graphics API with OpenGL there is no point in supporting DirectX, especially since this graphics engine will have to be specific to the game.

When constructing the graphics engines, many things need to be decided early on. Are you going to use stencil shadows or shadow maps? How will the material system work? What's going to be hardcoded and what isn't going to be hardcoded? Some things just have to be hardcoded or they'll be too damn slow. How are you going to cull things? How is the engine going to handle local lighting as opposed to the gigantic sun that's omnipresent in all missions? What about multiple suns? What about integrating a terrain system? How will the physics be handled? How much are you integrating physics into the graphics engine and how compatible is it? What's the timespan of this overhaul? What areas of code need to be encapsulated in order for this to work? How will you work with beams? How will you handle postprocessing effects when they aren't screenspace effects (I.E. billboard related distortion effects that don't effect things in the foreground)? You guys need to make a list of everything that you want to support, and more precisely a list of SITUATIONS that you want to support and how you want each situation to look like. Then use this to figure out what the graphics engine needs to support and how best to implement it.

This is all from my personal experience of building a 2D graphics engine over the course of 2.5 years and counting.

[portej05]

Thanks for your responses so far!

We did have a look at Vega Strike briefly the other day, and whilst they are sporting an 'Ogre3D' badge, a glance through their code suggests that they abandoned it some time ago.
I can also attest to not wanting to use Ogre - it wasn't pretty last time I tried It's tailored more to building an engine around, not just using as a graphics front end (that was my experience, at least, and that may be entirely wrong).

@Dragon: It's not going to be done in-engine. Whilst the needs of the engine will be kept in mind, the majority of development will be done separately in order to ignore any limitations introduced by the engine. (which will need to be overcome at some point!). We'll need to get feedback at some point though!

@Raven2001: The issue with the engine is that it is incredibly 'coupled'. There is no real separation between components. This is needed in a modern engine and for coder sanity. Changing something at the moment is a nightmare because you don't know what else it changes! The collision detection code hasn't been touched because coders that venture there don't usually come back
It's not that we're focussing on the graphics engine, it's that we're focussing on removing the massive coupling to the graphics engine, which will likely result in better management of the scene.
'Technical Artist' is a term for someone who is able to do both art and coding and knows how they affect each other in a particular engine.

@karajorma: Collision detection code is.... ugly.

@Solatar: If you can read, we'll probably need you. It's amazing how often someone reading some documentation or planning says something like 'there's nowhere for the trucks to turn around' (true story from an engineering project in Australia).

@FreeSpaceFreak: We need people who use the engine (i.e. modders and coders) to tell us what they want and expect. We're planning to do this separately so that we can evaluate any existing options and/or design our own without having to consider any limitations of the engine.

@ChronoReverse: Technically, we have different requirements to MMO (or whatever you call those Age of Empires) style games, and that seems to be what Ogre is designed for. Legally, we have issues with licences because we're effectively 'no commercial use', which is incompatible with GPL and we'd need to talk to a lawyery kind of person if the software didn't allow us to do this (we have code in one case that a project developer has licenced separately to us, and we also have code that is under NDA (trackir) )

@blackhole: This project isn't going to be a coding project for a little while, it won't become a serious coding project until probably way after 3.6.12. You've added a pile of very useful questions. It's more likely to be a C++ API than a C API though. Hery did mention a desire to play with SM3.0 for particle systems.
If you're able to give us any time at all it would be great!


Thanks for your responses everyone! We'll keep you posted on what is going on (likely just organising style things for a while). Please keep an eye out for friends/family who do coding or have expertise in art or game design.

[JGZinv]

One thing I would like to bring up, in which I know not (fully) the particulars of how it goes from 0's and 1's to
lovely things on screen.... is such things as HUD moddability.

I've understood from past research around HLP and our own interests at FringeSpace, is that folks would like to have
every ship be capable of having a unique HUD. That HUD should also be capable of somewhat more complex animations
than what is possible with the ANIs. I can't articulate this very well, but think if you're comparing PCX to Flash.
You've simply got more options to play with in Flash (actionscript, et al). If this isn't partially related to what's being
discussed, please ignore me.

Furthermore, you might actually get more done by choosing the option which will solve the most feature
or bug requests at one time with this overhaul. IE. If HUDs, and shaders, and so forth are planned to be fixed in one
fell swoop, you can quit allocating people to those items - trying to hack a solution together in the current environment.
More than likely this is obvious to most.... but hey, what was said about those trucks in Australia?

Here's some guides on making yourself more productive, doing what you're doing. Perhaps in some small way it'll help.
http://lifehacker.com/5437929/top-10-mind-hacks-for-making-your-resolutions-stick
http://plpatterns.com/post/307982918/its-hard-to-change-a-little-its-much-easier-to
http://lifehacker.com/5437156/run-effective-google+style-meetings-by-focusing-on-data-not-politics
http://lifehacker.com/5435879/get-things-done-with-the-middle-way-method

[ChronoReverse]

@ChronoReverse: Technically, we have different requirements to MMO (or whatever you call those Age of Empires) style games, and that seems to be what Ogre is designed for. Legally, we have issues with licences because we're effectively 'no commercial use', which is incompatible with GPL and we'd need to talk to a lawyery kind of person if the software didn't allow us to do this (we have code in one case that a project developer has licenced separately to us, and we also have code that is under NDA (trackir) )

Oh definitely.  I didn't mean to use Ogre but rather if it might be a good idea to also look for and compile a set of existing 3D engines that might be suitable.

As for the licensing, I had no idea there was code licensed to the SCP as well as NDA code.  That's pretty neat.


In any case, the sentiment about first figuring out what the engine should do is the most prudent course of action.  In fact, even if this isn't done, such a list should be made anyway.  That is, an organized list of the things the current code can and is expected to do.  Then that could be cross-referenced with what's already used in retail, the mediavp's and the FS1 port.  Finally it should be checked against whichever mods people have vested interests in

[chief1983]

There's some enhancements to the custom hud gauges table coming down the pipe.  Should expand its flexibility quite a bit.  Just FYI.

[blackhole]

@blackhole: This project isn't going to be a coding project for a little while, it won't become a serious coding project until probably way after 3.6.12. You've added a pile of very useful questions. It's more likely to be a C++ API than a C API though. Hery did mention a desire to play with SM3.0 for particle systems.
If you're able to give us any time at all it would be great!

I sort of meant a C-influenced API since I thought you'd be working with the original codebase, which is still heavily C-inspired in its writing and has very little OOP aside from the stuff that's accumulated with random improvements over the years. If you're going to be massively rewriting everything then its a non-issue.

OpenGL is notorious for its hacked on support for SM3.0, so if you guys want to play around with that make sure its in a highly separated component from everything else to maintain stability. It's probably got better support nowadays, but it'll likely drive a couple of coders bonkers with driver incompatibilities. I'd still advise against adding support for directX because its too time consuming with little reward when most of your graphics coders know how to use openGL anyway.

The only time I can offer you is my advice, since I'm completely unfamiliar with the codebase and have my own projects to attend to.

[Reprobator]

i'm no coder at all so i won't be any help, anyway , for the end project there are things we would like to see, (i don't know for the feasability)

-a rewritten hud rendering : That's the last thing we need for a full triplehead support, if we could choose at wich resolution from the center of the screen the hud will stretch that would be cool.
-Having a full normal map support (i mean using all normal map channel, because the actual one doesn't use all channel and we loose some information)
-A improved reflexion system :
actually when we take off from an inside tube or hangar, if you look at your wingmate's cockpit glass you see the reflexion of the skybox, and it looks a bit weird, i think it should be improved if it's ever possible
-light and shadow system : having shadow would really improve the game looking.
-volumetric cloud/nebula system
-having Post process and aa etc... able to work together would be cool
I think that's already too much to ask

[headdie]

here is a random thought does this mean we will finally have a firm fix for the slight miss-aligned crosshairs that wide screen users report?