Hard Light Productions Forums
Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: Swifty on April 16, 2015, 03:32:58 am
-
Introduction
Since about 2008, Freespace Open was given the capability for programmable shaders. With programmable shaders came per-pixel lighting which, for the most part, emulated the behavior of OpenGL’s adhoc lighting model. OpenGL’s lighting model has served us quite well and moving from the per-vertex fixed function lighting to per-pixel has breathed a lot of life into the engine. But as GPUs are getting faster and home consoles are getting more powerful, it’s time we looked into replacing this lighting model to take advantage of the unused GPU power our users are increasingly obtaining. Physically plausible lighting seems to be the trend this console generation and game engines are shipping with physically-based renderers.
In an effort to not fall behind modern game engines, I am proud to introduce physically-based rendering in Freespace Open. After finishing off most of the principal development of soft shadows and deferred lighting (which should hopefully be hitting trunk not too far into the future), I’ve transitioned most of my efforts into upgrading Freespace Open’s lighting model. This started by first introducing high-dynamic range rendering into the engine. Once that was ready, I went forth and crafted much more realistic lighting shaders that take advantage of the increased range of color luminance values.
Builds with physically-based rendering no longer use the legacy adhoc lighting model which was plagued with arbitrary terms and constants. Now, the engine is using a much more accurate lighting function based off of the Blinn-Phong lighting model and Shlick’s Approximation for Fresnel reflections. It’s not perfectly accurate and will likely require tweaking and additional changes in the future but I’d say it’s a pretty good start
This post will aim to introduce developers targeting assets for Freespace Open on how to best take advantage of the more realistic lighting in the rendering engine.
New Lighting Parameter: Gloss
A lot of modders in this community are familiar with specular power. Specular power is a parameter for lighting that determines the size of specular highlights. This parameter originally could only be set game-wide through a command line parameter called -ogl_spec. Now, gloss can be set per-pixel via texture mapping.
Though specular power was configurable, the behavior of the resulting highlight did not match that of reality. The overall brightness from the of the highlights lessened as the the highlight got smaller which doesn’t respect energy conservation for reflected light. Our new lighting model, however, accounts for this. In addition to specular power, gloss also controls the the size of reflections from environment maps via MIP map selection which is a very important visual cue for metallics and smooth objects.
(http://i.imgur.com/AlSdfUM.gif)
(Gloss values of 0.0, 0.25, 0.5, 0.75 and 1.0)
Gloss has the subsequent effect of making surfaces look very smooth and shiny with high values. Subsequently, low gloss values make surfaces look rough and dull. Artist are encouraged to put a lot of detail into gloss textures. Much of the density of detail normally put into diffuse maps should instead be put into the gloss maps instead.
Gloss is stored in the alpha channel of Freespace’s Reflectance maps which will be explained in the next section.
Reflectance Maps
Reflectance maps are Freespace’s new specular texture format. Reflectance maps use the RGB channels for specular color while the alpha channel is used for per-pixel gloss. The RGB color channels are gamma corrected from the sRGB response curve into linear color space. Reflectance maps are defined by appending "-reflect" to the filenames of texture images.
Because the engine now uses a physically-based lighting model, specular color in reflectance maps should correspond to real-world specular reflectance values. When used in combination with diffuse maps, a wide range of real-life materials can be replicated.
Dielectrics (non-metal materials like rubber, plastic, cloth, stone, wood, etc), should have traditional diffuse maps while given very low specular monochrome color values in the reflectance map.
(http://i.imgur.com/jJhJGPg.png)
(A plastick-y looking Loki with a glossy finish thanks to a colored diffuse, low monochrome specular, and high gloss)
Metallic materials should be given a black or very dark diffuse map while given high specular color values; this is to simulate the fact that metals do not absorb or refract light but rather reflect all light. Metallics also change the color of their reflected light which means their specular colors do not have to be monochrome.
(http://i.imgur.com/BfxbZro.png)
(Gold Ulysses created by having a black diffuse and specular reflectance colors for gold)
There are a variety of reference values available provided by the vendors of various commercial engines.
CryEngine (http://i.imgur.com/vHaJSQf.png) (source (http://docs.cryengine.com/display/SDKDOC4/Physically+Based+Rendering))
Unity 5 (http://i.imgur.com/4e0Ik5u.png) (source (http://blogs.unity3d.com/2015/02/18/working-with-physically-based-shading-a-practical-approach/))
Remember Me (http://i.imgur.com/rT0YcEG.png) (source (https://seblagarde.wordpress.com/2012/04/30/dontnod-specular-and-glossiness-chart/))
Deprecation Model for Shine Maps
Legacy specular maps, known as shine maps, are still accepted by appending “-shine” to texture maps. Legacy specular maps do not support gloss channels as the alpha channel is specified to attenuate environment map reflections. The RGB channels are not gamma corrected. A constant gloss factor will be applied to the corresponding model instead of per-pixel gloss.
Conclusion
This isn’t exactly the best guide on physically-based workflows but hopefully it’ll be enough for artists to get started. So tl;dr, things to remember:
- Gloss is very important. Put a lot of detail into this.
- Low gloss = blurry reflection. High gloss = sharp reflection.
- Low gloss = wide, dull highlight. High gloss = small, bright highlight.
- Reflectance maps should be used to take advantage of physically-based lighting.
- Gloss lives in the alpha channel of reflectance maps.
- Specular color lives in the RGB channels of reflectance maps and are gamma corrected from sRGB.
- Reflectance maps are defined with “-reflect’.
- “-shine” still works but you can’t apply per-pixel gloss with them and the RGB values will not be gamma corrected. If both -shine and -reflect are found, -reflect will be used instead.
- When authoring a metallic texture map, apply black diffuse and high specular color. Specular does not have to be monochrome.
- When authoring a non-metallic texture map, apply a colored diffuse and low specular color. Specular must be monochrome.
-
Builds incorporating this new behaviour, as well as a quick conversion for the Ulysses, can be found here: http://www.hard-light.net/forums/index.php?topic=89550
-
This is just swell, I shall do my happy dance. :wakka:
-
Since gifs are kinda poor for showing this off:
-
WOOOOooooooOOOOOOOOOOOO______OOOOOOOOoooo
-
Since gifs are kinda poor for showing this off:
Around 1:08 I think there's adjustment to the instantly-more-pretty slider. It goes from plain to insta-pretty.
-
Around 1:08 I think there's adjustment to the instantly-more-pretty slider. It goes from plain to insta-pretty.
That's the gloss override slider. Sliding it all the way to the right basically turns every surface into a mirror.
-
I've praised Swifty a lot on irc, but I feel it is important that everyone sees my all important praise to Swifty in public.
Swifty, you ****ing amazing wizard.
-
o.O Shiny :D
-
What would you recommend as good parameters for colored glass? Do you think dark black diffuse + alpha with gold specular would work for the Pegasus/Ares/Erinyes to simulate (https://www.hq.nasa.gov/alsj/a11/AldrinLEVA1.jpg) instead of (https://www.thedailybrick.co.uk/media/catalog/product/cache/1/image/700x700/9df78eab33525d08d6e5fb8d27136e95/l/e/lego_brick_1_x_1__3005___30071__lego-transparent-orange-brick-1-x-1-30071-30-656416-105.png?)
(Same goes for Artemis cockpit glass)
And of course. Holy jeez this is wizardry. I guess the engine now supports materials. (Sorta, no subsurf shader, but that's only something mods could make use of)
-
The first picture is actually silvered glass, not just colored glass; treated to be more reflective.
Are the cockpit canopies supposed to be like that?
-
They're not 'supposed' to be like anything, they're supposed to look good.
-
If anything, those canopies are supposed to be like the ones on the F-22 Raptor or F117 Nighthawk.
-
i.e. the same stuff as on the astronaut's helmet.
-
SR-71 Helmet (windscreen, shown after, is clear):
(http://upload.wikimedia.org/wikipedia/commons/2/24/Brian_Shul_in_the_cockpit_of_the_SR-71_Blackbird.jpg)
(http://upload.wikimedia.org/wikipedia/commons/f/f1/SR-71_Pilots_cockpit,_canopy,_windscreen_ejection_seat_(4527969004).jpg)
F-22:
(https://grahampilot.files.wordpress.com/2011/04/f-22_1.jpg)
(http://upload.wikimedia.org/wikipedia/commons/6/6b/Defense.gov_News_Photo_110331-F-KE851-954_-_An_F-22_Raptor_pilot_lines_up_his_aircraft_to_be_refueled_by_a_KC-135_Stratotanker_on_March_31,_2011.jpg)
(https://virquodmachina.files.wordpress.com/2010/01/f-22-raptor-amber-cockpit1.jpg)
(http://upload.wikimedia.org/wikipedia/commons/c/cd/F-22_cockpit_close-up.jpg)
(https://c2.staticflickr.com/4/3535/4576613556_239c8c0f43_z.jpg)
-
The gold coating visible in the third-last picture is just that: a very thin layer of gold. If you're trying to recreate it in the engine, treat it as such.
-
Are you going to need to bake AO into the diffuse or is the engine going to handle that?
-
Are you going to need to bake AO into the diffuse or is the engine going to handle that?
As I understand it, baked AO is going to become its own texture so you should not bake it into the diffuse.
-
Bake it into your diffuse maps for the time being if you can. The problem is with metallic materials as they usually have black or close to black diffuse maps. Then again, specular reflectance shouldn't really affect ambient diffuse anyway. I might introduce a separate map for ambient occlusion but I think I need to do a bit more research on ambient diffuse lighting.
-
SO
I decided to share some things I researched in terms of environment mapping. As the material is rougher,so gets the reflection it gives. Currently there is code in the shader which uses mip level 0 of the envcube at max gloss, and mip level 7 at 0 gloss. The gloss itself goes from iirc specpower 2 to 2048 (gloss goes from 0 to 1, and specpower = 2^(1 + 10*gloss)).
There is an old program called cubemapgen, originally developed by IIRC AMD, which allows certain operations on skyboxes. As it was abandoned, a modified version was made, which allows blurring the skybox using phong/blinn equation to precompute the way the skybox would be reflected at certain specppower of the material. I took my computation time to blur a sample saturn skybox at certain specpower levels to compare the results.
The input Skybox (http://oi61.tinypic.com/2nk2agi.jpg)
Blur at 2^25 specpower, well beyond current gloss scale, no actual difference with input (http://oi57.tinypic.com/1zlzuk4.jpg)
Blur at 2^22 specpower, well beyond current gloss scale, minimal difference with input (http://oi62.tinypic.com/f4g0lh.jpg)
Blur at 2^18 specpower, well beyond current gloss scale, stars are becoming blurry (http://oi58.tinypic.com/6hhstf.jpg)
Blur at 2^14 specpower, beyond current gloss scale, stars are too blurred to be visible (http://oi59.tinypic.com/2mz04ud.jpg)
Blur at 2^11 specpower, current max specpower, blurry (http://oi58.tinypic.com/eppzx0.jpg)
The modified cubemapgen can be found at http://code.google.com/p/cubemapgen/downloads/list (http://code.google.com/p/cubemapgen/downloads/list). It is a memory hog so don't give it skybox faces larger than 1kx1k each. It is also slow and literally takes hours.
Here is the sample skybox I used to generate the envmap http://www.mediafire.com/download/1yfj61pypwl8ca9/data.7z (http://www.mediafire.com/download/1yfj61pypwl8ca9/data.7z)
Just to give some reference.
-
Yes, I'm aware that even a specular power of 2048 will still blur a cube map. However, I doubt we need specular power greater than 2048. Gloss is only 8-bits in the reflectance map which means 2^(n*gloss + 1) will start getting a lot of quantization if n is too high.
If artists want to prefilter the the MIPs of their cube maps, they're more than welcome. I tried to make PBR easier to adopt by automatically generating the MIP maps albeit it's done using downsampling. It's of course not 100% accurate but the results seem good enough for people to at least start iterating with it.
Most game engines tend to fudge the cube map filtering anyway. Very rarely is cube map matching done with 100% accuracy and most of the discretion is left to the artists how they prefilter the cube map.
Ideally I'll look into prefiltering cube maps in engine using importance sampling (which doesn't take as long as the brute force method in cube map gen), but it's not a high priority as other things.
-
Excellent! Since I'm doing work in Unreal 4, the material conversion to FS2 should be much easier.
-
Just a slight bug I noticed -- In the target view, ships that have reflection maps will appear completely black. Ships without them will look normal.
-
Just a slight bug I noticed -- In the target view, ships that have reflection maps will appear completely black. Ships without them will look normal.
Do you mind registering this as an issue on the project's GitHub Issues tracker (https://github.com/scp-fs2open/fs2open.github.com/issues), including details of your graphics card and OS. Swifty or one of us will take a look at it reasonably soon I'm sure. Thanks for the bug report.
-
It's a known problem, actually; the target box doesn't do any lighting. It also shouldn't go on the main FSO repo's issue tracker when it's a problem with a fork's branch that hasn't had a pull request issued for merging it with master.
-
Fair enough.
-
Excuse me, Swifty. What about ambient occlusion? Do you have plans to integrate some modern form of dynamic screen-space ambient occlusion shading into the physically-based rendering pipeline? I hope this frees modelers such as myself from the burden of baking AO maps and mixing them to diffuse maps, thus saving model-making time.
Depending on what type of shaders you'll implement in the near future, ambient occlusion maps are either generated by the engine on the fly or perhaps a new texture format, which involves the appending of "-ambient" to the filenames of texture images.
-
Excuse me, Swifty. What about ambient occlusion? Do you have plans to integrate some modern form of dynamic screen-space ambient occlusion shading into the physically-based rendering pipeline? I hope this frees modelers such as myself from the burden of baking AO maps and mixing them to diffuse maps, thus saving model-making time.
Depending on what type of shaders you'll implement in the near future, ambient occlusion maps are either generated by the engine on the fly or perhaps a new texture format, which involves the appending of "-ambient" to the filenames of texture images.
There are no plans to implement screen-space AO. What's probably going to happen is that we're going to add support for ambient occlusion maps similar to the way glow and normal maps are handled.
-
There are no plans to implement screen-space AO. What's probably going to happen is that we're going to add support for ambient occlusion maps similar to the way glow and normal maps are handled.
That's cool. I will try to find ways to implement screen-space AO while you plan to add support for ambient occlusion maps similar to the way glow and normal maps are handled. By the way, I've added you as a friend on Twitter and I hope you can follow me.
-
That's cool. I will try to find ways to implement screen-space AO while you plan to add support for ambient occlusion maps similar to the way glow and normal maps are handled. By the way, I've added you as a friend on Twitter and I hope you can follow me.
Screen space AO is a bad fit for FreeSpace.
The problem is that SSAO works best when the player camera is close to a lot of geometry, this is always the case for first- or third-person shooters for example. In these cases, you have lots of geometry that interacts dynamically, in which case the gain for visual quality makes the investment in render time worth it.
In FSO, you have a bunch of mostly static models which mostly keep their distance from each other. SSAO doesn't offer much of a benefit in these cases, AO maps provided by the artists will always be of higher quality than what a shader can provide (Not to mention that an AO map lookup is orders of magnitude faster than a shader invocation).
In short: Do not waste your time implementing SSAO.
(As for Twitter, I'm sorry, but I only follow people who I know have interesting/funny things to say on there. Since your profile is protected, I was unable to check whether my Feed would be improved by adding you to it.)
-
The_E, would it be possible to integrate ambient occlusion in a manner that would allow for easy intensity tweaking in FRED?
I say this because, while I do love ambient occlusion, it's incredibly un-realistic in deep dark space, where light in any object should go from extremely lit to absolute darkness. It is far more realistic in two scenarios: nebulas and well lit skyboxes (thinking about that particular mission in AoA where you meet the Vishnans, for instance).
Now, perhaps other modders would disagree and would just love to paint their ships in their mods and campaigns with a ton of ambient occlusion to get a different aesthetic, so that's why I ask if it would be possible to have a lever to its intensity right there on FRED.
e: I'm being an idiot, am I not? The lever already exists and it's called "ambient". **** me.
-
I don't see any particular difficulty in doing that. After all, it would just mean we'd have to pass another parameter into the shader; nothing really complex about it (And the move to PBR means we have to give mission designers more granular tools to affect lighting anyway)
-
no no no no the lever already exists it's right there on FRED, I was being a dumbass
[attachment deleted by nobody]
-
Oh, right, that one. I didn't even think of how those would interact with AO, but you're completely right.
-
so, there are actually plans to support AO maps? in this case i guess it is time to stop to bake AO into the diffuse :)
-
I'm going to miss the days of showing off in-progress model textures and having people only respond by asking where the ao bake is....
-
ao bake on the thick cables is too light, make it more like retail please
-
And what about -albedo and -ao maps?
-
Albedo can be placed in the diffuse channel (I see no real difference game-wise)
We still need the AO map and an Emissive map.
However what we really could use is a Metal-Roughness/Gloss-AO-Opacity merged map. Since each of those channels are greyscale it would be more memory effective to merge all 4 into one nice little RGBA file.
-
We have already have AO maps which channel pack Ambient Occlusion and Cavity Occlusion. With respect to emissive, we either already have that with glow maps, or you're talking about a feature that would be very out of place performance-wise in FSO at the moment. We're most likely not going to move to a Metalness workflow as the Specular workflow is already established and changing it now would throw modders into even more of a tailspin.
-
We have already have AO maps which channel pack Ambient Occlusion and Cavity Occlusion. With respect to emissive, we either already have that with glow maps, or you're talking about a feature that would be very out of place performance-wise in FSO at the moment. We're most likely not going to move to a Metalness workflow as the Specular workflow is already established and changing it now would throw modders into even more of a tailspin.
Opps nevermind.. forgot about glow, btw can you overdrive glow? That would give you some bloom effects.
Good we have AO.. my checklist is getting filled up :-)
How does Specular compare to Metal?
-
Opps nevermind.. forgot about glow, btw can you overdrive glow? That would give you some bloom effects.
Good we have AO.. my checklist is getting filled up :-)
How does Specular compare to Metal?
We have HDR (http://i.imgur.com/VEdwMgR.jpg) Bloom (http://www.hard-light.net/forums/index.php?topic=25406.msg1781438#msg1781438) now. If you make the glow bright enough, it will practically blind the player at times.
Specular works just as well as Metalness. It also makes more sense in terms of currently made assets for FSO. Specular should also exist in any tool that supports metalness, since it's possibly to convert between them using special filtering techniques on the diffuse and spec/metal map.
I know you use Substance Painter, but I've been using Quixel Suite 2, which natively supports over 15 different workflows. The only metalness workflows it supports are for UE4, Unity 5, and Arnold. It supports far more Specular workflows and Producer specific GGX workflows (Disney GGX for example). The default Specular PBR has extremely similar results in Quixel Suite's native renderer 3Do and FSO, so that is my preferred workflow thus far.
-
Opps nevermind.. forgot about glow, btw can you overdrive glow? That would give you some bloom effects.
Good we have AO.. my checklist is getting filled up :-)
How does Specular compare to Metal?
We have HDR (http://i.imgur.com/VEdwMgR.jpg) Bloom (http://www.hard-light.net/forums/index.php?topic=25406.msg1781438#msg1781438) now. If you make the glow bright enough, it will practically blind the player at times.
Specular works just as well as Metalness. It also makes more sense in terms of currently made assets for FSO. Specular should also exist in any tool that supports metalness, since it's possibly to convert between them using special filtering techniques on the diffuse and spec/metal map.
I know you use Substance Painter, but I've been using Quixel Suite 2, which natively supports over 15 different workflows. The only metalness workflows it supports are for UE4, Unity 5, and Arnold. It supports far more Specular workflows and Producer specific GGX workflows (Disney GGX for example). The default Specular PBR has extremely similar results in Quixel Suite's native renderer 3Do and FSO, so that is my preferred workflow thus far.
Second pic is exactly what I was looking for. How is that done? In Unreal you just overamp the RGB values (i.e. instead of going from 0-1 you can go to 2 or higher)
-
Just make the color of your texture bright enough. We don't really have the ability to do math on textures nor do we have a UE4 style material system, so you'll have to adjust the actual texture.
-
One of the things I've always found entertaining (although admirable) on HLP:
(http://i65.tinypic.com/2rrlydg.png)
-
Hehe....
well since Star Citizen's forum has basically gone pay, I'm kinda returning to my roots LOL
Just make the color of your texture bright enough. We don't really have the ability to do math on textures nor do we have a UE4 style material system, so you'll have to adjust the actual texture.
Ok that's easy enough. I just want to make sure I'm not painting myself into a corner, although with the lack of uv channels I kinda am, but things will get figured out.