Author Topic: A UV mappers plight: Normal maps (IMG warning!)  (Read 10737 times)

0 Members and 1 Guest are viewing this topic.

Offline Water

  • 210
Re: A UV mappers plight: Normal maps (IMG warning!)
That...isn't very convincing. How come this issue doesn't show up in mediavps and other models? Mediavps models have also made-in-photoshop noral maps, and yet they don't turn half black when the light catches them in a different angle. What's the difference?
I'm guessing to answer that question you'll need to find out how many were converted with PCS1-Cob and PSC2-Cob vs PCS2-Dae.

I'm going to redo the tests tomorrow with a different model as double-check.

 

Offline pecenipicek

  • Roast Chicken
  • 211
  • Powered by copious amounts of coffee and nicotine
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • PeceniPicek's own deviantart page
Re: A UV mappers plight: Normal maps (IMG warning!)
That...isn't very convincing. How come this issue doesn't show up in mediavps and other models? Mediavps models have also made-in-photoshop noral maps, and yet they don't turn half black when the light catches them in a different angle. What's the difference?
I'm guessing to answer that question you'll need to find out how many were converted with PCS1-Cob and PSC2-Cob vs PCS2-Dae.

I'm going to redo the tests tomorrow with a different model as double-check.
*points to War in Heaven's Solaris for an example of a model demonstrating that particular behavior*


In short, if poly's share UV space, for some reason, the mirrored poly's W value (UVW dontcha know?) is -1, which directly affects the "normal" of said polygon when normal mapping is involved.

Me? i'm convinced its something wrong from the fso's render-engine side, and in very rare cases, its related to the actual models and corruption in the poly/vertex normal stuff. word of advice? use newest PCS availible exclusively from Spicious's sig.
Skype: vrganjko
Ho, ho, ho, to the bottle I go
to heal my heart and drown my woe!
Rain may fall and wind may blow,
and many miles be still to go,
but under a tall tree I will lie!

The Apocalypse Project needs YOU! - recruiting info thread.

 

Offline Talon 1024

  • 29
  • How do you turn this on?
    • Mods, Games, and Stuff
Re: A UV mappers plight: Normal maps (IMG warning!)
I think the real issue here is that the normal map wasn't processed properly before it was imported/converted.  I tried to reproduce the normal map in GIMP, and it worked just fine when I viewed the model in Blender.

And then, I noticed that the original normal map was brighter than the one I made in GIMP; It had a background colour of (RGB 154, 154, 255), while mine had a background colour of (RGB 127, 127, 255)

EDIT: Well, part of the problem, anyway...  The recessions/elevations seemed to be flipped on the mirrored side of the model. :nervous:
« Last Edit: June 26, 2011, 04:40:09 pm by Talon 1024 »
To understand religion, you need to understand morality first. | WCSaga website | WCSaga Forum | 158th website | 158th forum | Project Leader: WC: Hostile Frontier | WCHF Thread at CIC | Wing Blender | Twist of Fate | Multipart turrets on angled surfaces, tutorial included. | My Google Drive stuff | To convert speeds from WC to WCS, multiply both the cruise speed and the Afterburner speed by 0.15625 (5/32)

FS2 Mods I'm waiting on: Inferno 10th Anniversary
Current Project: Contestant Android app, Learn4Life iOS app, Blender Commander (importer).
The FreeSpace Font Foundry is back in action!

 

Offline Water

  • 210
Re: A UV mappers plight: Normal maps (IMG warning!)
I think the real issue here is that the normal map wasn't processed properly before it was imported/converted.  I tried to reproduce the normal map in GIMP, and it worked just fine when I viewed the model in Blender.
Ah.. Didn't spot that at all. A new photoshop normal map worked fine. Looks like the older cob generated models have more leeway with normal map quality.

In Blender you can have a second material for the mirrored side with the Nor button clicked twice to invert it. (for viewing in Blender, not FS)
« Last Edit: June 26, 2011, 10:00:50 pm by Water »

 

Offline pecenipicek

  • Roast Chicken
  • 211
  • Powered by copious amounts of coffee and nicotine
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • PeceniPicek's own deviantart page
Re: A UV mappers plight: Normal maps (IMG warning!)
EDIT: Well, part of the problem, anyway...  The recessions/elevations seemed to be flipped on the mirrored side of the model. :nervous:
thats the core of the problem. when normal maps get flipped the other way around, the smoothing tends to also get flipped. most of the time its not too noticeable, but with some it is.
Skype: vrganjko
Ho, ho, ho, to the bottle I go
to heal my heart and drown my woe!
Rain may fall and wind may blow,
and many miles be still to go,
but under a tall tree I will lie!

The Apocalypse Project needs YOU! - recruiting info thread.

  

Offline Enioch

  • 210
  • Alternative History Word Writer
Re: A UV mappers plight: Normal maps (IMG warning!)
*points to War in Heaven's Solaris for an example of a model demonstrating that particular behavior*

What? I suppose it's possible that the indents/outsets are reversed (the normal map is 'shallow' and kinda confuses me) but the model certainly doesn't turn half black when I turn it around in the ship lab.

Quote
use newest PCS availible exclusively from Spicious's sig.

I use the Feb 19 build.

Quote
A new photoshop normal map worked fine.

What do you mean? 'Fine' as in 'not black, but reversed indents?' or 'fine' as in 'perfect'? If the latter, give me your PS settings NAO!  :p

Quote
In short, if poly's share UV space, for some reason, the mirrored poly's W value (UVW dontcha know?) is -1, which directly affects the "normal" of said polygon when normal mapping is involved.

Could a MAX user export a .pof and see if the problem persists while using the .pof exporter? I think the whole thing might have something to do with the blender .dae exporter. I think it doesn't properly generate/export bitangents.
'Violence is the last refuge of the incompetent'  -Salvor Hardin, "Foundation"

So don't take a hammer to your computer. ;-)

 

Offline The E

  • He's Ebeneezer Goode
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Re: A UV mappers plight: Normal maps (IMG warning!)
*points to War in Heaven's Solaris for an example of a model demonstrating that particular behavior*




I am not quite sure what you mean.
If I'm just aching this can't go on
I came from chasing dreams to feel alone
There must be changes, miss to feel strong
I really need lifе to touch me
--Evergrey, Where August Mourns

 

Offline Enioch

  • 210
  • Alternative History Word Writer
Re: A UV mappers plight: Normal maps (IMG warning!)
Quote
*snip

I am not quite sure what you mean.

My point exactly.

Now, please give me a step by step of how that beauty was exported and converted, and I'll call it a day.  :p
'Violence is the last refuge of the incompetent'  -Salvor Hardin, "Foundation"

So don't take a hammer to your computer. ;-)

 

Offline The E

  • He's Ebeneezer Goode
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Re: A UV mappers plight: Normal maps (IMG warning!)
Esarai can probably fill you in on that one in particular.

Personally, I haven't had any problems with mirrored geometry/UV layouts when I did the optimized Steve-O ships, my workflow there was basically to do the UV first, then mirror it using Cinema4D's mirror tool.

As for FSO rendering maybe being buggy, I can't say. The problem is, from the perspective of a shader, there is no way to tell if the geometry being rendered has "normal" or "mirrored" UV coordinates. That's just info that's getting lost during export from whatever modeller you're using.
If I'm just aching this can't go on
I came from chasing dreams to feel alone
There must be changes, miss to feel strong
I really need lifе to touch me
--Evergrey, Where August Mourns

 

Offline pecenipicek

  • Roast Chicken
  • 211
  • Powered by copious amounts of coffee and nicotine
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • PeceniPicek's own deviantart page
Re: A UV mappers plight: Normal maps (IMG warning!)
*points to War in Heaven's Solaris for an example of a model demonstrating that particular behavior*

I am not quite sure what you mean.
while doing conversion work for HWBP we got the flipped indents problem. i'm not writing off the loss of data in between conversion steps and such, but just saying.
Skype: vrganjko
Ho, ho, ho, to the bottle I go
to heal my heart and drown my woe!
Rain may fall and wind may blow,
and many miles be still to go,
but under a tall tree I will lie!

The Apocalypse Project needs YOU! - recruiting info thread.

 

Offline Water

  • 210
Re: A UV mappers plight: Normal maps (IMG warning!)

Quote
A new photoshop normal map worked fine.

What do you mean? 'Fine' as in 'not black, but reversed indents?' or 'fine' as in 'perfect'? If the latter, give me your PS settings NAO!  :p

Could a MAX user export a .pof and see if the problem persists while using the .pof exporter? I think the whole thing might have something to do with the blender .dae exporter. I think it doesn't properly generate/export bitangents.
It's fine as in perfect. Your model is ok. The normal map needs replacing.

Unfortunately I don't actually use use PS or nVidea filters, so can't provide any settings. I use xNormal for high poly baking and also use its tool section for generating normal maps from height maps. Then convert them to DXT5nm using ATI's Compressionator.

Just out of curiosity, what method did you use to create and convert the normal map?

 

Offline Enioch

  • 210
  • Alternative History Word Writer
Re: A UV mappers plight: Normal maps (IMG warning!)
Open Photoshop -> New Image (1024^2) -> Layer from Background -> Set Layer Style to: Pattern Overlay (Overlay a pattern of white tiles with black 'mortar') -> Flatten Image.

Save As -> .dds (Opens NVidia Plugin)

From this point on, I've tried two things, with similar results:

a) Compression: 8.8.8.8 (ARGB) (Uncompressed). Generates a blue-ish normal map. Copy green channel to alpha channel and red channel to green channel. No go (I've tried leaving the blue and red channels as-is, or deleting them(full white) after that, something that generates a pink-ish normal map, similar to the ones in esarai's Emperor. No go, again).

b) Compression: DXT5-nm (Compression: DTX5). Generates a grey-ish normal map. I use that as-is. No go.

@ Water, and PLEASE answer me this: Does the normal map generated by xNormal (either by hi-poly baking or by heightmap conversion) differ in any way (concerning the color information) in comparison to the Photoshop-generated one? They also have four channels, of which FSO uses two (green and Alpha). How  is it that a baked normal map contains 'better' (more) information? It does not make sense!

P.S. please post the normal map you generated. I'd like a look.
'Violence is the last refuge of the incompetent'  -Salvor Hardin, "Foundation"

So don't take a hammer to your computer. ;-)

 

Offline The E

  • He's Ebeneezer Goode
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Re: A UV mappers plight: Normal maps (IMG warning!)
Ummm.

Please read HerraTohtori's GIMP tutorial. It has the correct info on how to generate normal maps. Pinkish ones are not guaranteed to work correctly.
If I'm just aching this can't go on
I came from chasing dreams to feel alone
There must be changes, miss to feel strong
I really need lifе to touch me
--Evergrey, Where August Mourns

 

Offline Enioch

  • 210
  • Alternative History Word Writer
Re: A UV mappers plight: Normal maps (IMG warning!)
Sigh

I read Herra's guide before even trying to make normal maps. I have been following instructions. The only reason I went for light pink maps is because the other kind didn't work. And the light pink maps don't work either, btw.

Please, read what I've been doing. Then read what Herra's telling me to do. Then tell me what I'm doing wrong, if you please. I really can't see it.

Oh, and I do realize that Herra says 'Copy Red to Alpha and blacken Red and Blue', while I've been Copying Green to Alpha, Red to Green and blackening (OK, whitening) Red and Blue. Re-did it. No noticeable difference.

'Violence is the last refuge of the incompetent'  -Salvor Hardin, "Foundation"

So don't take a hammer to your computer. ;-)

 

Offline Herra Tohtori

  • The Academic
  • 211
  • Bad command or file name
Re: A UV mappers plight: Normal maps (IMG warning!)
On a standard RGB tangent space normal map, red channel contains the vertical details, green channel contains horizontal details, and blue channel is technically used for Z coordinates but FS2_Open's normal mapping implementation ignores that entirely, and tangent space normal mapping in general does not really use it as far as I know.


FSO's shader implementation is built around DXT5nm style normal maps, as it was (at the time) the best alternative concerning quality and compression. That means the shaders read the alpha and green channels from a map with -normal tag.

This also means that you really need to move RED channel specifically to Alpha. If you switch the channels, the shaders start looking for vertical normals info from a channel that contains horizontal normals, and vice versa.

It is a matter of preference whether you then make the red and blue channels black, or copy the green channel contens to them, but you must pick one to achieve ideal compression results with DXT5.

I'm sure there are converters that can import a bluish-magenta normal map, allow you to simply select -DXT5nm option, and out comes a normal map that works correctly on FSO, but then you really need to know that your converter does it right.

Personally, I prefer to manually adjust the channels, save as PNG or TGA, then simply convert that as DXT5. It removes all doubt of the final outcome, and also allows you to use uncompressed (u8888 or TGA) formats if you so choose.

Using white as the flood colour for red and blue channels is sort of nonstandard solution. Technically, if you do the channel management correctly (red to alpha, green stays on place), then FSO would read this type of map correctly.

If you use u8888 or TGA files which are lossless formats, this will yield identical results to gray+alpha or green+alpha style normal maps.

However, I am unsure how that would affect the DXT5 compression algorithms If I recall right, the algorithm does some sort of comparisons between channels and thus there is some crossover - the contents of red and blue channels tend to affect the contents of green channel on the compressed image. There are two known ways to bypass this issue:

When red and blue are black, the difference between them and green is the same as the green channel's content.

When red and blue contain copy of green channel, difference is zero.

Both of these options result in ideal compression quality that does not cause any crossover interference between RGB channels.

However, when red and blue channels contain white, the difference between channels is sort of inverse of the green channel's content and I really don't know how this would affect the compression algorithm. And, well, better safe than sorry - staying with blackened red/blue channels or copies of green channel is a known solution for achieving the best quality.

Also, the pink maps are really ugly.


By the way, I don't know what you are trying to achieve by testing "different" types of normal maps.

The specification for what sort of normal maps FSO uses are fairly simple, and deviating from them to "fix" a model/mapping error by "tweaking" the map (ie. making it nonstandard), it's the same as coding broken web pages so that they can be viewed with a broken browser.

If there is a problem with model geometry/UV mapping causing mirroring issues, then I have no idea how to help and will gladly leave it for the more experienced model mappers and converters to handle at their convenience. But I can tell you that "fixing" such a problem in the map by reverse-breaking it is not a good solution.


The standard DXT5nm normal map with vertical normals on Alpha and horizontal normals on Green channel should work. If it doesn't work, something is wrong somewhere else; looking at the map beyond correct channel positioning will not yield any helpful results.
« Last Edit: June 27, 2011, 09:43:39 am by Herra Tohtori »
There are three things that last forever: Abort, Retry, Fail - and the greatest of these is Fail.

 

Offline Enioch

  • 210
  • Alternative History Word Writer
Re: A UV mappers plight: Normal maps (IMG warning!)
THANK YOU HERRA!  :D

Quote
It is a matter of preference whether you then make the red and blue channels black, or copy the green channel contents to them, but you must pick one to achieve ideal compression results with DXT5.

It actually isn't a matter of preference. This is the one thing I hadn't tried.  :banghead:

It works (sob). Perfectly. No normal inversion. No seams. No blackening.

Write this fact somewhere. With big bold letters. Red letters.

Many thanks to all of you, guys.
'Violence is the last refuge of the incompetent'  -Salvor Hardin, "Foundation"

So don't take a hammer to your computer. ;-)

 

Offline chief1983

  • Still lacks a custom title
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: A UV mappers plight: Normal maps (IMG warning!)
Wait, so which one did you do?  Make the channels black, or copy green to them?  Or you had done neither yet?
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 

Offline Herra Tohtori

  • The Academic
  • 211
  • Bad command or file name
Re: A UV mappers plight: Normal maps (IMG warning!)
THANK YOU HERRA!  :D

Quote
It is a matter of preference whether you then make the red and blue channels black, or copy the green channel contents to them, but you must pick one to achieve ideal compression results with DXT5.

It actually isn't a matter of preference. This is the one thing I hadn't tried.  :banghead:

It works (sob). Perfectly. No normal inversion. No seams. No blackening.

Write this fact somewhere. With big bold letters. Red letters.

Many thanks to all of you, guys.


Contents of red and blue channels don't have any relevance on how the final map itself performs on FSO shader solution. Only green and alpha channel are read for normals information.

Red and blue channel content is only important for achieving ideal DXT5 compression results.

The fact that you seem to be having different results with both makes me wonder what exactly you have ended up with. What compression utilities are you using, and are you in any phase using dxt5nm option for saving the file?

Could you post a normal map version that seems to be working?
There are three things that last forever: Abort, Retry, Fail - and the greatest of these is Fail.

 

Offline Enioch

  • 210
  • Alternative History Word Writer
Re: A UV mappers plight: Normal maps (IMG warning!)
I don't compress, but use a 8.8.8.8 ARGB .dds format. No DXT5(-nm or otherwise) at all.

Map 1 works for me. It was made by converting a heightmap to a blue-ish tangent map (8.8.8.8, ARGB, uncompressed), copying the Red channel to Alpha and the Green to Red and Blue.

Map 2 doesn't. It was made by straight conversion of the heightmap to a DXT5-nm tangent normal map.

Map 3 doesn't. It was made by converting a heightmap to a blue-ish tangent map (8.8.8.8, ARGB, uncompressed), copying the Red channel to Alpha and filling Red and Blue with black. Note that the 'blackening' is greatly reduced, but present nonetheless. Note: for some reason, when I try to convert this file to .dds (8.8.8.8, ARGB, uncompressed), what I get is a perfect Alpha channel, two black Red and Blue channels and a bright White green channel i.e. the NVidia plugin turns the Green channel all white. I have no idea why, but here's the TGA version).

Edit: Scratch that. I managed to convert Map 3 to .dds (8.8.8.8). Seems to work as well. Weird, considering I'd tried this before. Map 2 still doesn't work

[attachment deleted by ninja]
« Last Edit: June 27, 2011, 11:43:59 am by Enioch »
'Violence is the last refuge of the incompetent'  -Salvor Hardin, "Foundation"

So don't take a hammer to your computer. ;-)

 

Offline Herra Tohtori

  • The Academic
  • 211
  • Bad command or file name
Re: A UV mappers plight: Normal maps (IMG warning!)
I don't compress, but use a 8.8.8.8 ARGB .dds format. No DXT5(-nm or otherwise) at all.

Map 1 works for me. It was made by converting a heightmap to a blue-ish tangent map (8.8.8.8, ARGB, uncompressed), copying the Red channel to Alpha and the Green to Red and Blue.

This map is ok.

Quote
Map 2 doesn't. It was made by straight conversion of the heightmap to a DXT5-nm tangent normal map.

You shouldn't rely on any software to convert a height map directly into a finalized DDS normal map. Better convert height map into normal map, and normal map into DDS. Don't know what exact utility you're using (Photoshop with DDS plugin?) but regardless - bundling the height->normal conversion in the DDS export phase is just asking for trouble as you don't see the normal map itself at all and thus don't know what to expect.

In this case, individual channels are very weak and the normal map is, to put it lightly, weirdly converted, as there are both horizontal and vertical lines in both relevant layers (albeit horizontal ones are weaker in one, and stronger in other and vice versa, but point still stands that there shouldn't be any horizontal lines in the vertical normals channel).

In addition to that, the green and alpha channels both are "centered" at 60% grey, which means all the normals that should be flat in the map will be deflected 10% of the maximum possible normal angle (0% or black is one extreme, 50% gray is the "straight up" normal direction of flat areas, and 100% white is the other extreme angle).

This is probably what causes "blackening" and the seam issues you have been having...

Quote
Map 3 doesn't. It was made by converting a heightmap to a blue-ish tangent map (8.8.8.8, ARGB, uncompressed), copying the Red channel to Alpha and filling Red and Blue with black. Note that the 'blackening' is greatly reduced, but present nonetheless. Note: for some reason, when I try to convert this file to .dds (8.8.8.8, ARGB, uncompressed), what I get is a perfect Alpha channel, two black Red and Blue channels and a bright White green channel i.e. the NVidia plugin turns the Green channel all white. I have no idea why, but here's the TGA version).

Edit: Scratch that. I managed to convert Map 3 to .dds (8.8.8.8). Seems to work as well. Weird, considering I'd tried this before. Map 2 still doesn't work

Yep, can't see any reason why Map 3 wouldn't work. In fact I think it should work as TGA, if you don't have any overriding DDS file in the directory, but correctly converted u8888 version of that should work just fine.

No DDS conversion utility should magically alter the image content, so you might want to take a look at your NVidia plugin settings or change your converter. Green channel turning all white would be a perfect explanation for weird stuff happening, as it would throw the horizontal normal details tilted to one extreme angle (and possible cause a global black spec problem) on the whole texture.

Personally, I have had good results using nvDXT command line utility, and it is especially handy for batch conversion. It doesn't even really disrupt workflow if you keep a console window open at your working directory, you just save as PNG or TGA and then convert it to DDS.
There are three things that last forever: Abort, Retry, Fail - and the greatest of these is Fail.