Hard Light Productions Forums

Modding, Mission Design, and Coding => The Modding Workshop => Topic started by: Enioch on June 25, 2011, 06:54:11 am

Title: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 25, 2011, 06:54:11 am
Greetings to all.

In the process of unwrapping and texturing ships for the Renegade Legion Mod, I have run into a brick wall. Not literally, of course. That would be silly. I meant it metaphorically.

I have decided to consult the Modding and Coding Deities of HLP, in the hope that I might regain my sanity.

Consider, dear readers, the following, crudely UVmapped, box-like thing:

(http://img190.imageshack.us/img190/1889/mainex.png)

It has been mirrored-X and its mirrored side UV map has been offset 1 square UV map 'area' to avoid the auto-merging PCS2 issue that leads to nasty seam effects. Areas have been marked green and red, so that the relation between 3d view and UV map view is clear.

This 'box' has been exported as .dae, imported into PCS2 without a hiccup and exported as .pof.

As a height map, I have created a light gray/black grid, that, when converted with the Photoshop NVIDIA plugin gives me a DXT5-nm file that simulates a 'checkerboard' of raised panels. See below:

(http://img585.imageshack.us/img585/7252/ex3m.png)

"There's nothing wrong with that!" you might say.

There is.

If I turn the model just a bit, this happens:

(http://img828.imageshack.us/img828/5990/ex2y.png)

And then, this:

(http://img812.imageshack.us/img812/5830/ex1v.png)

In the immortal words of Mr. Spock:

(http://img835.imageshack.us/img835/9176/spocksspeakstruth.jpg)

Of course, I'm sure it makes logical sense. Somehow. Somewhere along the route, I have screwed up royally, but I'm not sure where.

Thus, I would like your opinions on the matter.

What am I doing wrong? How can I mirror a ship while getting rid of this nasty lighting effect? Give me RULES give me a GUIDELINE give me something in the line of "This appears only if you [insert case here]", so I can tailor my UVmapping to address the problem.

For all it's worth, I have a Geforce GTX 260M, running the latest drivers and I'm running my mod on the latest Nightly, with these flags:

-spec -glow -env -mipmap -nomotiondebris -noscalevid -missile_lighting -normal -3dshockwave -post_process -img2dds -no_vsync -cache_bitmaps -orbradar -rearm_timer -ballistic_gauge -ship_choice_3d -3dwarp -warp_flash -snd_preload -output_scripting  -ambient_factor 100 -no_emissive_light -spec_exp 16.7 -spec_point 0.6 -spec_static 0.9 -spec_tube 1 -bloom_intensity 150


Thank you for your time.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: The E on June 25, 2011, 07:11:23 am
Could you post the model, textures and tables here?
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 25, 2011, 07:17:11 am
Of course.



[attachment deleted by ninja]
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Spoon on June 25, 2011, 07:55:46 am
3 possible lighting related issues
The two mirrored pieces are not welded together
Smooth is not assigned properly
Model has not been reset x-form'd  (which may have a completely different name depending on the program you're using, or may not be there at all)


edit: and then I saw the pof that got posted before I posted
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 25, 2011, 08:27:44 am
Smooth is not assigned properly

Explain please.

The model has no smooth sides. None. Everything is "Set Solid" in Blender.

Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Spoon on June 25, 2011, 08:50:30 am
I dont speak blender so I have no idea what means  :p
I only know that smooth (groups) does things with the lighting in freespace
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Angelus on June 25, 2011, 08:56:09 am
When UVing, avoid 90degree angles in one element.
Either, break ( or whatever it's called in Blender) all elements that are in 90deg to each other, in the UV editor or champfer the edges.
The latter has the downside that it increases polycount plus it make s UVing even more annoying.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 25, 2011, 09:07:43 am
I dont speak blender so I have no idea what means  :p
I only know that smooth (groups) does things with the lighting in freespace

 :p
It means that nothing is smooth. There are no smoothgroups. All surfaces are 'solid' i.e. not smooth.

When UVing, avoid 90degree angles in one element.
Either, break ( or whatever it's called in Blender) all elements that are in 90deg to each other, in the UV editor or champfer the edges.

Whut.

You do realize that the problem lies somewhere in the mirroring, since it shows up in the central 'recess' where I have two mirrored, co-planar polygons? No 90 degree corner there...
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: pecenipicek on June 25, 2011, 09:26:27 am
Enioch, you might also take a look at this joyful mess... (http://www.hard-light.net/forums/index.php?topic=74548.msg1473786#msg1473786)
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: chief1983 on June 25, 2011, 10:17:28 am
I get deja vu every time I see a thread about normal maps on mirrored polygons.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 25, 2011, 10:30:43 am
Enioch, you might also take a look at this joyful mess... (http://www.hard-light.net/forums/index.php?topic=74548.msg1473786#msg1473786)

I did. Nothing came out of this, did it? zookeeper suddently could no longer reproduce the problem. And I skimmed through the discussion, without finding an explanation.

I get deja vu every time I see a thread about normal maps on mirrored polygons.

Well, if we get a clear answer that fixes the problem, there would be no more need for such threads [/irritated and already sorry for snapping]
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: chief1983 on June 25, 2011, 06:52:51 pm
I thought there was another discussion of the issue with a solution in the forums somewhere.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Water on June 25, 2011, 10:19:01 pm
If I remember right, the linked discussion was mostly about wielded vertices affecting smoothing/normals rather than mirrored normals.

Three images
1st - cob - separate sub-object - uv's not offset. All looks good.

2nd - same again but uv's offset. Not so good.

3rd - dae - uv's offset didn't matter. Separate sub-objects didn't help.

First two were done with PCS2RC3d (Jan08) only because the latest version crashes during importation of the cob.
I also tested the cob with PCS2.1 (Jan09)

(http://i229.photobucket.com/albums/ee67/waternz/mesh/Normal-fail01.jpg)

[attachment deleted by ninja]
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: chief1983 on June 25, 2011, 10:34:48 pm
There are much more recent versions, if you have a model that crashes PCS2's most recent releases, please mention it in the FSO Tools forum.  Spicious or someone will probably take a look at it and see if there's anything wrong with PCS2's handling of it.  The Jan09 version is not the latest release by far.  Check the tools forum, specifically Spicious' signature if you can't find a link anywhere else.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Water on June 26, 2011, 12:51:56 am
Using  dae with a proper High poly > Low poly baked map.

Short version: If you want to use photoshop derived normal maps on mirrored geometry use the obsolete COB format. (Edit: wrong answer. was only true for poor quality normal map)
(http://i229.photobucket.com/albums/ee67/waternz/mesh/Normal-fail02.jpg)
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 26, 2011, 01:02:42 am
Using  dae with a proper High poly > Low poly baked map.

Short version: If you want to use photoshop derived normal maps on mirrored geometry use the obsolete COB format.

*Sigh* Oh dear. WHY? What's the difference? Why does it work with COB and not dae? Any ideas?

AND: Why a different sub-object? Is it necessary?

AND: Does COB support smoothgroups, or am I gonna get a balooney mess? I think I remember some issues on that point...

AND: What is different about baked and photoshop derived normal maps that makes such a difference?

Oh, and Water - THANKS A LOT.  :nod:
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Scooby_Doo on June 26, 2011, 01:39:54 am
Or just don't share the uvmap for the mesh sections :-)

or..... do uvmapping AFTER you've done the symmetry.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 26, 2011, 02:10:54 am
Or just don't share the uvmap for the mesh sections :-)
What mesh sections?  :blah: Don't get what you're saying. Are you saying I shouldn't share UV space on any faces? That's the point of this thread. That's what I'm trying to do. I'm asking HOW to do it properly, and the answer is 'don't do it'? :wtf:

or..... do uvmapping AFTER you've done the symmetry.

The whole point is to save UVmapping time and texture space. If I UVmap after the mirroring, I'll have to UVmap twice the faces.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Water on June 26, 2011, 02:39:32 am
Cob and sub-objects were for testing. The format had a smoothing type called Faceted, maybe Dae treats it differently.

Simple solution don't mirror the central section. Re-unwrap that section as one piece.

The main difference, photoshop normal maps don't take any account of the underlying model's normals. Using High poly > Low poly baking, this info becomes part of the normal map.
If you want to get into Hi poly baking both Blender and xNormal are good choices.

If you look at the colours you'll notice on each poly they shift from top to bottom and also left to right. That's from the models normals. The whole model was smoothed. The less colourful parts had the smoothing controlled by edge support loops. It effectively made them look like they were set solid, while actually set smooth.

(http://i229.photobucket.com/albums/ee67/waternz/mesh/Normal-fail03.jpg)
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 26, 2011, 04:10:37 am
That...isn't very convincing. How come this issue doesn't show up in mediavps and other models? Mediavps models have also made-in-photoshop noral maps, and yet they don't turn half black when the light catches them in a different angle. What's the difference?

By the way, the issue is not limited in the central section. The entire mirrored half of the model gets darkened. If the central part looks good and the 'outside' poly looks black, the model will still look weird.

Quote
Using High poly > Low poly baking, this info becomes part of the normal map.

Huh? :wtf:

It doesn't show.

The normal map you posted has shifting colors, sure, but that's because, as you said, the entire model was smoothed. Thus, the 'baked' normal map tries to mimic the 'smoothing' and the normals of the surfaces shift.

That's all fine and dandy. But the 'edge-split' faces, the ones that are smoothed yet separated, in order to 'look' non-smooth look pretty standard to me (i.e. similar to a photoshop-generated normal map). How does the normal information in that map differ from the normal information in a photoshop-generated map? If the colors are the same, the normal information in the map should be the same as well.

Thanks again for the continuing feedback.

Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Water on June 26, 2011, 08:43:39 am
That...isn't very convincing. How come this issue doesn't show up in mediavps and other models? Mediavps models have also made-in-photoshop noral maps, and yet they don't turn half black when the light catches them in a different angle. What's the difference?
I'm guessing to answer that question you'll need to find out how many were converted with PCS1-Cob and PSC2-Cob vs PCS2-Dae.

I'm going to redo the tests tomorrow with a different model as double-check.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: pecenipicek on June 26, 2011, 03:04:11 pm
That...isn't very convincing. How come this issue doesn't show up in mediavps and other models? Mediavps models have also made-in-photoshop noral maps, and yet they don't turn half black when the light catches them in a different angle. What's the difference?
I'm guessing to answer that question you'll need to find out how many were converted with PCS1-Cob and PSC2-Cob vs PCS2-Dae.

I'm going to redo the tests tomorrow with a different model as double-check.
*points to War in Heaven's Solaris for an example of a model demonstrating that particular behavior*


In short, if poly's share UV space, for some reason, the mirrored poly's W value (UVW dontcha know?) is -1, which directly affects the "normal" of said polygon when normal mapping is involved.

Me? i'm convinced its something wrong from the fso's render-engine side, and in very rare cases, its related to the actual models and corruption in the poly/vertex normal stuff. word of advice? use newest PCS availible exclusively from Spicious's sig.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Talon 1024 on June 26, 2011, 04:36:24 pm
I think the real issue here is that the normal map wasn't processed properly before it was imported/converted.  I tried to reproduce the normal map in GIMP, and it worked just fine when I viewed the model in Blender.

And then, I noticed that the original normal map was brighter than the one I made in GIMP; It had a background colour of (RGB 154, 154, 255), while mine had a background colour of (RGB 127, 127, 255)

EDIT: Well, part of the problem, anyway...  The recessions/elevations seemed to be flipped on the mirrored side of the model. :nervous:
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Water on June 26, 2011, 06:54:17 pm
I think the real issue here is that the normal map wasn't processed properly before it was imported/converted.  I tried to reproduce the normal map in GIMP, and it worked just fine when I viewed the model in Blender.
Ah.. Didn't spot that at all. A new photoshop normal map worked fine. Looks like the older cob generated models have more leeway with normal map quality.

In Blender you can have a second material for the mirrored side with the Nor button clicked twice to invert it. (for viewing in Blender, not FS)
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: pecenipicek on June 26, 2011, 08:40:26 pm
EDIT: Well, part of the problem, anyway...  The recessions/elevations seemed to be flipped on the mirrored side of the model. :nervous:
thats the core of the problem. when normal maps get flipped the other way around, the smoothing tends to also get flipped. most of the time its not too noticeable, but with some it is.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 27, 2011, 03:31:10 am
*points to War in Heaven's Solaris for an example of a model demonstrating that particular behavior*

What? I suppose it's possible that the indents/outsets are reversed (the normal map is 'shallow' and kinda confuses me) but the model certainly doesn't turn half black when I turn it around in the ship lab.

Quote
use newest PCS availible exclusively from Spicious's sig.

I use the Feb 19 build.

Quote
A new photoshop normal map worked fine.

What do you mean? 'Fine' as in 'not black, but reversed indents?' or 'fine' as in 'perfect'? If the latter, give me your PS settings NAO!  :p

Quote
In short, if poly's share UV space, for some reason, the mirrored poly's W value (UVW dontcha know?) is -1, which directly affects the "normal" of said polygon when normal mapping is involved.

Could a MAX user export a .pof and see if the problem persists while using the .pof exporter? I think the whole thing might have something to do with the blender .dae exporter. I think it doesn't properly generate/export bitangents.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: The E on June 27, 2011, 03:44:39 am
*points to War in Heaven's Solaris for an example of a model demonstrating that particular behavior*

(http://blueplanet.fsmods.net/E/pics/solarisnormal.png)
(http://blueplanet.fsmods.net/E/pics/solarisnormal2.png)

I am not quite sure what you mean.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 27, 2011, 04:13:08 am
Quote
*snip

I am not quite sure what you mean.

My point exactly.

Now, please give me a step by step of how that beauty was exported and converted, and I'll call it a day.  :p
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: The E on June 27, 2011, 04:19:38 am
Esarai can probably fill you in on that one in particular.

Personally, I haven't had any problems with mirrored geometry/UV layouts when I did the optimized Steve-O ships, my workflow there was basically to do the UV first, then mirror it using Cinema4D's mirror tool.

As for FSO rendering maybe being buggy, I can't say. The problem is, from the perspective of a shader, there is no way to tell if the geometry being rendered has "normal" or "mirrored" UV coordinates. That's just info that's getting lost during export from whatever modeller you're using.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: pecenipicek on June 27, 2011, 05:39:23 am
*points to War in Heaven's Solaris for an example of a model demonstrating that particular behavior*

I am not quite sure what you mean.
while doing conversion work for HWBP we got the flipped indents problem. i'm not writing off the loss of data in between conversion steps and such, but just saying.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Water on June 27, 2011, 05:53:48 am

Quote
A new photoshop normal map worked fine.

What do you mean? 'Fine' as in 'not black, but reversed indents?' or 'fine' as in 'perfect'? If the latter, give me your PS settings NAO!  :p

Could a MAX user export a .pof and see if the problem persists while using the .pof exporter? I think the whole thing might have something to do with the blender .dae exporter. I think it doesn't properly generate/export bitangents.
It's fine as in perfect. Your model is ok. The normal map needs replacing.

Unfortunately I don't actually use use PS or nVidea filters, so can't provide any settings. I use xNormal for high poly baking and also use its tool section for generating normal maps from height maps. Then convert them to DXT5nm using ATI's Compressionator.

Just out of curiosity, what method did you use to create and convert the normal map?
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 27, 2011, 08:03:57 am
Open Photoshop -> New Image (1024^2) -> Layer from Background -> Set Layer Style to: Pattern Overlay (Overlay a pattern of white tiles with black 'mortar') -> Flatten Image.

Save As -> .dds (Opens NVidia Plugin)

From this point on, I've tried two things, with similar results:

a) Compression: 8.8.8.8 (ARGB) (Uncompressed). Generates a blue-ish normal map. Copy green channel to alpha channel and red channel to green channel. No go (I've tried leaving the blue and red channels as-is, or deleting them(full white) after that, something that generates a pink-ish normal map, similar to the ones in esarai's Emperor. No go, again).

b) Compression: DXT5-nm (Compression: DTX5). Generates a grey-ish normal map. I use that as-is. No go.

@ Water, and PLEASE answer me this: Does the normal map generated by xNormal (either by hi-poly baking or by heightmap conversion) differ in any way (concerning the color information) in comparison to the Photoshop-generated one? They also have four channels, of which FSO uses two (green and Alpha). How  is it that a baked normal map contains 'better' (more) information? It does not make sense!

P.S. please post the normal map you generated. I'd like a look.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: The E on June 27, 2011, 08:20:35 am
Ummm.

Please read HerraTohtori's GIMP tutorial. It has the correct info on how to generate normal maps. Pinkish ones are not guaranteed to work correctly.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 27, 2011, 08:55:47 am
Sigh

I read Herra's guide before even trying to make normal maps. I have been following instructions. The only reason I went for light pink maps is because the other kind didn't work. And the light pink maps don't work either, btw.

Please, read what I've been doing. Then read what Herra's telling me to do. Then tell me what I'm doing wrong, if you please. I really can't see it.

Oh, and I do realize that Herra says 'Copy Red to Alpha and blacken Red and Blue', while I've been Copying Green to Alpha, Red to Green and blackening (OK, whitening) Red and Blue. Re-did it. No noticeable difference.

Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Herra Tohtori on June 27, 2011, 09:39:16 am
On a standard RGB tangent space normal map, red channel contains the vertical details, green channel contains horizontal details, and blue channel is technically used for Z coordinates but FS2_Open's normal mapping implementation ignores that entirely, and tangent space normal mapping in general does not really use it as far as I know.


FSO's shader implementation is built around DXT5nm style normal maps, as it was (at the time) the best alternative concerning quality and compression. That means the shaders read the alpha and green channels from a map with -normal tag.

This also means that you really need to move RED channel specifically to Alpha. If you switch the channels, the shaders start looking for vertical normals info from a channel that contains horizontal normals, and vice versa.

It is a matter of preference whether you then make the red and blue channels black, or copy the green channel contens to them, but you must pick one to achieve ideal compression results with DXT5.

I'm sure there are converters that can import a bluish-magenta normal map, allow you to simply select -DXT5nm option, and out comes a normal map that works correctly on FSO, but then you really need to know that your converter does it right.

Personally, I prefer to manually adjust the channels, save as PNG or TGA, then simply convert that as DXT5. It removes all doubt of the final outcome, and also allows you to use uncompressed (u8888 or TGA) formats if you so choose.

Using white as the flood colour for red and blue channels is sort of nonstandard solution. Technically, if you do the channel management correctly (red to alpha, green stays on place), then FSO would read this type of map correctly.

If you use u8888 or TGA files which are lossless formats, this will yield identical results to gray+alpha or green+alpha style normal maps.

However, I am unsure how that would affect the DXT5 compression algorithms If I recall right, the algorithm does some sort of comparisons between channels and thus there is some crossover - the contents of red and blue channels tend to affect the contents of green channel on the compressed image. There are two known ways to bypass this issue:

When red and blue are black, the difference between them and green is the same as the green channel's content.

When red and blue contain copy of green channel, difference is zero.

Both of these options result in ideal compression quality that does not cause any crossover interference between RGB channels.

However, when red and blue channels contain white, the difference between channels is sort of inverse of the green channel's content and I really don't know how this would affect the compression algorithm. And, well, better safe than sorry - staying with blackened red/blue channels or copies of green channel is a known solution for achieving the best quality.

Also, the pink maps are really ugly.


By the way, I don't know what you are trying to achieve by testing "different" types of normal maps.

The specification for what sort of normal maps FSO uses are fairly simple, and deviating from them to "fix" a model/mapping error by "tweaking" the map (ie. making it nonstandard), it's the same as coding broken web pages so that they can be viewed with a broken browser.

If there is a problem with model geometry/UV mapping causing mirroring issues, then I have no idea how to help and will gladly leave it for the more experienced model mappers and converters to handle at their convenience. But I can tell you that "fixing" such a problem in the map by reverse-breaking it is not a good solution.


The standard DXT5nm normal map with vertical normals on Alpha and horizontal normals on Green channel should work. If it doesn't work, something is wrong somewhere else; looking at the map beyond correct channel positioning will not yield any helpful results.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 27, 2011, 09:55:45 am
THANK YOU HERRA!  :D

Quote
It is a matter of preference whether you then make the red and blue channels black, or copy the green channel contents to them, but you must pick one to achieve ideal compression results with DXT5.

It actually isn't a matter of preference. This is the one thing I hadn't tried.  :banghead:

It works (sob). Perfectly. No normal inversion. No seams. No blackening.

Write this fact somewhere. With big bold letters. Red letters.

Many thanks to all of you, guys.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: chief1983 on June 27, 2011, 10:04:49 am
Wait, so which one did you do?  Make the channels black, or copy green to them?  Or you had done neither yet?
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Herra Tohtori on June 27, 2011, 10:19:45 am
THANK YOU HERRA!  :D

Quote
It is a matter of preference whether you then make the red and blue channels black, or copy the green channel contents to them, but you must pick one to achieve ideal compression results with DXT5.

It actually isn't a matter of preference. This is the one thing I hadn't tried.  :banghead:

It works (sob). Perfectly. No normal inversion. No seams. No blackening.

Write this fact somewhere. With big bold letters. Red letters.

Many thanks to all of you, guys.


Contents of red and blue channels don't have any relevance on how the final map itself performs on FSO shader solution. Only green and alpha channel are read for normals information.

Red and blue channel content is only important for achieving ideal DXT5 compression results.

The fact that you seem to be having different results with both makes me wonder what exactly you have ended up with. What compression utilities are you using, and are you in any phase using dxt5nm option for saving the file?

Could you post a normal map version that seems to be working?
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 27, 2011, 11:10:43 am
I don't compress, but use a 8.8.8.8 ARGB .dds format. No DXT5(-nm or otherwise) at all.

Map 1 works for me. It was made by converting a heightmap to a blue-ish tangent map (8.8.8.8, ARGB, uncompressed), copying the Red channel to Alpha and the Green to Red and Blue.

Map 2 doesn't. It was made by straight conversion of the heightmap to a DXT5-nm tangent normal map.

Map 3 doesn't. It was made by converting a heightmap to a blue-ish tangent map (8.8.8.8, ARGB, uncompressed), copying the Red channel to Alpha and filling Red and Blue with black. Note that the 'blackening' is greatly reduced, but present nonetheless. Note: for some reason, when I try to convert this file to .dds (8.8.8.8, ARGB, uncompressed), what I get is a perfect Alpha channel, two black Red and Blue channels and a bright White green channel i.e. the NVidia plugin turns the Green channel all white. I have no idea why, but here's the TGA version).

Edit: Scratch that. I managed to convert Map 3 to .dds (8.8.8.8). Seems to work as well. Weird, considering I'd tried this before. Map 2 still doesn't work

[attachment deleted by ninja]
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Herra Tohtori on June 27, 2011, 12:34:35 pm
I don't compress, but use a 8.8.8.8 ARGB .dds format. No DXT5(-nm or otherwise) at all.

Map 1 works for me. It was made by converting a heightmap to a blue-ish tangent map (8.8.8.8, ARGB, uncompressed), copying the Red channel to Alpha and the Green to Red and Blue.

This map is ok.

Quote
Map 2 doesn't. It was made by straight conversion of the heightmap to a DXT5-nm tangent normal map.

You shouldn't rely on any software to convert a height map directly into a finalized DDS normal map. Better convert height map into normal map, and normal map into DDS. Don't know what exact utility you're using (Photoshop with DDS plugin?) but regardless - bundling the height->normal conversion in the DDS export phase is just asking for trouble as you don't see the normal map itself at all and thus don't know what to expect.

In this case, individual channels are very weak and the normal map is, to put it lightly, weirdly converted, as there are both horizontal and vertical lines in both relevant layers (albeit horizontal ones are weaker in one, and stronger in other and vice versa, but point still stands that there shouldn't be any horizontal lines in the vertical normals channel).

In addition to that, the green and alpha channels both are "centered" at 60% grey, which means all the normals that should be flat in the map will be deflected 10% of the maximum possible normal angle (0% or black is one extreme, 50% gray is the "straight up" normal direction of flat areas, and 100% white is the other extreme angle).

This is probably what causes "blackening" and the seam issues you have been having...

Quote
Map 3 doesn't. It was made by converting a heightmap to a blue-ish tangent map (8.8.8.8, ARGB, uncompressed), copying the Red channel to Alpha and filling Red and Blue with black. Note that the 'blackening' is greatly reduced, but present nonetheless. Note: for some reason, when I try to convert this file to .dds (8.8.8.8, ARGB, uncompressed), what I get is a perfect Alpha channel, two black Red and Blue channels and a bright White green channel i.e. the NVidia plugin turns the Green channel all white. I have no idea why, but here's the TGA version).

Edit: Scratch that. I managed to convert Map 3 to .dds (8.8.8.8). Seems to work as well. Weird, considering I'd tried this before. Map 2 still doesn't work

Yep, can't see any reason why Map 3 wouldn't work. In fact I think it should work as TGA, if you don't have any overriding DDS file in the directory, but correctly converted u8888 version of that should work just fine.

No DDS conversion utility should magically alter the image content, so you might want to take a look at your NVidia plugin settings or change your converter. Green channel turning all white would be a perfect explanation for weird stuff happening, as it would throw the horizontal normal details tilted to one extreme angle (and possible cause a global black spec problem) on the whole texture.

Personally, I have had good results using nvDXT command line utility, and it is especially handy for batch conversion. It doesn't even really disrupt workflow if you keep a console window open at your working directory, you just save as PNG or TGA and then convert it to DDS.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 27, 2011, 12:38:30 pm
Duly noted. Thanks again.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Herra Tohtori on June 27, 2011, 12:49:34 pm
Duly noted. Thanks again.

You're welcome.

Just did a little check - sure enough, the original normal map in the attachement in original post had the same problem as Map 2 here - level grey at 60% grey instead of 50%.

That'll cause the surface to tilt in one direction, and when the model is mirrored that direction is now mirrored as well - as in, points to different direction as on the other side of model.

In other words, the model and maps are performing just as you should expect - if you understand the mechanics of how the normal maps change the lighting geometry.


I'm sort of surprised no one seems to have caught that.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Water on June 27, 2011, 07:28:15 pm
I'm sort of surprised no one seems to have caught that.
Talon 1024 did catch that the normal map was faulty. I haven't had any experience with faulty normal maps so it didn't occur to me to look at that area. That's why when the new baked map I created worked well, I went off on the wrong tangent.

Enioch
A simple way to create DXT5nm normal maps.

Open the compressionator and select the normal map.
(http://i229.photobucket.com/albums/ee67/waternz/mesh/Compressionator1.jpg)

Hit compress, It'll ask what format. DXT5nm is the same as DXT5 xGxR
(http://i229.photobucket.com/albums/ee67/waternz/mesh/Compressionator2.jpg)

Then save compressed under the file menu and it's complete. No channel swapping involved.
(http://i229.photobucket.com/albums/ee67/waternz/mesh/Compressionator3.jpg)

Down load the Compressionator http://developer.amd.com/tools/compressonator/pages/default.aspx (http://developer.amd.com/tools/compressonator/pages/default.aspx) Two versions x86 or x64
If it complains on installation then you'll need the Microsoft Visual C++ 2005 Redistributable Package either (x86) (x64)

And sorry for the confusion about the baked maps before Talon pointed out the original map itself was faulty.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Enioch on June 28, 2011, 02:16:34 am
Handy little software. I'd never heard about it.

Downloaded and played with. Very good first impressions.
Title: Re: A UV mappers plight: Normal maps (IMG warning!)
Post by: Herra Tohtori on June 28, 2011, 02:57:33 am
I'm sort of surprised no one seems to have caught that.
Talon 1024 did catch that the normal map was faulty. I haven't had any experience with faulty normal maps so it didn't occur to me to look at that area. That's why when the new baked map I created worked well, I went off on the wrong tangent.

Ah, right. I probably missed that post when I glanced through the thread, and concentrated on the last post on my reply.