[Scooby_Doo]

I've been having a terrible time with normal mapping.  Hopefully someone will know what the heck is going wrong.

Take this photoshop bumpmap texture (in jpg format)


Using xNormal to convert it:


and photoshop's dds to convert to:
http://scoobydoo.freespacemods.net/Problems/test-normal.dds

However this is what I'm seeing ingame:



I can't see any errors in the Xnormal conversion, so where the heck are those massive alias boxes coming from?

[taylor]

The base layer of the DDS appears fine, but the mipmaps are screwed up, and that's what you are seeing.  I'm not sure what options you have available in there but look for something which specifies that it's a normal map, or something about re-normalizing mipmaps.  Hopefully that will fix it.  It does just appear to be a crazy compressor issue though, so using something else might help as well.

Here is a version I quickly made with nvtt, using "-normal -bc3n", directly from test-1.jpg with no modifications.  See how it works for you in-game.

I'm also attaching a PNG where the top half is a mipmap level from your test-normal.dds, and the bottom part is from my test-1.dds.  The images were flattened and level adjusted to make things easier to see, but are otherwise unchanged.  Zoom in and you will clearly see the difference between the two.

[attachment deleted by ninja]

[Scooby_Doo]

Sorry that made a huge mess


I'll look into the mipmap and see if that'll help

[Scooby_Doo]

Sorry don't think it's a mipmap issue.  I rebuilt the dds file, turned off mipmap generation.  The results were the same

[taylor]

Strange.  Your test-normal.dds has the same obvious artifacts on the mipmap levels as your screenshot.  I don't see any in the version that I made, so I have no idea why it would still have problems, much less be worse.

Can you send me the models/maps in question so that I can have a look at it in-game?

[Scooby_Doo]

Check pvt mgs 

[taylor]

The artifacts on my image from due to the conversion from a JPG.  I should have realized that right away.

Anyway, after playing with it a bit I think it's due to a combination of mapping issues and compression artifacts.  The artifacts are there in all compressed images but you don't normally notice them unless you zoom in close.  In this case the texture actually is zoomed it pretty close when applied, so you end up getting a seriously close up view of the artifacts.  I was able to clean up the quality a bit by regenerating the normal map with various different settings, but the artifacts are still quite obvious.  Basically this just seems like more of a mapping issue rather than a map issue.

Other than changing the mapping to fix the problem (which I can only assume would be more difficult):

1) Bump the map up to 4096x4096 to get to a high enough resolution to hide the artifacts (I tested it, looks pretty good)

2) Go with uncompressed DDS instead, possibly lowering the resolution to 512x512 if possible (would then use just 2x the memory as the 1024x1024 compressed version).

An uncompressed DDS of 1024x1024 will use the same amount of memory as the compressed 4096x4096 version, which is why I would try to lower the dimensions if you go with the uncompressed option.  I gave a 2048x2048 version a quick test as well, and while it did look better than the 1024x1024 version, it was still rather obvious and nowhere near the quality of the 4096x4096 version.

[Scooby_Doo]

So either an compressed 4096² or a uncompressed 512² then?  I'll have to give them a try and see which one least impacts performance.

[taylor]

Yeah, either should work better than the 1024x1024 version.  When applied to the model it's about the same as going into Photoshop and zooming in 1000% into the image, it's simply impossible to hide the compression artifacts unless you either throw enough pixels at it or simply don't use compression.

I was off on the memory comparisons though (it was far too late to let my brain try and do math).  An uncompressed 1024² will use the same amount of memory as a compressed 2048², and an uncompressed 512² will use the same amount of memory as the compressed 1024².  The compressed 4096² will require 16m of memory whereas it's 4m and 1m for the others respectively.  So if you can get away with an uncompressed 512² then you aren't going to use any additional memory, and going with an uncompressed 1024² would use 4x the memory as before.

[Scooby_Doo]

Great now I can't get it to export uncompressed. Complains about too many channels (the alpha channel)

Also the 4096 didn't seem to work very well... The result was very odd.

[taylor]

Great now I can't get it to export uncompressed. Complains about too many channels (the alpha channel)

What uncompressed format are you trying to save as?  Sounds like it is trying to make it u888 (RGB) instead of u8888 (RGBA).  For testing purposes you can probably just save it as TGA, try it out and see if the works in-game, then convert it to uncompressed DDS later.  Just remember to get any existing DDS versions out of the way first, since the game will prefer loading those over the TGA.

Also the 4096 didn't seem to work very well... The result was very odd.

Does your card support 4096² textures?  If not then it would get resized by the card/drivers and most likely end up strange.

[Scooby_Doo]

I finally got it to convert, not sure what the problem was about the extra channel....

But another issue has come up, the normal map looks inverted, the areas that are suppose to be darkened look light and visa versa.

[Water]

But another issue has come up, the normal map looks inverted, the areas that are suppose to be darkened look light and visa versa.

Try +X -Y +Z in xNormal

[Scooby_Doo]

Thanks, thats the "flip button" i was looking for