Hard Light Productions Forums

Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: Talon 1024 on October 18, 2007, 05:39:03 pm

Title: How do I create a normal map that works?
Post by: Talon 1024 on October 18, 2007, 05:39:03 pm
Hi there.  Since I've upgraded my drivers, i've just been able to make the "normal maps" feature work.  It all looks very cool and the geometry of the model looks more defined. :yes:

I'm having some problems creating normal maps myself.  Somehow, the light seems to behave oddly at different positions.  I'm using the latest version of Crazybump and trying to render a normal map that looks "somewhat" like yours.

(http://www.ciinet.org/kevin/wc1remake/images/screen0108.png)

(http://www.ciinet.org/kevin/wc1remake/images/screen0107.png)

And if you want the original normal map, here you go.

(http://www.ciinet.org/kevin/wc1remake/images/Normalmap.png)
Title: Re: How do I create a normal map that works?
Post by: taylor on October 18, 2007, 05:44:44 pm
Did you convert the map to DXT5nm?  That's how it looks actually, like you didn't convert the map to the proper texture format.
Title: Re: How do I create a normal map that works?
Post by: Talon 1024 on October 18, 2007, 06:10:44 pm
I knew there was DXT5, but i didn't know there was DXT5nm.  I'm using DDS Converter 2.1 (http://files.filefront.com/DDS+Converter+21/;4470512;/fileinfo.html) to convert to DDS.
Title: Re: How do I create a normal map that works?
Post by: Nuke on October 19, 2007, 08:33:55 pm
the only real difference is what chanels the normal data gets stored it.
Title: Re: How do I create a normal map that works?
Post by: taylor on October 19, 2007, 10:03:45 pm
the only real difference is what chanels the normal data gets stored it.
Well, actually DXT5nm has much higher image quality for normal maps, and it only stores 2 of the 3 channels required for the normal map (given 2 channels the shader can generate the 3rd itself).  But, this only applies to normal maps, since they are special and allow this sort of thing.  In all respects though, DXT5nm is technically identical to regular DXT5, it's just that the "nm" version takes advantage of the particulars of how normal maps work combined with how the DXT5 compression format works.


The problem for us is that it's got to be one way or the other by default, since the shaders expect the image to be in one particular format (since with DXT5nm it must generate the 3rd channel).

On the main topic though, I have never used DDS Converter, so I have no idea whether it supports DXT5nm or not.  You could try using either NVIDIA's DDS utilities, or go with ATI's Compressonator (http://ati.amd.com/developer/compressonator.html).
Title: Re: How do I create a normal map that works?
Post by: Nuke on October 20, 2007, 03:28:28 am
when i re-downloaded the photoshop plugin from nvidia it had support for the dxt5nm format. really you just need something that can save or convert to dxt5 from a 4 channel format. its just a matter of moving red into alpha and clearing clearing red and blue before converting/saving. as i understand it, green and alpha are the channels which suffer less loss from the compression, and x goes into one and y into the other and the z is extrapolated. it would be cool to have an option between dxt5nm and an uncompressed, 2 channel 16 bit format though.
Title: Re: How do I create a normal map that works?
Post by: Scooby_Doo on October 20, 2007, 05:01:56 am
Do normal maps need to be DXT5? What about DXT1?
Title: Re: How do I create a normal map that works?
Post by: taylor on October 20, 2007, 08:06:47 am
Do normal maps need to be DXT5? What about DXT1?
With the default shaders, it must be DXT5nm.  If you create your own shader set then you can make the normal maps any format that you want.
Title: Re: How do I create a normal map that works?
Post by: DaBrain on October 20, 2007, 04:36:48 pm
when i re-downloaded the photoshop plugin from nvidia it had support for the dxt5nm format. really you just need something that can save or convert to dxt5 from a 4 channel format. its just a matter of moving red into alpha and clearing clearing red and blue before converting/saving. as i understand it, green and alpha are the channels which suffer less loss from the compression, and x goes into one and y into the other and the z is extrapolated. it would be cool to have an option between dxt5nm and an uncompressed, 2 channel 16 bit format though.

That's what the material system is for... or better "will be for".

Everything else we can do now would be a hack. Unless ou completely switch to another format.
DXT5nm is the best normal map format anyway imho.
Title: Re: How do I create a normal map that works?
Post by: Scooby_Doo on October 20, 2007, 05:18:00 pm
Do normal maps need to be DXT5? What about DXT1?
With the default shaders, it must be DXT5nm.  If you create your own shader set then you can make the normal maps any format that you want.

Ahhh nuts, now I need to go back and redo them.
Title: Re: How do I create a normal map that works?
Post by: taylor on October 20, 2007, 09:23:42 pm
... it would be cool to have an option between dxt5nm and an uncompressed, 2 channel 16 bit format though.
Somehow I missed this last sentence when I was reading your post.  But, as DaBrain said, this is something that the material system would allow for.  I did actually add support for L8A8 DDS format (which is 2 8-bit channels, giving a 2-channel 16-bit format), but at the moment I have no plans to add this code to CVS (not for 3.6.10 anyway).  It was determined that DXT5nm would provide all of the quality that we need (it loses 2-bits of precision on the luminance channel compared to L8A8), and requires half of the memory.  Memory efficiency was far more important than that little bit of extra quality.  But as I've said, if you want to use a different format then just modify the shaders and include them with your mod.  All normal maps that you use would be required to be in the same format, but that is a choice that you can make and deal with on your own.

The problem with the current code (in general, not the shader/OpenGL code specifically) is that it's all or nothing.  In order to use L8A8, every single normal map would have to be in that format.  That same with DXT5nm, you either use it or you don't, you can't simply pick and choose for each map.  The material code would allow this restriction to be removed, but there is no way to do it with the current code.
Title: Re: How do I create a normal map that works?
Post by: DaBrain on October 21, 2007, 03:25:34 am
Btw, you should try to fix a problem in your nomal map before you convert it.

There seems to be a smoothing group problem on the high or low-poly model. Maybe it's simply unclean geometry (two vertex points where only one is supposed to be).

(http://i20.tinypic.com/16asmfc.jpg)
Title: Re: How do I create a normal map that works?
Post by: Herra Tohtori on December 08, 2007, 11:46:14 am
 :bump:

Old topic, but related questions, so I figured it better to bump the topic instead of making a new one...

In relation to a little experiment (landscape model, basically) I'm working on, I'm trying to make animated water surface with (naturally) animated normal map to take care of fancy reflections, but I'm a bit stumped. I now have something like this:

(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)
(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)
(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)
(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)

Tileable, 60 frames, original size 512x512 (I might reduce it to 256^2 in the end). So that's supposed to be the elevation of the water's surface so that dark equals bottom of a wave and bright equals top of a wave.

Obviously the zeroth question is if it's even possible to use animation as a normal map. If yes, I'm assuming a standard *.eff would work, using frames of individual normal maps, right?

Now, what I'm wondering is how to make a normal map out of this kind of grayscale elevation image (well, it's not really greyscale, it's RGB but it just happens to be gray so I'm using that word a bit loosely here). I suppose I need to convert it to that cyan/magenta thingy where different slopes are identified by different colours? If so, how do I do it?

I'm using NVidia DDS utilities, and the command

nvdxt -file wave_00.tga n9x9

does produce something that has blue and violet stripes in it - am I right to assume that's one way to generate a normal map? Please correct me if I'm completely wrong. The file the command produces is named wave_00.dds. But is this file in which format then? Do I need to first convert the grayscale images into normal map format, and then convert them to DXT5nm format with

nvdxt -file wave_00.dds -dxt5nm

or what?

Secondly, if I use command

nvdxt -file wave_00.tga -dxt5nm

or, in other words, convert the black/white elevation tga map directly to DXT5nm format, it makes a file where (when viewed with GIMP) dark parts become transparent and bright parts remain visible... what is this, then? Can I use that directly without going through the colourful phase?


Final question - if I wanted to use a height map for the wave animation as well, what format would that use? Simple greyscale DXT5 or what? And would it even be worth it to do that in the first place?

Any and all help appreciated very much. :)
Title: Re: How do I create a normal map that works?
Post by: DaBrain on December 08, 2007, 02:09:29 pm
The last time I checked it animated normal maps didn't work. I think they would be useful for a few things though.

Anyway, I think for water, texture translastion would be a good method to save memory. At least if it is a rather simple animation like yours (left to right). That means animated UVs.

The DXT5nm one should work already. I bet Gimp is displaying the Alpha channel (the Y-Axis for the normals) as transparence.


Height should be either uncompressed TGA/DDS 8:8:8, or DXT1. (-height)
Title: Re: How do I create a normal map that works?
Post by: Herra Tohtori on December 08, 2007, 02:24:00 pm
Yeah, I pretty much figured out animated normals doesn't work yet.

But at any case, if you pay attention to individualy waves you'll notice they don't simply move from left to right, they slightly change their form as well. So simple texture translation wouldn't work as I intended.


Apparently animated shinemaps don't work either. Main texture and the glowmaps seem to be the only ones that work. Oh well, seems like I need to simply include the waves into the main texture as animation... and perhaps some day the normal maps will work animated as well.
Title: Re: How do I create a normal map that works?
Post by: taylor on December 08, 2007, 03:33:21 pm
When I originally did the normal map code it was without shaders.  It worked ok, but not great.  It didn't support DXT5nm either, which meant that you had a choice between really bad quality images, or complete memory hogs.  And that's why I didn't allow for animated normal/height maps.

There isn't much reason to keep it that way now though, so I'll add that to my list of things to do next week.


Apparently animated shinemaps don't work either. Main texture and the glowmaps seem to be the only ones that work. Oh well, seems like I need to simply include the waves into the main texture as animation... and perhaps some day the normal maps will work animated as well.
Support for animated specmaps was added over a year ago.  It's been a while since I actually tried it out though so I suppose it's possible that something got b0rked at some point.  I'll take a look at it tomorrow and make sure that everything is still good.
Title: Re: How do I create a normal map that works?
Post by: Herra Tohtori on December 08, 2007, 04:26:31 pm
Thanks for information. :)

I did manage to make a normal map out of the files, but since the animations don't work yet I'm dropping that lane of thought for now.

As to animated spec maps, well, I was trying with -spec after the texture name instead of -shine, so it's not surprising that didn't work at that point... :rolleyes: which doesn't rule out the possibility of it not working after all, haven't tried with correct filenames yet.
Title: Re: How do I create a normal map that works?
Post by: Talon 1024 on December 08, 2007, 08:57:19 pm
Btw, you should try to fix a problem in your nomal map before you convert it.

There seems to be a smoothing group problem on the high or low-poly model. Maybe it's simply unclean geometry (two vertex points where only one is supposed to be).
Thank you.  That really helped me out.

I generated the normal maps with xNormal.  Apparently, xN can't read smoothing group data from a 3DS model, so I had to stick with ms3d format instead.  That seems to fix the problem.
Title: Re: How do I create a normal map that works?
Post by: Nuke on December 08, 2007, 11:28:00 pm
:bump:

Old topic, but related questions, so I figured it better to bump the topic instead of making a new one...

In relation to a little experiment (landscape model, basically) I'm working on, I'm trying to make animated water surface with (naturally) animated normal map to take care of fancy reflections, but I'm a bit stumped. I now have something like this:

(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)
(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)
(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)
(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)(http://i4.tinypic.com/71q6jbm.gif)

Tileable, 60 frames, original size 512x512 (I might reduce it to 256^2 in the end). So that's supposed to be the elevation of the water's surface so that dark equals bottom of a wave and bright equals top of a wave.

Obviously the zeroth question is if it's even possible to use animation as a normal map. If yes, I'm assuming a standard *.eff would work, using frames of individual normal maps, right?

Now, what I'm wondering is how to make a normal map out of this kind of grayscale elevation image (well, it's not really greyscale, it's RGB but it just happens to be gray so I'm using that word a bit loosely here). I suppose I need to convert it to that cyan/magenta thingy where different slopes are identified by different colours? If so, how do I do it?

I'm using NVidia DDS utilities, and the command

nvdxt -file wave_00.tga n9x9

does produce something that has blue and violet stripes in it - am I right to assume that's one way to generate a normal map? Please correct me if I'm completely wrong. The file the command produces is named wave_00.dds. But is this file in which format then? Do I need to first convert the grayscale images into normal map format, and then convert them to DXT5nm format with

nvdxt -file wave_00.dds -dxt5nm

or what?

Secondly, if I use command

nvdxt -file wave_00.tga -dxt5nm

or, in other words, convert the black/white elevation tga map directly to DXT5nm format, it makes a file where (when viewed with GIMP) dark parts become transparent and bright parts remain visible... what is this, then? Can I use that directly without going through the colourful phase?


Final question - if I wanted to use a height map for the wave animation as well, what format would that use? Simple greyscale DXT5 or what? And would it even be worth it to do that in the first place?

Any and all help appreciated very much. :)

what i would do is make a plane in max and map the texture to it so it fits entirely on the plane. then id apply alot of iterations of mesh smooth and a displace filter to it (use a frame of the map as your height). then id eport it for ati's normal map renderer. then remove the modifiers and save it as your low-poly model (you may need to lower it first).

hard part is you need to do this on a per frame basis. once for each frame. also it will take awhile to crunch so you might want to set up a batch file to do it overnight.  then when theyre all rendered to tga, open in photoshop and re-save as dxt5_nm.
Title: Re: How do I create a normal map that works?
Post by: Herra Tohtori on December 09, 2007, 12:03:56 am
...what? I understood only the first sentence completely, and even if I knew how to do iterations and other more advanced modelling stuff, I don't have Max. Thanks for suggestions, though... I guess.

As far as normal map making goes, I've pretty much figured out how to make them (- thanks for help about the formats!)... at least to some degree. Judge for yourself:

(http://i6.tinypic.com/7w4jk9s.png) (http://i2.tinypic.com/72gd4it.png)
(http://i2.tinypic.com/8b4u1zr.png) (http://i9.tinypic.com/81ge7wp.png)
(http://i10.tinypic.com/73p91c7.png) (http://i6.tinypic.com/7xsqmgg.png)
(http://i3.tinypic.com/7yn0js2.png) (http://i10.tinypic.com/6ybd6io.png)

These are made based on the main textures' seam lines, nothing fancier than that. I simply drew them on a layer over the texture, then made a normal map out of that and coverted that to dxt5nm format. Works better than none anyway... :p
Title: Re: How do I create a normal map that works?
Post by: Nuke on December 09, 2007, 01:25:19 am
well when you use the mesh smooth modifier, theres an iterations box that lets you specify how many passes of mesh smooth you want to do (higher number = more polies). dont go over 6 cause it will slow max down to an unuseable level. thats all you need for that one. youre really just using it as a subdivider (and the subdivide modifier sucks :D ).

on top of that modifier use displace modifier. set a displacement value, 2 or 3 works well. a decay value if you wish. then you get a couple buttons to load an image as a height map. a blur value, adjust as needed, no blur = jagged, high blur = smoother. 0.2 works well on youre map. then you get a section for mapping coords, just click the use existing mapping check box. some tweaking and you get a very high poly mesh which you can use to generate a normal map with. for your low res model use a copy without mesh smooth and displace.

(http://i213.photobucket.com/albums/cc103/Emperor_of_Nihil/Untitled-7.jpg)

i think this gives a better result than converting a height to a normal with the nvidia photoshop plugin. i did this in 2 minutes btw, the normal map compiler takes a little longer :D

btw, cool normal maps :D
Title: Re: How do I create a normal map that works?
Post by: DaBrain on December 09, 2007, 04:28:26 am
@Herra Tohtori

Yepp, not too bad. :)

Now you should really try to help me out with the FS2 normal maps.

Just pick a texture and ask me if it's still needed to be done, in this thread:
http://www.hard-light.net/forums/index.php/topic,50309.0.html


You'll get better at this in no time. ;)
Title: Re: How do I create a normal map that works?
Post by: Nuke on December 09, 2007, 05:59:48 am
are you re-uv mapping the ships to better accept the normal maps?
Title: Re: How do I create a normal map that works?
Post by: DaBrain on December 09, 2007, 06:33:04 am
So far - no, but the use of the UV space is quite sadly low for a few ships, so I'm always tempted to redo the whole thing.

I was able to resist that urge though. So far... I plan on redoing one FS2 fighter from the scratch in real next-gen quality in the future.