Hard Light Productions Forums
Community Projects => The FreeSpace Upgrade Project => Topic started by: taylor on December 10, 2005, 08:35:51 am
-
There is a growing problem of large and detailed models, which do look great, but their use causes big slowdowns and excessive memory usage. There have been conversations about this stuff before but most of those were between Lightspeed and myself when he was making all of the new texture map for old models. Similar conversations have taken place over effects but most of those too were more private than public. In a Mantis bug about mipmaps and LODs, VA asked numerous questions about how to do some the things that I was hinting at for reducing memory usage. Considering he is making a lot of the really nice models, the fact that he asked those questions made me a happy man. There isn't going to be one solution to all of this but at some point it's the model makers that are going to have to take blame for the ever increasing memory usage and slower render times. As coders we can only do so much until our fixes no longer do enough to control the problems. At that point there gets to be more pressure inside the ranks to impose certain limits on effects and texture sizes. I don't think that's a good thing for anyone but it's up to the makers of the content to do the right thing, impose some creative controls on themselves, and provide the rest of the help needed to keep the main game and mods working as well as they really could.
I'll go over the basic problems we have first and then get to questions/solutions:
1) Too many bitmaps are being used.
There is currently a limit of 4,750 bitmaps that can be loaded at one time. Not all of those will actually be in memory at any given moment but all of them need to be available for use constantly. That number may appear high but a typical mission with MediaVPs in use will easily use over 3,000 of the available slots. When it goes over the max is when we have the texture corruption and various other rendering issues that we used to have really bad. 3,500 used to be the limit for retail and after considerable work on changing how bmpman works a new limit of 4,750 was put into place. THAT LIMIT WILL NOT BE INCREASED AGAIN. The limit was calculated (by myself) based on the code changes to bmpman, the largest texture map using mission available at the time, plus an extra buffer factor for the future. In theory, if the effect and model producers followed certain rules (which almost no one knows at this point), then it would actually be possible to drop that limit to a lower level given the changes to bmpman.
2) Textures are too large and detailed.
While this may not seem like a real problem, I guarantee you that 2048x2048 model textures serve almost no purpose in any game. 99% of the time you just can't see that much detail, especially in the FS2 engine, but you have this massive texture stuck in memory anyway. Now I'm not about to say what size textures should be since that will always depend on the model or effect but this is where the self control comes into play. You should always try to use the smallest texture map possible to not only improve rendering speed but decrease memory usage. Just because a compressed DDS texture allows you twice the map size for the same memory usage does not mean you should actually do that. Remember that this is a game not an art show. There is a ton of content that needs to be loaded and rendered, and you have to remember that any model is nothing but a very small part of a very big game. Mods get a bit more play with that statement than anyone making upgrades for standard ships but mod makers probably know this already since they must take the mod as a whole into account when doing anything.
---2011 Addendum---
While this was certainly true back in 2005, recent releases have used 2048^2 as the default texture resolution. In the end, the specific size used doesn't matter, taylor's main point is still valid. Just because modern cards can handle texture sizes up to 16k^2, doesn't mean you should use them. At the moment, 4096^2 is the maximum resolution that is still practical, there are only very few scenarios where going beyond that makes sense.
3) Inefficient use of technology.
Stay away from TGA textures unless you really need them. Reuse textures from other models when possible rather than including a copy with a different filename. Use compressed DDS whenever possible and make sure that mipmaps are generated with it. Include IBX files with all models that get released. Keep animation frame counts to a minimum. The list goes on but you get the idea. There is lots of advice for making things work more efficiently so just pay attention to that and apply as many of those things as possible to whatever you are making. Too many people don't.
Now to those questions:
1) What does FS do with LOD textures? Ie, does it load them all into memory and switch when needed, or does it only load them when it needs to switch?
When a model is loaded all of the textures used by that model are also loaded. Switching LODs would be painfully slow unless all of the needed content was already in memory, plus there will be many times that the same basic model is going to be on the screen many times using various LODs.
2) Do different formats of textures affect performance/memory taken up? And if so, by how much?
A DXT5 texture will take at most 1/4th of the memory of a TGA texture of the same size. A DXT1 texture will take between 1/8th and 1/6th of the memory of that TGA texture. If you feel the need to use TGA then make sure to use 24-bit TGA if you don't need an alpha channel. All DXT1, DXT3, and DXT5 textures are used as is, there is no additional processing on the images before they get shown on the screen. All other textures require some amount of work before they can be shown and so are inherently slower. Those compressed textures come at a price or course, quality. That loss of quality will have to be considered by whoever is making the images. As a general rule you can determine memory usage like this: (width * height * bytes_per_pixel). An 8-bit file is 1-byte per pixel, a 16-bit file is 2-bytes, a 24-bit file is 3-bytes, and a 32-bit file is 4-bytes. So, a 1024x1024 32-bit TGA needs 4meg of RAM (1024 * 1024 * 4). A PCX image is 8-bit but used as 16-bit in the game so you always use "2" for the bytes_per_pixel when looking at PCX. JPG images are always 3-bytes and use the same formula for size calculation. The DXT? formats do not use that formula, when loaded in memory they will use the same amount as the size of the file (ie. a 1meg file will take 1meg of RAM).
3) Approximately what would be the 'perfect LOD' consist of?
The fewer textures the better. Lots of polys can usually be handled pretty well but you don't want to be wasteful with that. If you are really close to something then more polys is great. The further away you are then you'll tend to get more models in view so if you aren't dropping enough polys going from LOD0 to LOD1 to LOD2 then it hurts overall performance. You don't want it to look like crap of course but if LOD1 looks exactly the same as LOD0 then you are probably doing something wrong, or you are really, really good at what you do. Todays graphics cards are very fast at rendering the basic model but it's the texturing always tends to slow things down. Use few textures, try to keep the overall dimensions of those textures as small as possible without losing too much detail, use DXT? textures with mipmaps, try to use those same textures on as many LODs as possible. If you can get away with using the LOD0 texture on LOD1 then do so, if it has mipmaps then you get faster rendering, better quality, and a significant reduction in memory usage. Use smaller textures for subsequent LODs if you aren't reusing the same texture, if LOD0 is 1024x1024 and you use a different texture for LOD1 then that LOD1 texture shouldn't still be 1024x1024. Remember that LOD2 and beyond are used when the model is getting far away so keep making smaller and smaller textures for those (or reuse the same texture, with mipmaps, from earlier LODs). If using animated glowmaps then drop the number of frames as well as the dimensions for different LODs, by LOD2 you usually can't see the animation anyway. And when I say to try and use the same texture for LOD0 that was used for LOD1 I don't mean one really big texture but the same texture. If it's mipmaped then it will work much better than a different texture. Doing it this way can't always be done but it should be whenever possible.
4) Is it more or less efficient to have a really big ship use 2 big 2048 res maps as opposed to say, 8 smaller ones (256), and is the vastly superior visual quality worth the cost in terms of RAM usage?
The fewer textures the better. Try not to waste the texture, nearly every part of the texture image should apply to some part of the model. If you've got a bunch of black space in the texture then you didn't map it very well. As mention in the above responses you have to try and control the want of quality with the need for usability. Not only is the visual quality usually not worth the RAM and general slowdown of a 2048x2048 texture but since the texture will have to get resized 99% of the time it's on the screen the visual quality will actually decreased in when use. Use the fewest and the smallest textures that you can in order to get the required detail across to the gamer. This will mean sacrificing something but when it comes down to sacrificing some detail in order to keep the game playable I would think that no hesitation would come from ditching that extra detail. Remember than each model is just a piece of the game and needs to be created knowing that the more detail that's included in the model's textures is a price that the game as a whole has to pay. What you can get away with will always differ based on the model but that's why this is a consideration for the model makers and those that create textures. Oh and never, EVER use a texture larger than 2048x2048. That's the largest size that most cards can use and anything larger will get resized to that automatically but will also seriously hurt rendering speed, memory, and image quality.
Again, modern GPUs, even Intel chips, can be expected to support texture resolutions going up to at least 2048^2. That being said, this does not mean every fighter needs a map that detailed, so choose your battles. As a rule of thumb, the larger the ship, the greater the impact of a high-res map.
5) Are nameplates fine as they are or are they costing us precious frames per second/megs of RAM?
They should be fine. Those are extra of course but generally add enough to the game experience to justify them being there. Just remember to use PCX or DXT5 for those so you don't waste unnecessary memory on TGA versions. It's quite likely that nameplates will be automatically generated at some point down the road since that's possible with render-to-texture and future font code. That's not going to happen tomorrow or anything but is something to look forward to.
6) Big one: Is the cost of storing even efficient LODs in the RAM worth it?
(Especially on the highest detail settings, where the lods don't appear to be used anyway?) Could it be more efficient RAM-wise to keep using the highest detail model, but with smaller textures?
LODs do get used always but the distance at which they change will vary when using the various detail settings. If you don't LOD efficiently (ie. you don't drop enough model detail to really justify LOD drop) then that can hurt efficiently. Properly using DDS textures with mipmaps will greatly help speed though. Even on the same LOD the mipmap levels will automatically change to help speed rendering and reduce visual quality loss from resized textures. Use fewer/smaller textures and let that be how you save RAM. As already mentioned, most modern video cards to fully capable of rendering very detailed models without skipping a beat. The OpenGL code especially has been optimized to deal with complex models effectively and there is still room to improve the code in the future. It's just the textures that cause the problems. For some like the hi-poly Triton for example, I use the highest detail version but with lower quality textures (even if I just resize the hi-quality ones myself). It still looks great and renders fast but I come out way ahead on memory saving. There is a comment I added to bmpman.h when I rewrote how it handles most images. This comment is something of a flame since I was pretty pissed about having to deal with this problem because of the artists. I used as an example the Terran Mara (before the hi-poly one) as an example of the texture abuse that was going on. The Terran Mara was using 117 bitmaps, taking into account the glow maps where each frame is considered one texture by the game.
And while I'm here, might it be possible to dynamically resize textures based on the in-game detail settings? Before they're loaded into RAM in the first place that is? Ie, if they have it at 50% detail, all the textures above say, 512 res are halved in size. At 80%, they get 80% of the full textures etc.
Basically it puts the efficiency cleanly in the end users control. If they have trouble running at full, they can just bite the bullet and tone down the settings, that way those who've paid for powerful rigs can get the best graphics, and those with less powerful rigs can get as much as they can.
This is already controlled by the "3-D Hardware Textures" slider in the detail options in game. There are two problems with this though, it doesn't have any affect on compressed DDS textures, and it doesn't save any system memory. That option only does anything to help rendering speed. There has been something else proposed recently though. WMCoolmon proposed different filenames so that high, medium, and low quality textures could be always be available and a new setting would choose what gets loaded by default. I rather like the idea of different directories myself, use controlled by an existing in-game detail setting. For the directory option you would have "data/maps_lo" for low quality maps, "data/maps" would be for medium quality, and "data/maps_hi" would be for high quality maps. "data/maps" would always be a fallback if the other directories didn't have the file. Both proposals are possible, we just have to decided which is better for the modders to use (obviously with their input).
And something that wasn't asked but an old feature that I wish more people would use, special models for the hud target box. The ships.tbl has an entry for a special model to use for the hud target box, this model is always rendered at full detail. Using a much lower poly model for this would seriously help rendering speeds when you have something targeted. Also be sure to use as small of textures as possible since that model won't ever be rendered very large. That's extra work so it's not something that I would want to force on anyone. It does help rendering speeds though and it would allow something like TBP to use quick rendering wireframe models for the targetbox since wireframe hi-poly models are chaotic to look at and very slow to draw.
In summary:
- Use LODs effectively
Try to use the same textures on more than one LOD, but be sure so include mipmaps with the texture. This will reduce memory usage and rendering time. Make sure that there is enough geometry difference between LODs to help rendering. Always test in game when deciding how much geometry you can drop, but drop as much as you can. LOD changes are based on distance and detail levels. The "Model Detail" slider in the options screen can set these levels, it takes effect instantly and you can adjust the slider during a mission (by pressing F2 and going to the Details tab) to see the various effects.
- Use smaller textures
Try to keep textures at or under 1024x1024 if at all possible. Going with 1024x512, 512x512, or 512x256 should take care of most models without sacrificing much, if any, detail. Include mipmaps so those textures are used effectively. The graphics API will scale up textures if need be and you should let it rather than trying to make the texture as large as possible. Use smaller glow and spec maps whenever you can, those tend to not need as much detail as the main texture.
- Use a special hud targetbox model if you can
This won't save any memory, it will actually use more. But it can greatly increase rendering speed which would negate any impact the small amount of extra memory might take. It would also allow effective use of wireframe views in the targetbox since they are much slower to draw with complex models.
- Use DDS textures when at all possible
There are compressed (DXT1, DXT3, DXT5) and uncompressed DDS images available so use compressed when you can and go with uncompressed otherwise. Compressed images are both faster to process and use and consume at least 1/4th of the memory as uncompressed versions but do decrease quality to some degree. Always generate mipmaps for your DDS images.
- Always use power-of-2 textures
Textures are much faster to use if they are power-of-2. Power-of-2 textures are some combination in width or height of these dimensions: 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048.
- Choose your texture sizes sensibly
While the usable texture sizes increase regularly as new GPU generations are rolled out, be aware that there's a point where increases in size do no longer yield appreciable differences in quality. In addition, note that textures use the lions' share of GPU memory, and that texture lookups are rather expensive to perform. Right now (Right now being defined as 2011), a resolution of 2048^2 is established as the standard size, with some mods offering 4096^2 sized maps as optional downloads.
- Try to reuse other people textures when possible
Things like cockpit graphics, glow points, and even some base textures may be shareable between models. Rather than create a copy of the same image with a different filename just use the same filename. This lets other models use the same texture which saves memory.
- Stay away from animated glow maps when at all possible
They look cool but every frame of that animation counts as another texture that your model uses. If you feel the need to use an animation then try to use an EFF with compressed DDS frames when at all possible. Keep the number of frames used for your animation to a minimum. 25 frames for a glow map would be considered high, not average. Try not to use animated glows for anything less than LOD1, you probably couldn't see it anyway, go with static glows instead. Reusing the LOD0 glowmap (with mipmaps when using EFF/DDS) for at least LOD1 is also a good option.
- Always try to include the .bx file(s) when you distribute a model
This isn't required of course but it would save everyone who uses your model from having to do generate it themselves.
- Remember that one model is just one model, a small piece of the game
There are many other models and textures and effects that need to get loaded every mission. The more detail and the larger the textures used for one model the more it hurts the game as a whole. Every model needs to be designed with the knowledge that it's only one piece of the experience and compromises may have to be made to enhance the experience rather than the model.
Anyway, everyone feel free to comment on this stuff. It would be nice to get opinions and feedback, or even more suggestions, and then maybe something easier to follow can get added to the wiki for everyone follow.
-
Very very interesting. I knew a lot of this before, but the mipmapping issue was new to me.
I remember everyone told me to avoid mipmaps, cause FS2_open didn't use them or would just create it's own mipmaps anyway.
So should I use DXT1/3 textures with mipmaps for standard maps?
The '-mipmap' flag turns on automatic mipmapping, right?
Edit: This should really be sicky!
-
Kudos, I think it was about time somebody tackled this subject.
I really love some of the artwork that you guys create for Freespace but every time I turn my fighter and bring a new Orion or a science vessel (I think this has since been resolved though) into view my framerate takes a massive hit. I consider my computer to be fairly powerful and this makes me wonder how many new people have been put off trying SCP/FSU material because their computer simply wasn't up to the job.
I think there's a certain amount of detachment between what a model looks like when it's being worked on and what it looks like when you're burning past it in a fighter. I know as well that when you're building a new model you're almost entirely fixed upon it and forget that it has to work in game with other meshes and perform well.
Although I haven't built anything for FS2 in a while I give it a decent amount of thought.
-
animated glowmaps could be done away with once hte materials.tbl comes out, saving a significant amount of memory, from the looks of it....
-
I remember everyone told me to avoid mipmaps, cause FS2_open didn't use them or would just create it's own mipmaps anyway.
That did used to be the case but that's slowly been changing over the past year and a half. I'm still not sure of how D3D handles this but OGL is getting a mipmap overhaul and D3D can do the same things, if it doesn't already. There is the difference between mipmaps and LODs which is where some of the confusion is coming from. LODs are basically mipmap levels. In the case of models this means both textures and polys, in the case of effects it's just textures. That's basically what the original bug report was about, the game using LODs rather than mipmaps. That's not true, the game will use LODs for polys and textures but the basic use of mipmaps still applies to the textures regardless of what LOD is in use. Use of mipmaps is automatic and not determined by the game engine but by the graphics API. Use of LODs is done by the game engine. The idea here is to reduce what the LODs do (ie. use the same textures for multiple LODs and let the graphics API deal with the mipmaps) so that we can save memory by not using textures which may not really be needed in the first place but still get the reduction in number of polys that lower LODs allow us.
So should I use DXT1/3 textures with mipmaps for standard maps?
Yes and they will always get used automatically (same for DXT5). What ever you use to create the DDS textures with should give you the option as to how many mipmap levels to create so you don't have to make all of them. It usually goes all the way down to 1x1 for mipmap generation but you just go down as far as 64x64 if you wanted. For effects using DXT? you would want to create only one LOD but with mipmaps. Then set that effect as having just one LOD in weapon_expl.tbl and then the graphics API will use the mipmap level automatically to speed up rendering.
The '-mipmap' flag turns on automatic mipmapping, right?
That option turns on automatic mipmap generation but actual use of mipmaps is automatic whether you use the option or not. Use of mipmap levels for compressed DXT? will get used automatically and automatic mipmap use for uncompressed DDS will get fixed for OGL real soon. Basically use DDS (either compressed or uncompressed) for pretty much everything and you will not ever need -mipmap since the image should already have to mipmaps created. Uncompressed DDS is basically TGA so you don't have to lose any quality using it and you get the option of having mipmaps already generated.
animated glowmaps could be done away with once hte materials.tbl comes out, saving a significant amount of memory, from the looks of it....
Well I don't know if they could actually be done away with on a whole but many of the same effects could probably be done without animated maps. Also remember that we have texture translation which could be used to similate a moving texture when it's really not. This is used in the subspace effect but there is no reason that we couldn't apply it to anything else, if it's not already. Some effects can really only be done with an animated texture map but more options to not require that would only benefit everyone.
-
It seems these comments are focused towards VasudanAdmiral and myself, and we will take note of them in the future. I've always used the medium- or low-detail textures VA provides, particularly on the shine and glowmaps, while I try to use existing FS2 textures that are used on other models.
A big offender in the texture department is the Mjolnir; I wasn't as familiar with the standards as I am now, so (if someone could direct me to the Max scene used for conversion) I'll bake those maps onto a single texture when I can.
And something that wasn't asked but an old feature that I wish more people would use, special models for the hud target box. The ships.tbl has an entry for a special model to use for the hud target box, this model is always rendered at full detail. Using a much lower poly model for this would seriously help rendering speeds when you have something targeted. Also be sure to use as small of textures as possible since that model won't ever be rendered very large. That's extra work so it's not something that I would want to force on anyone. It does help rendering speeds though and it would allow something like TBP to use quick rendering wireframe models for the targetbox since wireframe hi-poly models are chaotic to look at and very slow to draw.
This is something that's bugged me a lot in recent builds. The target view box used to render LOD1 in retail, but now it's rendering LOD0, which is totally unneccesary. The 'simplified model' you want already exists, and is even already in memory, AND was originally coded in, so it's up to you guys to fix that.
In fact, it may be forcing LOD0 down to LOD1 anyway; going into the Lab, all the LOD1s are using LOD0 geometry. Turning down the detail settings doesn't change anything so it must be something else. We go to a lot of trouble to make LODs, it would be nice if they were used. :(
I recommend stickying this thread, and/or copying its contents to the Wiki.
-
It's really directed at everyone who makes either models or textures and especially at mod makers so that they can get a better idea on how to properly balance the user experience. Although I used VA's work as some examples (even if I didn't actually say it was his model) it wasn't really directed towards him or you. Most of that was actually directed towards Lightpseed at one point and I just rehashed those talks from memory and added some new stuff.
As far as the targetbox goes, it does use LOD1 and that has never changed. Detail settings don't affect that since it's independant of the forced detail level use to render the model. Even so though the current crop of hi-poly models are so detailed that even LOD1 is much more geometry than is needed there. "simplified" doesn't mean what it used to and I'd bet that there are LOD2 models now that have more polys than even LOD0 of the original :v: models. Using the special models for the targetbox isn't about reducing memory but about increasing rendering speed. It's a lot of work on the GPU/CPU to render those detailed models in that little box. The textures have to be scaled to fit the scaled geometry and that's processing power better spent on the rest of the models on screen. I'm sure that there are some models that probably increase the drawing time of the hud by a factor of 3 just because of what's needed to render that high quality model in that tiny little box. I know it's extra work which is why I've never really mentioned it before but it is one very effective way to speed things up. It's an option.
-
Not only that, but target box models have to be drawn last, so the textures may have to be copied to the video card again.
-
Linkaged from the FS MODding forum for great justice! (http://www.hard-light.net/forums/index.php/topic,37171.0.html)
-
The target box issue is something that should illustrate a point. If the model shown there is LOD1, then LOD1 should be made to offer minimal visual difference from the lowest LOD at the point when it is the same size on screen as it is in the target box.
-
Awesome, thanks mate. :)
Mipmaps are new to me - I've known roughly what they are, but not how they work or even if FS used them at all. So, for the first of my next set of questions:
- How do we go about getting our ships to use them so we don't need lod textures? (ie, is there something we need to do to the dds texture or some way we need to set up the model?)
- How does the game know(or how do we let it know) when to use a mipmap instead of the full texture?
- Should shine and glowmaps also be mipmapped?
I ask because I've hated creating yet more textures for LOD use, and if mipmaps mean I can get all lods to use the same 1 texture and save memory and save loading time all at the same time, I'll be very happy indeed. :D
4) Is it more or less efficient to have a really big ship use 2 big 2048 res maps as opposed to say, 8 smaller ones (256), and is the vastly superior visual quality worth the cost in terms of RAM usage?
The fewer textures the better. Try not to waste the texture, nearly every part of the texture image should apply to some part of the model. If you've got a bunch of black space in the texture then you didn't map it very well. As mention in the above responses you have to try and control the want of quality with the need for usability. Not only is the visual quality usually not worth the RAM and general slowdown of a 2048x2048 texture but since the texture will have to get resized 99% of the time it's on the screen the visual quality will actually decreased in when use.
^That question was pretty much refering to the Arcadia I'm in the very early stages of building. The arcadia has a lot of surface area that needs to be textured, and I'm completely torn about how to go about it. :\
The problem is, I really hate how texture tiling looks unless it's done really, REALLY well (such as on Omni's Galactica), but in the case of the arcadia, the shear size of the thing means that it will be very difficult if not impossible to get a decent level of detail on it when close up with UV mapped big textures.
What I'm currently considering is pretty much having the Arcadia's basic hull use texture tiling, so that's how it'd be seen from medium range, and using detail boxes to create localised detail all over it, hopefully covering up any shortcommings caused by the use of tiling.
Can you recommend a better solution? Cos I still don't much like this one. :\
It's quite likely that nameplates will be automatically generated at some point down the road since that's possible with render-to-texture and future font code.
Now that would be seriously cool. :D
WMCoolmon proposed different filenames so that high, medium, and low quality textures could be always be available and a new setting would choose what gets loaded by default. I rather like the idea of different directories myself, use controlled by an existing in-game detail setting. For the directory option you would have "data/maps_lo" for low quality maps, "data/maps" would be for medium quality, and "data/maps_hi" would be for high quality maps. "data/maps" would always be a fallback if the other directories didn't have the file. Both proposals are possible, we just have to decided which is better for the modders to use (obviously with their input).
That does sound like a good idea. :) I imagine the simplest way to go about it would be to organise the media VPs so that all textures (and effects??) have 3 levels. The default set is medium, recommended for most users. This would have nothing bigger than 1024 anywhere, except perhaps in special cases like the lucifer, where it looses huge ammounts of detail and becomes rather blurred with a half size texture.
Then you'd have something like the aeffects VP we currently have. This would include all the huge textures (mostly 2048) that were shrunk for the regular media VPs. It would work by being a sort of expansion VP for the regular texture VP.
And finally, you'd have a low detail set, which would completely replace the regular texture VP - downloaded in it's place really. It would contain everything in the medium VP, but they'd all be halved in size.
This way, if someone wants to go with medium, all they need to do is install FSO, download the default media VPs and play. This would be the default option.
Then, if someone has a lot of RAM, they can decide they want the highest quality stuff - which would only require them to download an additional hi_textures VP and play.
If someone has very little RAM and wants to go with the lowest resolutions possible, they can download the lo_textures VP instead of or to replace the regular texture VP.
Also remember that we have texture translation which could be used to similate a moving texture when it's really not. This is used in the subspace effect but there is no reason that we couldn't apply it to anything else, if it's not already.
We can translate textures?!?! How do we do that? That would be incredibly awesome for a number of things we have planned in TI. :D
Ok, so having taken all this into account, would this sort of release be best (using the Triton as the example):
Main Release (this would go into the media VPs)
This would include the following attributes:- Three mesh detail LODs: Highest quality, Target box quality, and very distant quality
- A single 1024 res DDS texture, fully mipmapped for use on the lods
- A 512 res DDS shinemap
- A 512 res DDS glowmap
- A 512 res DDS debris texture
- A 256 res DDS debris glowmap
- A PCX/DDS nameplate
- The IBX file
Highest Detail Expansion (This would be downloaded in addition to the main release if wanted)
This would include the following attributes:- A single 2048 res DDS texture, fully mipmapped for use on the lods
- A 2048 res DDS shinemap
- A 1024 res DDS glowmap
- A 1024 res DDS debris texture
- A 512 res DDS debris glowmap
Low Detail Release (this ones textures would go into the lo_textures VP)
This would include the following attributes:- Three mesh detail LODs: Medium quality, Target box quality, and very distant quality
- A single 512 res DDS texture, fully mipmapped for use on the lods
- A 256 res DDS shinemap
- A 256 res DDS glowmap
- A 256 res DDS debris texture
- (No debris glowmap)
- A PCX/DDS nameplate
- The IBX file
Now, is there anything you see in the above that you reckon could be optimised further? I need a good idea of what the optimal release would consist of, so I can follow it on all future ones (and more than likely, I'll end up doing it for all the older HTL ships too). ;)
-
I don't really want to quote everything so hopefully this isn't too difficult to follow and just gets the past couple of posts at once :) ...
LOD0 through LOD3 is assumed to be usable by the engine for distance use, ie. at 1000 meters it would use LOD1, 1500 meters LOD2, etc. Or at least I think that's right. Bobboau would probably be able to answer that better than I can explain it. Based on the code though that's what it does and the distances are specified in ships.tbl ($Detail distance:). If you have a low quality LOD1 then you have to make sure that the tbl allows for this or it's going to look like crap when it hits that distance and converts to that LOD. The targetbox uses LOD1 because :v: wanted it faster but still basically good looking. This is something that you just need to play with, but the best way (though probably not from the modeler point of view) to handle the targetbox is to make a separate low-detail model with a special/small/single texture and with only LOD0. If there is no LOD1 then it will just fall back on what's available. You may just be able to take LOD1 or LOD2 and make a POF with just that one LOD and use it with a custom texture map which is smaller and wouldn't have a seperate glow map or anything. I can only come at this problem from a coder point of view so I'm biased. One of the model makers will just have to sit down, take a model and play with it, figure out what really works best for them and report back as to whether it's actually worth the work. Some models will get a bigger boost from doing that than others. You probably don't need to worry about it for a fighter but capital ships (and something like the large BSG and TBP models) could see a big improvement from this.
Mipmaps:
- When you create a DDS image it should give you the option to create mipmaps with it. This is usually done by default unless you specifically tell it not to. If you get the option to specify the number of mipmap levels to create then you really just need to go down to 32x32 and that's it. You would typically just specify the number of levels than an actual size but just consider that each level is exactly half the width and height of the prevous level. If you use NVIDIA's nvdxt cmdline tool then you can just convert an entire directory of TGA images to DDS with mipmaps at one time. nvdxt has a lot of options but most are useless. I'll be happy to point out the best options to use if you don't have/use the Photoshop plugin or the GIMP plugin.
- Mipmap use is determined by the graphics API. When you render some textured geometry on the screen it knows what size that geometry is and picks the mipmap level which is a closest match to that size. This is automatic and requires nothing from the user side and other that just making sure you create the mipmaps it doesn't require anything from the model maker side either.
- Both shine and glow maps can and should be mipmapped. Animated glowmaps using an EFF with DDS frame can also have mipmaps and they will get used the same as anything else.
Regarding the Arcadia question, fewer maps is better but few small sized maps is better than that. So if you can get away with 2 2048x2048 textures then ok. But if you could make use of 3 1024x1024 textures instead then that's even better. This, like the Colossus and Lucifer, are cases where the model will dictate slightly different rules. Tile where you can if possible, map when otherwise needed. As long as you aren't being wasteful then you won't get any complaints from me. Some models are just going to require that extra couple of maps and/or those slightly larger maps. Be smart about it but in the end you have to go with what works best.
Texture translation/panning is just where you give a new set of UV coords and the texture translates to that position. Right now it's only used to make the subspace effect appear to be animated. The same effect could apply to most in-game effects as well with a little work. When that might happen I don't know. My todo list is already long enough and until I get some more of my bigger code changes finally into public hands the first of next year I'm not really taking on anything else.
Those releases look good. The main release is just what I would expect to see, it should still look good and even more it will perform well on most everyone's system. I would probably try to go with an even smaller main texture (main release) if possible and smaller specmap on the high detail version but if using only the one set of textures for all LODs it will be a tremendous improvement either way. Your average planet in a mission is 1024x1024 and its typically larger than any ship texture in the view. Graphics cards today can do some great filtering when magnifying textures so try and let it do it's thing when possible. Better filtering control (at least with OGL) and support for more techniques (ATI and NVIDIA specific stuff) is in the works, anisotropic filtering level just became a registry option for instance. Either way what you've got listed there is a great improvement so run with it. As the graphics capabilities start getting even better perhaps it can be revisited in another year and we'll see what can be done with textures then.
Remember that PCX nameplates won't be able to use mipmaps and a compressed DDS nameplate, with mipmaps, will use half the memory of the PCX version plus offer a full alpha channel. If you don't need the alpha channel then go with DXT1 and it will use even less memory. Nameplates are typically simple graphics so compression artifacts shouldn't be an issue.
-
Right, thanks. That pretty much answers all my questions. :) (for now anyway ;) )
Having looked at the triton and aeolus at 512 res textures, I think I'll stick with 1024 sizes as the main release size for ships of similar size. At 512, most of the details make you think you need glasses, and edges become....bluredly jagged. :\
As a main release for a fighter/bomber it might be ok, but not really a cruiser sized freighter. :)
So if you can get away with 2 2048x2048 textures then ok. But if you could make use of 3 1024x1024 textures instead then that's even better.
Well, the main problem is texture area->ingame surface area, and 2 2048 res textures would give me the area equivilant to 8 1024 res ones, but that's already a heck of a lot of memory. :\
I *think* what I'll do is to aim for 1 2048 res, but probably end up with two. These would obviously go into the highest detail release, with 2 1024 res versions as the main. (I wouldn't dare go lower for a station the size of the Arcadia - it's (http://www.freewebs.com/twisted-infinities-va/HTL-Arcadia/ArcadiaFullyLoaded1.jpg) huge! (http://www.freewebs.com/twisted-infinities-va/HTL-Arcadia/ArcadiaFullyLoaded2.jpg)
Thanks again. :)
-
Why not 1x 2048^ and 1x 1024^ ?
-
Because of the weird shape of the Arcadia, there are actually very few places where I can get away using the same texture portion twice, so to get an even and not-blurred coverage, I'll need a lot of texture area to play with. A single 2048 is....rather optimistic. ;)
Well, we'll see how it goes at least. ;)
-
Actually I suggested a table. I didn't want to bring that in to the discussion, since it's not particularly relevant, but I hate being misquoted.
Hmm, what about some kind of quality table? This would hook into the bm_load and bm_load_animation functions. Essentially, how it'd work would be that whenever bm_load or bm_load_animation receives a name for an image to load, it checks the quality table. If the quality table exists, it checks the quality that the user has specified, and retrieves the name of the texture with that quality.
So let's say...
#Qualities
$Name: Low
$Name: Medium
$Name: High
#End
#Images
$Name: fighter2t-01
+Low: low_fighter2t-01
+Medium: med_fighter2t-01
+High: hgh_fighter2t-01
$Name: cruiser01
+Low: low_cruiser01
+Medium: med_cruiser01
+High: hgh_cruiser01
#End
Replace the model names with the appropriate texture names.
That gives us a much more expandable + flexible quality system for mods or whatever, (ie a mods or materials section would be possible since it's just simple string replacement) and would really only require a command-line option and/or dropdown box in the upcoming options screen.
Although with mipmaps and LODs, it would probably be better if people just used the options screen to lower the values to fit their computer. :p
-
Good read and I agree with most of that. The biggest issues were actually with some of the effects a while ago, as there were a lot of unnecessarily large TGAs that could be made smaller and into PCX or DDS files without any loss in quality. Most of that stuff has been fixed now, but I think a few bloated effects may still remain.
However, I actually really like the 2048x2048 textures, provided that they are used judiciously and have the detail to justify their size, in which case they can make a remarkable difference when running the game in high resolutions. They shouldn't be in the standard textures package, but a separate adveffects style file sounds like a good idea. Large textures are something sorely lacking in most modern games and they are in my opinion one of FSO's biggest assets as far as the graphical quality is concerned. A detail texturing technique in FSO might make the largest textures unnecessary (I brought this up about two years ago and Bobboau said he had plans for it in the long term future), but for now they seem to be the way to go for high end systems.
On this point, one thing I think is worth mentioning is that these huge textures (and in general high polygon counts as well) should only be used for big capital ships. The new fighter models look great in the tech room, camera mode and so on, but you don't really notice their increased detail anywhere nearly as much as that of, say, a destroyer while actually playing the game.
Do the shinemap textures need to be the same size as the base texture? I'm wondering how much quality would be lost by making those smaller.
[q], anisotropic filtering level just became a registry option for instance[/q]
What's the flag? I would like to try this out.
-
However, I actually really like the 2048x2048 textures, provided that they are used judiciously and have the detail to justify their size, in which case they can make a remarkable difference when running the game in high resolutions.
Good point. That's correct. So it's for high-end PCs anyway. High-res maps will look a lot better with a higher screens res.
Do the shinemap textures need to be the same size as the base texture? I'm wondering how much quality would be lost by making those smaller.
NEVER use a lower res for the shine maps when you're texturing a usual ship. The optical result is weird. You could even say 'ugly'.
This only works in some special cases.
Example: Shadows of Lylat is using 3d landscapes with gigantic maps, but the shinemap is very small, cause we don't need it. (AFAIR 4*4 or 16*16).
In some other cases the shinemap is just white (glass or something like that). In this case you can also stick to a smaller shinemap.
-
The target box issue is something that should illustrate a point. If the model shown there is LOD1, then LOD1 should be made to offer minimal visual difference from the lowest LOD at the point when it is the same size on screen as it is in the target box.
Not necessarily. A good LODing offers a good performance/visual curve based on the size of the model. (Bigger = more detailed and generally *more* LODs).
Just because a LOD1 is not a good model for targeting box rendering doesn't make the LODing bad.
IMHO the best approach would be if it would be possible to set which LOD of the given model is to be used.
-
Actually I suggested a table. I didn't want to bring that in to the discussion, since it's not particularly relevant, but I hate being misquoted.
Yep. Sorry about that. I wanted to see if there was interest in either idea without going to much detail for either proposal but I did completely forget about the table.
And I do think that lowering the detail levels can help decide the top-level mipmap that gets used. I'd thought of that before but never really looked at what would be needed. Probably worth looking into in a few months since if everyone is going to finally start making use of mipmaps then we can drop detail levels with those and there won't be a real need to make high, low, and medium detail textures. You may have to remind me of that though, I'm far to forgetful.
IMHO the best approach would be if it would be possible to set which LOD of the given model is to be used.
Now that's a damn good idea. It could be added to the ships.tbl entry and if it's specified it will use that LOD number of the original model rather than the default. I'll see if I can't add that sometime next week, or WMCoolmon can do it if he has time.
[q], anisotropic filtering level just became a registry option for instance[/q]
What's the flag? I would like to try this out.
It's a string value, "OGL_AnisotroicFilter", but it only takes numbers (a float value). Minimum value is 1.0, which is to not enable the filter, max value is determined by the video card (will likely be 16.0 on current cards). It will cap the value if you go out of that range. Also, if using a debug build, you can change the setting in game by bringing up the debug console with "Shift-Enter" and using the "ogl_anisotropy" command. Use "help ogl_anisotropy" to get a general idea of it, "ogl_anisotropy" with no options will reset to the default/registry value, "ogl_anisotropy x" will set the filter to "x" value. The default value is 1.0.
The anisotropic filter did work before but not properly, and it was always set to the max.
-
taylor, what's your take on detail-boxes?
When and how should they be used? What's the efficient texture usage with them?
Omniscaper already made a huge impact with his Deathstar, however AFAIK few modellers truely know what they are and how to use them.
On a sidenote: Shinemaps are a lot more than 8-bit black and white images!
Almost none of our models (except the Rahu, Shivan gasminer) use them to their true potnetial.
Bobbau designed something that's way ahead it's time it can:
-Convey the 'texture' of the surface: rough, smooth, jagged, ect // Just apply the apropiate noise maps to the area
-Convey the shine 'color' of the surface. Once again brighting/darkening the main map can work, but there's so much more to this. Ever seen the Arcadia with the blue tint? ....brass, copper, lead, silver, gold, ruby, emerald....all are possible if you're willing to experiment with shine colors different from the main texture.
-
IMHO the best approach would be if it would be possible to set which LOD of the given model is to be used.
Now that's a damn good idea. It could be added to the ships.tbl entry and if it's specified it will use that LOD number of the original model rather than the default. I'll see if I can't add that sometime next week, or WMCoolmon can do it if he has time.
We already have the field $POF target file: that can point to an external file. Could we extend this to have the options LOD0, LOD1, LOD2, or LOD3 to force a certain LOD? I'd really like to see bombs use LOD0 since their target view is often bigger than the 'real' view, and there are some big and detailed models whose target view could use LOD2 instead of LOD1.
-
taylor, what's your take on detail-boxes?
When and how should they be used? What's the efficient texture usage with them?
Omniscaper already made a huge impact with his Deathstar, however AFAIK few modellers truely know what they are and how to use them.
Don't really have a take on them since I don't much about them either. The basic exturing rules should apply the same way though. How and when they should be used is a question better directed to someone who has used them before, or more appropriately, Bobboau.
On a sidenote: Shinemaps are a lot more than 8-bit black and white images!
<--snip-->
Agreed. And if they are going to be used I'd much rather have them really take advantage of what they can do. Just making things shiny is rather boring. I remember when Lightspeed was re-texturing the old models and just how awesome some of them were with the shinemaps (like the Rahu). If I'm going to waste memory on an extra set of maps then that's how I want to do it. :)
With nice ones like that is where you start to have the differing size problems that DaBrain hit on. That's just another situation where the content can dictate what the final textures are going to be. But even so, if using one set of textures (base/glow/shine) for multiple LODs you will very quickly and very easily save any extra memory that would have been used. And though I didn't really touch on this in my original post, if reusing textures for multiple LODs then LOD switching is not only faster it's also much more seamless since you won't notice the geometry change as much as a texture change.
We already have the field $POF target file: that can point to an external file. Could we extend this to have the options LOD0, LOD1, LOD2, or LOD3 to force a certain LOD? I'd really like to see bombs use LOD0 since their target view is often bigger than the 'real' view, and there are some big and detailed models whose target view could use LOD2 instead of LOD1.
Yeah, that's what I want to add/change. Basically there would be a $POF target LOD: and you would just give a LOD number. The targetbox will still use LOD1 by default but check this entry if found and update the model detail level before rendering. The would work instead of $POF target file: since if you created a separate file then you would probably only make one LOD anyway, but both options would still be supported. It would take about 10 minutes to add and make usable the new option but I don't really have 10 minutes for that today. :sigh:
-
Yeah, that's what I want to add/change. Basically there would be a $POF target LOD: and you would just give a LOD number. The targetbox will still use LOD1 by default but check this entry if found and update the model detail level before rendering. The would work instead of $POF target file: since if you created a separate file then you would probably only make one LOD anyway, but both options would still be supported. It would take about 10 minutes to add and make usable the new option but I don't really have 10 minutes for that today. :sigh:
Ok, it's in CVS now. In ships.tbl $POF target LOD: goes before $Detail distance:. In weapons.tbl it goes before $External Model File:. Values are 0 to whatever number of LODs the POF has. If you use a LOD number higher than is in the POF then it will get capped to the next lowest LOD number. This will specify what LOD is used to render the ship (the ships.tbl entry) in for the targetbox, what LOD to use for a weapon (the weapons.tbl entry), and if using weapon view then it determines the LOD of the ship (the ships.tbl entry again) that the bomb/missile is seeking.
And looking at the code I did find where it's using an offset for the LOD that gets shown in both the lab and the targetbox. I'm not sure why this change was made, but based on the comment that person didn't really understand/consider what's going on there. So, now the lab and the targetbox will really use the correct LOD. It was down by one so LOD0 was trying to use -1 and getting set to 0, LOD1 was trying to use 0, LOD2 was 1, etc. Basically every LOD shown was actually one higher than it should have been in those two places (but not the normal game). Galemp called it and I didn't believe it at first, but I've now proven myself wrong. :) That incorrect LOD handling change with locked detail levels was made 2 years ago btw.
This should show up in the next available CVS build.
-
I'd been wondering why that happened. I suspected it was to force LOD0 models to get rendered on the HUD, which would have made sense 2 years ago before all of the HTL models started appearing, but I'm glad it's fixed.
-
I'd been wondering why that happened. I suspected it was to force LOD0 models to get rendered on the HUD, which would have made sense 2 years ago before all of the HTL models started appearing, but I'm glad it's fixed.
It was done to (according to the comment) stop swapping LOD textures in and out of memory. The problem with that is they are all loaded already and though they might have to go in and out of API memory again, any possible texture speed increase is completely reversed by the extra cycles and memory needed to render a higher poly count and higher texture quality model. What the change was trying to prevent just went against how the code actually works. The proper fix to get LOD0 models to render on the HUD is just a very small, one line change to the targetbox code, though that wasn't what was done.
But it is correctly fixed now and you will get the option of using any LOD you wish for the targetbox on a per ship basis. Plus it doesn't have the negative impact on the lab functionality (ie. you never got to see the final LOD since it used the one before it) that the previous change was making.
-
Hallelujah!
-
And looking at the code I did find where it's using an offset for the LOD that gets shown in both the lab and the targetbox. I'm not sure why this change was made, but based on the comment that person didn't really understand/consider what's going on there. So, now the lab and the targetbox will really use the correct LOD. It was down by one so LOD0 was trying to use -1 and getting set to 0, LOD1 was trying to use 0, LOD2 was 1, etc. Basically every LOD shown was actually one higher than it should have been in those two places (but not the normal game). Galemp called it and I didn't believe it at first, but I've now proven myself wrong. :) That incorrect LOD handling change with locked detail levels was made 2 years ago btw.
w00t :)
-
I'd also like to know more about when to use detail boxes.
On my ursa i'm thinking of making a single detail box that contains all the grebles for ultra-close-in viewing, wich would could the model overall would use one less lod because otherwise lod 1 and lod 2 would be the same except with and without grebles.
-
There's no difference graphically between that and an extra LOD (I'm not sure that detail boxes are useful on fighters, except maybe for pilot models) and you actually hurt performance when you're up close because then there's another submodel to render and another submodel rendering call that has to be made. Detail boxes make sense for detailing parts of a larger ship where only one or two particular parts will be visible close enough for any scrutiny at any given time; for a fighter (and using a single box) it's not the same win.
-
Thanks for the advice. However I am unable to save in DDS for some odd reason. I have that Nvidia thing to use it but when I go to the save menu the save button is greyed out.
-
Thanks for the advice. However I am unable to save in DDS for some odd reason. I have that Nvidia thing to use it but when I go to the save menu the save button is greyed out.
You mean you are using the Photoshop plugin or something else? Is the image power-of-2 and 24 or 32-bit and a supported read format (BMP, TGA, TIF, plus a couple of others, may be a restriction to save)?
-
Yes its a photoshop plugin. I'm not sure about power of 2. Its a 32 bit TGA 800x800 texture.
-
Try it as 1024x1024 or 512x512 and see if it works.
-
I keep waiting for CVS to get working again before I post this but I guess I'll go ahead and give everyone the heads up...
Full support for non-compressed DDS mipmaps will hit CVS as soon as the server starts working again. The compressed DDS mipmap handling has been tweaked a little as well. Also hitting CVS at the same time is support for bilinear and trilinear filtering (since those two things work with mipmaps). Just add "TextureFilter=1" to the registry (for Windows, a DWORD value) or ini file (for Linux and OSX) to enable trilinear filtering. The default ("TextureFilter=0") is bilinear whenever an image has mipmap levels. These settings don't have any affect unless the texture is properly mipmaped though so this is just some enticement for the map makers out there. ;)
This also makes the recent anisotropic filter update for OGL actually work worth a crap now, since it's basically the quality of mipmap filtering. Anisotropic filtering will sharpen up textures that have been hit by bilinear or trilinear filtering. For those that didn't catch that registry setting before, it's "OGL_AnisotropicFilter", which is a string value that is set from "1.0" to what ever your video card can handle. The max value is capped to what the video card supports to feel free to do "99.0" or something to always get the max value. Note that "1.0" (the default value) actually turns the anisotropic filter off so you need a value of at least "2.0" to enable it.
All of this is OGL only for the moment of course, but eventually someone will modify the D3D code to have the same support.
-
Kudos, I think it was about time somebody tackled this subject.
I really love some of the artwork that you guys create for Freespace but every time I turn my fighter and bring a new Orion or a science vessel (I think this has since been resolved though) into view my framerate takes a massive hit. I consider my computer to be fairly powerful....
I only have one thing to say: Fenris/Leviathan.
-
freespace wont use textures bigger than 2048^2? why do i just find this out now? the biggest texture ive done so far would be the 4096^2 texture i did for the ragnarok/valhalla class corvettes and the planned carrier varient as well. yes the texture is huge but the 3 ships all use the same texture and only one that texture, where the old versions used 4 textures. the ships are ~ 750 meters long and the textures look to damn good and if there getting redced to 1/4 the pixel count they once had then thats pretty good texturing. ive been giving many ships in nukemod the htl makeover, doing texture merges and so on also while adding detail and cockpits to the models. ive been using the rules that were established by bob way back when htl was just implemented, which were esentally use fewer textures (my mod now uses 50% fewer textures), and no mipmaps (as back then freespace didnt use them and generated their own), and the others regaurding lods which i still havent made yet (i plan to, eventually, when im really bored). i like to use big textures for future proofing my models, just cause the game cant handel a certain texture doesnt mean you shouldnt make it that big, just so long as you release with a smaller version. originally i was going to offer the gargantuan textures as a seprate download but sence freespace doesnt support them i might as well forgo the hassel. your info is certainly helpfull and cleared up a few questions i had regaurding mipmaping and lod textures. the only thing that concerns me is that the engine isnt using the full texture for the ragnarok. i figured my 256 meg video card would handel the 8 meg texture just fine (thats how big a 4096^2 dxt1 is). i didnt know there was a cap in the software.
-
freespace wont use textures bigger than 2048^2? why do i just find this out now?
It has nothing to do with Freespace, it's a video card thing. And it's not that it won't use larger than 2048, it's that any texture larger will automatically get resized to 2048 by the video card before using it. Which basically means that you have forced a 4096 on everyone and you get nothing for it but wasted memory and slower performance. Most video cards have a max texture size of 2048x2048 and anything larger than that is just wasting everyones time. My video card supports 4096x4096 but I will banish you to the deepest depths of Hell if you use a 4096 for anything but huge installations, and even then it would be accepted under protest since 99% of the time that the texture is on screen it's being minimized and distorted.
just cause the game cant handel a certain texture doesnt mean you shouldnt make it that big
Just because the ocean lies at the base of that 300 meter cliff doesn't mean that you should jump off the damn thing and go for a swim.
As I have said already it's about creating efficient textures. That doesn't necessarily mean fewer textures. You want to try and use fewer textures per ship but you are seriously misunderstanding that statement if making one massive texture and using it for multiple ships is how you do it. Using fewer textures overall doesn't help anyone if the textures just end up being larger and the video card has to do more work to use them. Fewer textures only means reducing the number of textures a single ship must be rendered with.
-
I dion't really mind as long as it's not causing me significant slowdowns.
-
i think you interpreted what i said as a complaint rather than a question. my video card is a 256 meg card, and it can handel theese big textures, so does that mean its getting resized anyway? i want to know so i can make a comparison to see for my self what would be better. i have no intention of releasing the mod with that big texture, mainly because i cant expect everyone to have as much texture memory as i do. 8 megs is alot, and thats not including shine and glow maps. and on a 64 meg card it will be completely unplayable. recently i made a bunch of new effects and i was contemplating using tgas cause they looked slightly better than dxt1s. bat after compairing the tga files with the dxt1s i dumped the tgas. it was too much a cost for such little gain. my reasoning for making big textures is that i might want to use the model for something else down the line. i can imagine several years from now people will comment on how low res these 2048^2 maps are. that is why i make them that big. seeing as the textures are rendered in photoshop from literally hundreds of layers (now hats a big image file, half a gig), it doesnt cost any detail to resize them in photoshop. im sure [v] did there maps at a much bigger scale, and then scaled them down for the final product.
-
my video card is a 256 meg card, and it can handel theese big textures, so does that mean its getting resized anyway?
Memory has nothing to do with it. You can have a terabyte of memory on your video card but it still might have a 1024x1024 or 2048x2048 max texture size limit. The limit is just what the video card can actually process and going over it either won't work or it gets resized by the API to fit within the limit. Just run a debug build and in the graphics init part of the log it will say what the max size is for your card. The higher end cards now will do 4096x4096, but most still do 2048x2048.
-
yep my card does support that resolution. so i guess that my setup can handel that texture. i bought this card specifically for texture work. however i cant expect everyone to have it. so ive made the decison to distribute the mod with scaled down textures.
another thing. im currently working on an explosion animation. its gonna be an eff with 15 frames, 256^2 dxt1 with mips. now from what you said the mips should be all the loding this explosion needs. is this a correct assumption?
-
another thing. im currently working on an explosion animation. its gonna be an eff with 15 frames, 256^2 dxt1 with mips. now from what you said the mips should be all the loding this explosion needs. is this a correct assumption?
Correct. It's also a good idea to include a modular weapon_expl table (or an entry in weapon_expl itself) to say that it has only the 1 LOD, just to avoid the stupid warning messages. The graphics system will take care of using the mipmaps as LODs automatically.
Oh and unless the explosion animation is really small I'd recommend making it 512^2 instead. The mipmaps will keep it from being any slower to use than a 256^2 version and you'll get the added detail when it has to magnify the image. There isn't much point in going over 512^2 since it is an animation and that is memory probably best used on shockwaves or texture maps or something. There is also the benefit of user controlled detail settings now. The "3D Hardware Textures" slider will now affect base (the most detail) mipmap levels, so if the user can't spare room for the 512^2 plus full mipmaps they can lower the slider and only get a max size of 256^2 or 128^2, etc. for the effect in video memory. This would allow you to create a detailed effect without having to worry too much about what the user will have to deal with since they can control it.
And that also brings up the point of making sure that you do your testing with that detail slider maxed out. Otherwise you aren't seeing the full detail possible from the map/effect. And this is just with OpenGL for now of course, D3D is still behind on the better mipmap support (used to be the other way around). D3D support is coming though, once I figure out how to do the same things in it that OGL does.
-
that animation came out to about 2.5 megs (i went 512 btw), which iis about the same as an average ship texture. i only need to make one other short explosion so im not too worried about memory usage. i have about 4 other anis to do but they are just for various particle effects and wont need very much detali. thanks for the advice.
-
'nother question:
I need to texture a LOT of polys on a rather over-detailed ship over on SG (here (http://www.sectorgame.com/forums/viewtopic.php?t=2125)), and I intend to use 1 or 2 large 2048² res maps. However, only small parts of it need glowmaps, so there'd be no sense in making a huge glowmap just so those bits can glow.
My question is, would I be better off putting those glowing areas onto a separate, roughly 256 res texture, and not putting a glowmap on the big one(s), or is the performance cost of switching textures great enough that that'd be slower than having a big glowmap (and thus glow map render pass) on the main texture?
Thanks in advance. :)
-
It really depends on how many of the 256 res textures you need to use. The fewer the better, but with that said, you could probably get away with using 4 or 5 of them before you will start to hit any sort of speed issue that the transfer limit of a 2048² texture could impose. Also keep this in mind, if not everything in the mission can fit into video RAM then it would be quite a bit easier/faster to swap in/out those 256 res textures than it would be to handle that 2048² one. Having to use a couple of 256² textures, even with texture switching, would greatly make up for the speed decrease and extra memory usage imposed by that one 2048² texture. For those of us with the hardware to handle the 2048² without flinching, the little extra texture work for the 256² versions wouldn't mean squat in the grand scheme of things. But those without that beefy hardware could benefit nicely from the 256².
And the main texture and the glowmap are rendered in the same pass, btw.
-
I'd be interested in a few clarifications if you have time. I've read the thread a few times and feel like I'm still missing a few concepts.
I get the idea that DDS is more efficient than everything else. And you say TGA is to be avoided as possible. But how does TGA compare to JPG and PCX? According to the bits per pixel thing, am I right in thinking PCX is better than JPG, which is better than TGA? (provided, of course, that color pallete is not an issue.) And that was for memory usage; is performance any different?
Hmm, that makes me curious, why is the PCX support only 8 bit? I'm not complaining, just curious.
I see a couple mentions of 'uncompressed DDS' but cannot find how to make this. Or rather, upon searching I think I can with nvdxt but am not sure what format would be best for FSOpen. u888 perhaps? What do you recommend for options/switches? But if this method no longer has the problems noted at http://www.penguinbomb.com/lightspeed/TGADDS.html, then that sounds like the thing to use for everything for which the compressed DDS is unacceptable! :yes:
-
I get the idea that DDS is more efficient than everything else. And you say TGA is to be avoided as possible. But how does TGA compare to JPG and PCX? According to the bits per pixel thing, am I right in thinking PCX is better than JPG, which is better than TGA? (provided, of course, that color pallete is not an issue.) And that was for memory usage; is performance any different?
PCX is 8-bit, but the game converts it to 16-bit for it's use in order to blend better and get transparency. PCX always has a 256-color palette (not in the format, just for what the game supports) so you will always only have 256 colors to work with. PCX is a very old format and so is better suited to the hardware it was written for, hardware that only used 16 or 256 colors and where disk space was a premium. PCX is slower than all of the other support formats in FSO because it's converted out of it's native format in order for it to be used better by the game. A PCX texture uses width x height x 2 worth of memory.
TGA is 8-bit, 16-bit, 24-bit or 32-bit depending on what you saved it as. 8-bit TGA isn't supported by the game so you can safely ignore that one. 16-bit is very uncommon so you typically won't see it, but the game supports it. 24-bit is an RGB image with no alpha and is pretty common. 32-bit is RGBA image, having an alpha channel. A TGA texture uses width x height x (Bpp/8) worth of memory, where "Bpp" is 8, 16, 24, 32 corresponding to what format the file is.
JPG is always 24-bit. It's compressed as a file format, but when it actually get's used it has to be uncompressed. A JPG texture uses width x height x 3 worth of memory, the same as a 24-bit TGA. Compression degrades image quality though and so JPG really only saves you disk space and nothing else.
I see a couple mentions of 'uncompressed DDS' but cannot find how to make this. Or rather, upon searching I think I can with nvdxt but am not sure what format would be best for FSOpen. u888 perhaps? What do you recommend for options/switches? But if this method no longer has the problems noted at http://www.penguinbomb.com/lightspeed/TGADDS.html, then that sounds like the thing to use for everything for which the compressed DDS is unacceptable! :yes:
Pretty much anything that supports saving DDS images should also be able to save uncompressed DDS. Currently supported uncompressed DDS formats are u1555, u888 and u8888. "u1555" is a 16-bit format with 1-byte alpha, this is what PCX images are turned into by the game engine. "u888" is a 24-bit RGB image, bascially the exact same thing as a 24-bit TGA image. "u8888" is a 32-bit RGBA image offering a full alpha channel, basically the exact same thing as a 32-bit TGA image. As you can guess from "uncompressed", it doesn't save any memory to use these formats, and the memory needed for the image is the same as the size of the file (ie, a 2meg file will need 2meg of both system RAM and video RAM). The only really advantage to uncompressed DDS is that you can have a full set of mipmaps in the one file, and you can make cube maps (used for environment mapping) out of them. It's not going to save any memory or disk space.
Always try to use a compressed DDS format (DXT1, DXT3, DXT5) whenever possible since it saves a ton of memory. For instance, a DXT1 image will use 1/6th of the memory that a 24-bit TGA or u888 image uses. The compression artifacts are an issue so just test it and see if it's acceptable or not. If not then use one of the uncompressed formats. Compressed DDS is the fastest of the formats used by FSO since it requires no additional processing (like everything else does), it's sent to the video card as is and the video card uses it the same way.
-
It really depends on how many of the 256 res textures you need to use. The fewer the better, but with that said, you could probably get away with using 4 or 5 of them before you will start to hit any sort of speed issue that the transfer limit of a 2048² texture could impose. Also keep this in mind, if not everything in the mission can fit into video RAM then it would be quite a bit easier/faster to swap in/out those 256 res textures than it would be to handle that 2048² one. Having to use a couple of 256² textures, even with texture switching, would greatly make up for the speed decrease and extra memory usage imposed by that one 2048² texture. For those of us with the hardware to handle the 2048² without flinching, the little extra texture work for the 256² versions wouldn't mean squat in the grand scheme of things. But those without that beefy hardware could benefit nicely from the 256².
And the main texture and the glowmap are rendered in the same pass, btw.
Ah ok - that's excellent. Thanks muchly. :)
-
Very clear, complete, and helpful answers. Thanks. :yes:
Steel01 and I are doing some testing on how much performance is gained by turning just about everything in the mediavps to DDS. For some of the effects in particular, IMO, compression may not be a good idea -- because of, as you say, the compression artifacts. So while uncompressed doesn't save memory, still since there's no additional processing there's some performance gain, correct?
Here comes a few questions about effects and mipmaps that don't really have to do with 'creating efficient models', so split me off into a separate thread if you prefer.
How many mipmaps would you say are necessary for large background images, like nebulas and planets? They are big and don't get resized often (only once per mission, right?), so maybe 2? 1? 0? (How would the 3D Hardware Textures slider affect this, being backgrounds?) I notice the DDS versions of the planets in mv_effects have up to 11, is this a good idea?
How about stuff like lasers, thrusters, and explosions? Those get resized a lot. Of course, some start out pretty small to begin with. I notice most of the current EFF-animated DDS series of laser effects have no mipmaps at all. Meanwhile the WarpMaps, a 512x512 EFF-animated DDS series, have 9. I like that with the mipmapped explosions we can eliminate the LOD files, but do I follow the (exp4.ani 256x256) example and have 2 or so mipmaps, or would more be better?
I'm guessing for some things there are no concrete answers, but it will be nice to hear some general guidelines. Thanks for putting up with my questions ;)
-
So while uncompressed doesn't save memory, still since there's no additional processing there's some performance gain, correct?
There's not really any extra processing going on during load for a uncompressed DDS, but the video card does have to do more work. An uncompressed 32-bit DDS with a size of 512x512 is a total size of 1meg. A compressed 32-bit DDS with a size of 512x512 is a total size of 256k. It's going to take more time for the video to get the image, manipulate the image, and use the image if it's uncompressed. In that case it doesn't matter if it's uncompressed DDS or not. I'm more willing to deal with the compression artifacts than the speed decrease in most instances.
But that's why we have -img2dds (when it actually works for everyone). Then you can have a 32-bit TGA image (uncompressed DDS isn't affected by -img2dds!!) without any sort of compression artifacts and let the user decide if having that image compressed is the best option or not.
So while uncompressed doesn't save memory, still since there's no additional processing there's some performance gain, correct?
How many mipmaps would you say are necessary for large background images, like nebulas and planets? They are big and don't get resized often (only once per mission, right?), so maybe 2? 1? 0? (How would the 3D Hardware Textures slider affect this, being backgrounds?) I notice the DDS versions of the planets in mv_effects have up to 11, is this a good idea?
This is actually a loaded question so try and stick with on it, it may not make much sense...
A mipmaped file is one that contains various sizes of a particular image. So, when you use a 1024x1024 image of a planet, with full mipmaps, you are sending to the video card a 1024x1024 image, plus a 512x512 version, a 256x256 version, a 128x128 version, a 64x64 version, a 32x32 version, a 16x16 version, a 8x8 version, a 4x4 version, a 2x2 version, and a 1x1 version. 11 mipmaps == 11 images. For that planet you could probably just get away with the first 3 (1024x1024 -> 256x256) and let the video card just resize the smallest version itself. Of course you could just not use mipmaps for those at all. The mipmaped version will be slightly faster to render, since it will just use the version that best matches the size in use, but then you are wasting all of the other mipmap levels which are also loaded into memory. So while you will get an increase in rendering speed, and probably an increase in quality, you are using more memory to do it. You aren't wasting much memory on the low side (the size of all mipmap levels, below 512x512, *combined* are 1/3rd the size of just the 512x512 level in memory) but you do waste memory on the high side (if you only need the 256x256 version then are wasting at least 75% of the memory used by the image). In other words, this part is tricky. ;)
Eventually I plan to fix this so that the code will only try to use the mipmap level that best matches the size of the starfield bitmap (suns, planets, nebula) in use. This is possible, just not a high priority. When that gets fixed though you can do whatever you want with the mipmap levels and the game will go with what works best automatically.
If you want an answer on what to do then I can't really give it to you. For now it's totally a judgement call based on the image. My basic opinion is (if using a 1024x1024 image) to make at least 3 mipmap levels, just to try and help people out rendering speed wise, but making the full set of mipmaps is only going to use another 21k of memory so why bother not doing it.
How about stuff like lasers, thrusters, and explosions? Those get resized a lot. Of course, some start out pretty small to begin with. I notice most of the current EFF-animated DDS series of laser effects have no mipmaps at all. Meanwhile the WarpMaps, a 512x512 EFF-animated DDS series, have 9. I like that with the mipmapped explosions we can eliminate the LOD files, but do I follow the (exp4.ani 256x256) example and have 2 or so mipmaps, or would more be better?
I don't think lasers matter much, but I mipmaped mine and really like the result. It just looks nicer in my opinion. Thrusters and glowpoints, these are considered part of the model so they should have full mipmaps like any other ship texture.
Explosions and warpmaps are two things that can benefit heavily from mipmaps. I say that it's basically a crime at this point to not use mipmaps for those. Of course you have to tell the game (via a weapon_expl.tbl, or the XMT version) to only use the one LOD image in order to avoid also mixing in retail effects. For both of these types you aren't really going to waste memory by making a full set of mipmaps since nearly every level will end up being used by the game. This certainly falls into the "more is better" category, do the full set and be happy knowing that you are doing everyone a favor.
-
Me again :D Thanks for your answers... even the opinions on the questions that can't have clear answers yet, they were what I needed to know. And I understood the one that might not have made sense ;)
But that's why we have -img2dds (when it actually works for everyone). Then you can have a 32-bit TGA image (uncompressed DDS isn't affected by -img2dds!!) without any sort of compression artifacts and let the user decide if having that image compressed is the best option or not.
This sounds awesome, when it works. Might part of the reason it has troubles be because of some textures that are not power-of-two?
Eventually I plan to fix this so that the code will only try to use the mipmap level that best matches the size of the starfield bitmap (suns, planets, nebula) in use. This is possible, just not a high priority. When that gets fixed though you can do whatever you want with the mipmap levels and the game will go with what works best automatically.
I understand why it's not a high priority, but it sounds great for someday :)
My basic opinion is (if using a 1024x1024 image) to make at least 3 mipmap levels, just to try and help people out rendering speed wise, but making the full set of mipmaps is only going to use another 21k of memory so why bother not doing it.
Heh, I didn't think about that all the way through. Makes sense now. Full mipmaps it is, then.
Now I have a question about alpha channels. This came up while messing with DXT1/3 vs DXT5. Basically, are there any effects that NEED the alpha channel? (so I should leave them as DXT5 rather than DXT1 or 3?) I know Freespace does a lot of automatic transparency with most of the effects -- guns, explosions, thrusters, etc -- so in which cases is the alpha channel actually useful? (I'm guessing just the suns/sunglows, planets and nebulae?)
Minor thing: in an EFF animated series of DDS files, are there any issues with having some frames be one format (DXT1) and some be another (u1555), in the same animation?
Also, is run-length-encoding supported in the game's use of TGAs? I think I found a couple files in the mediavps compressed this way... would it be better to resave them as uncompressed, or leave them [and perhaps convert others to] compressed? (I'm guessing there's a tradeoff between memory and performance... do you have opinion on which is worse in this case?) Of course, moving to DDS, hopefully soon this point will be moot ;)
-
This sounds awesome, when it works. Might part of the reason it has troubles be because of some textures that are not power-of-two?
Actually I think I fixed the problem the other night. Just waiting for the next CVS build to know for sure. And power-of-two isn't an issue here, it tests first to make sure that the image is good for compression and a power-of-two test is part of that. If the texture isn't power-of-two then it just stays as it is, there is plenty of fail-over to prevent things like this from making you see no image. The problem here was that it thought that the image data was reversed for 32-bit images (making the alpha appear to be a primary color). Just a minor little thing that took me forever to finally notice. :)
Now I have a question about alpha channels. This came up while messing with DXT1/3 vs DXT5. Basically, are there any effects that NEED the alpha channel? (so I should leave them as DXT5 rather than DXT1 or 3?) I know Freespace does a lot of automatic transparency with most of the effects -- guns, explosions, thrusters, etc -- so in which cases is the alpha channel actually useful? (I'm guessing just the suns/sunglows, planets and nebulae?)
The game uses two basic forms of transparency: alpha, and alphablend. The alphablend images are the ones that use black to depict alpha color, these images can/should be DXT1 as they don't actually use a real alpha channel. Starfield bitmaps (suns and backgrounds) and most effects (explosions, lasers, etc.) are the best case of using this type of blending. The other form is, of course, a true alpha channel in the image. This is good for some ship textures (if you need alpha, like for glass or something) and also for planets (so that you can have a dark side to the planet without it also being transparent).
You only really need an alpha channel if you actually need it. A good point of reference is that if you are upgrading an effect then look at the old one. If it's using a black background then it's an alphablended image and you shouldn't need an alpha channel.
Minor thing: in an EFF animated series of DDS files, are there any issues with having some frames be one format (DXT1) and some be another (u1555), in the same animation?
Nope. They only have to be the same file type (DDS, TGA, JPG, etc.) and of the same dimensions. You can easily mix DDS formats for the different frames in an EFF, that was part of the point in fact. Each image is loaded separately by the game so each one can be a different basic format. And that can actually be different dimensions as well, but then it doesn't render properly (just in case you try it :)). I had originally wanted it to be possible to only use the size image that you needed for the frame content, but that didn't work out too well so I scrapped the idea.
Also, is run-length-encoding supported in the game's use of TGAs? I think I found a couple files in the mediavps compressed this way... would it be better to resave them as uncompressed, or leave them [and perhaps convert others to] compressed? (I'm guessing there's a tradeoff between memory and performance... do you have opinion on which is worse in this case?) Of course, moving to DDS, hopefully soon this point will be moot ;)
Yes, RLE compressed TGAs are supported. RLE is a lossless compression so the image will look the same whether it's compressed or not, but being compressed it saves on disk space. RLE compressed TGAs will use the same amount of memory and they don't load any slower (perhaps a little faster actually) so there is no harm in using them over uncompressed TGA. I have converted nearly all of the TGAs I use to DDS, but some of them can just look too bad when compressed to a DXT format. At that point it's better left up to the end-user to decide, let it use more memory for better quality, or use -img2dds to compress the images. The only real loss here is with mipmaps since they aren't created for such images (yet, that will get fixed at some point).
-
Just re-read this one, and I've been trying to change some of my PCX backgrounds to DDS using the nVidia plugin for Photoshop...
However the game doesn't show the backgrounds anymore, I've tried a few different settings but no joy. Is it something simple I'm missing, or do the backgrounds need to be .pcx?
-
IIRC Photoshop dds plugin is very buggy and does odd things time to time... you are much better of with either ATI's Compressonator or Nvidia's command line tool (Nvdxt or something).
Backgrounds work fine as dds. Just make sure you are using proper dds type (for backgrounds either u888 or DXT1 is probably the best) so that you dont get problems from bad alpha.
-
ahh - I sussed it. The image dimensions need to be a multiple of 2 (2,4,8,16,32,64,128,256,512,1024 etc etc) :)
-
Yeah, but AFAIK only for compressed dds files - for FreeSpace related graphics that equals either DXT1 or DXT5
-
Yeah, but AFAIK only for compressed dds files - for FreeSpace related graphics that equals either DXT1 or DXT5
I was under the impression that every graphic fils whould be a power of 2, otherwise it must be resized by the graphics card, wasting gpu cycles.....?
-
Yeah, but AFAIK only for compressed dds files - for FreeSpace related graphics that equals either DXT1 or DXT5
I was under the impression that every graphic fils whould be a power of 2, otherwise it must be resized by the graphics card, wasting gpu cycles.....?
It's resized by the game, not the graphics card, and it only has to be done when it's initially sent to the video card in the first place. But, for texture maps and effects, it's still better to have everything as power-of-2. Interface graphics don't have to be power-of-2, and the game will try to use them that way if possible. That only applies to interface graphics though and everything else will have to be power-of-2, whether the image was originally or the game makes it that way, unless you have a GeForce 6800+ video card that can actually use non-power-of-2 images natively.
The issue with compressed DDS is that they won't even load if they aren't power-of-2. If compressed DDS images aren't power-of-2 then it will fail to load them and you'll just get blank/missing texture related issues (graphical errors, warnings, etc.).
-
Good to know.
-
I hate bumping this thread for something so stupid, but I don't want to start a new thread to announce this, and it is sort of relevant...
I've been working on improved texture replacement functionality. It originally started out as a performance tweak, but since I had always wanted to fix the short-comings with that code, I just took it all the way. The current texture replacement functionality had serveral issues: 1) it only worked for the base maps, it couldn't change glow/spec/normal maps at all, 2) it wasn't as efficient as it could have been, 3) it's application was limited to per-mission use.
Issues 1 & 2 were addressed in the same manner. The code now keeps track of the full set of textures, a combination of the originals and any replacements, and can quickly and easily pass those around to be rendered. The same texture loading function works for these extra texture sets, so it will easily handle a new base map and automatically load up the respective glow/spec/normal map to go with it. This also works with "-amb" and "-trans" textures, allowing you to quickly and easily get the same level of control that you would have by editing the POF itself.
Issue 3 was solved by adding an extra level of texture replacement, through ships.tbl. So say that you have the Fenris and Leviathan, both of which are the same model, but the hi-poly versions are different POFs in order to get different textures. With texture replacement you can use the single POF and just change out the textures by default every time the ship is used. This works in mission as well as in the lab and techroom. It's essentially the same as using two POFs, except you save a good bit of system and video card memory by not having to load up an extra POF. The normal mission specific texture replacement works on top of this, so you still have full control of how the model will be presented in-mission.
The ships.tbl level texture replacement is mainly just to help out mods which already have multiple tbl entries, with the same basic POF, just to get a different set of default textures. The new texture replacement code will now allow you to just have the one POF, and use texture replacement to change the default textures for those ships. I already use this myself for the Fenris, Leviathan and the Cain, Lilith ships, allowing the use of the same POF but still getting the different texture sets that we are all used to. The texture replacement usage is the same in ships.tbl as in missions. So just make use of the same "$Texture Replace:" setup than you are used to, putting it after the POF filename specification in the ship tbl entry.
This new code will start showing up in builds starting next week, and will be included with 3.6.10. Hopefully it will be of some help to various modders and model makers out there.
-
Oooh, that sounds pretty cool. :)
Couple of questions:
1) Can/will we be able to change textures mid-mission via SEXP?
I'm under the impression we can't yet, but I reckon this would be the best way to handle custom cloaking devices and the like. :)
2) If you only wanted to change from one glowmap to another, would that be possible without creating a second base texture to switch to?
I've had an idea running around my head for a while to try and give some ships a cooler death sequence. I made an experimental render-vid in 3ds Max a while back which gets the idea across well enough: http://sectorgame.com/ti-file-dump/VasudanAdmiral/Videos/LightFlickker.avi
Obviously mid-mission texture replacement isn't the proper way to go about it if it was meant as a game-wide kinda feature, but it's a step in the right direction and would allow us to test it in-game. :)
3) I don't think I've heard of the "-amb" or "-trans" tags before - how do they work?
4) So what would the table entry actually be?
At a total guess, "$Texture Replace: (Fenris-HTL, Levvy-HTL)" ?
-
1) Can/will we be able to change textures mid-mission via SEXP?
It can't change via SEXP at the moment, but the new replacement framework does make such a thing possible. And it wouldn't be that much work to code it in either. I'm probably not going to do that myself unless there is a big enough push for it from enough people, but I suspect that Goober or karajorma could take care of it pretty easily with just a quick chat with me about how all of the new code works.
2) If you only wanted to change from one glowmap to another, would that be possible without creating a second base texture to switch to?
No, it's linked to the base filename, so you would have to change the base texture as well (and spec, and normal/bump). That sucks, I know, but there are probably better ways we could add in the effect that you want anyway. Something like a "death glow", which could either be another map entirely, or just a normal glow map which is animated but only shows one frame unless the ship is dying. I'm not really sure though, this is really something that we could/should probably add a new sort of effect system for. Then it could take care of death/birth animations and all sorts of in-between stuff like cloaking. Something to add to the todo list I guess. :)
3) I don't think I've heard of the "-amb" or "-trans" tags before - how do they work?
"-trans" will render a map as additive blended. The same way that the game blends most effects (where black == transparent) it could also render that particular model texture. It has limited uses, and I don't pretend to really know what all could be done with it, but I'm sure that some people could do some creative things. I know that DaBrain is using it for at least one ship texture for SoL. It that case it allows for a transparent glow that will blend properly with anything behind and in front of the model/texture.
As far as "-amb" goes, it's basically the same as rendering without lighting, but per texture rather than for the model as a whole. I'm not sure that the option actually does anything different in HTL mode though. But I don't think that there is a reason for that other than the fact that support for it has just never been added to the HTL render path.
4) So what would the table entry actually be?
For the Levy, it would just be something as simple as this:
$POF file: cruiser01.pof
$Texture Replace:
+old: fenris-htl
+new: levy-htl
Here is a the tbm that I've been using for the Levy and Lilith: mv_models-shp.tbm (http://icculus.org/~taylor/fso/misc/mv_models-shp.tbm)
Obviously you need a new build, which isn't out yet, for that tbm to work, but it should give you an idea of what changes would be needed.
-
It can't change via SEXP at the moment, but the new replacement framework does make such a thing possible. And it wouldn't be that much work to code it in either. I'm probably not going to do that myself unless there is a big enough push for it from enough people, but I suspect that Goober or karajorma could take care of it pretty easily with just a quick chat with me about how all of the new code works.
That's great news. :)
I can tell you for sure that TI would use it a lot. In fact I have a number of fairly detailed special effect sequences or tricks I typed up a year ago when I thought mid-mission texture replacement was possible. :nervous:
Things like force-field effects (that need to be easily and visibly switched on or off), turret model glowmaps that light up when the turret is preparing to fire, cloak effects that have a particular look (ie, you define how they look come on or off via a short animated texture rather than using a pre-defined cloak function), creating big damage effects in special effect sequences, changing glowmap effects when a ship goes into attack mode, and a couple of others I've probably forgotten.
Believe me, it'd be a very powerful little feature. :D
No, it's linked to the base filename, so you would have to change the base texture as well (and spec, and normal/bump). That sucks, I know, but there are probably better ways we could add in the effect that you want anyway. Something like a "death glow", which could either be another map entirely, or just a normal glow map which is animated but only shows one frame unless the ship is dying. I'm not really sure though, this is really something that we could/should probably add a new sort of effect system for. Then it could take care of death/birth animations and all sorts of in-between stuff like cloaking. Something to add to the todo list I guess. :)
Something like that was what I was thinking would be perfect if it were to be a game-wide feature rather than just the 'idea' it is now. ;)
That said however, I don't think an entire new effect system is the only way to do this sorta stuff. I mean, if we could change textures mid-mission, I think this would already be possible to do via the scripting system - same sorta thing as Wanderer's awesome little flashy deaths script, except set it to switch ships over to their death animation glowmaps if they have them as they're beginning to go pop.
"-trans" will render a map as additive blended. The same way that the game blends most effects (where black == transparent) it could also render that particular model texture. It has limited uses, and I don't pretend to really know what all could be done with it, but I'm sure that some people could do some creative things. I know that DaBrain is using it for at least one ship texture for SoL. It that case it allows for a transparent glow that will blend properly with anything behind and in front of the model/texture.
Ah - another cool feature that snuck past me. :(
So is this just a tag you add to the base map or is it a separate texture? Also, is the end result much different from setting a base texture to be transparent via the alpha channel? I'm a bit unclear there.
Thanks again for all these answers btw. Really appreciate it. :)
-
Something like that was what I was thinking would be perfect if it were to be a game-wide feature rather than just the 'idea' it is now. ;)
That said however, I don't think an entire new effect system is the only way to do this sorta stuff. I mean, if we could change textures mid-mission, I think this would already be possible to do via the scripting system - same sorta thing as Wanderer's awesome little flashy deaths script, except set it to switch ships over to their death animation glowmaps if they have them as they're beginning to go pop.
A new effect system wouldn't be required to do what you want, obviously, but it would make the task considerably easier. A new effect system would really just be seriously upgraded shipfx code, something that you can have far greater control over. Things like warp in/out, cloaking, death sequences, and other large scale type of effects would run through there. You could do things like the Shadow shimmer for TBP for instance, or a real BSG style FTL thing. Such effects could probably be done now using scripting and the particle system, but the particle system just really isn't suited for such things. A new effects system would also allow far easier tie-in for shaders as well, giving you the ability to bring a far more detailed and customized effect.
So is this just a tag you add to the base map or is it a separate texture? Also, is the end result much different from setting a base texture to be transparent via the alpha channel? I'm a bit unclear there.
It is just what you add to the base map. So if your texture is normally "fenris-htl" then rename it to "fenris-htl-trans" for both the filename and the name that is referenced in the POF. It will then render as transparent. Make sure to change glow/spec map names as well, so that they can still load. This doesn't necessarily preclude the use of real alpha though. In the effect that SoL is using, they have a small black base texture (which is -trans, and rendered transparent) with a glow map which makes use of real alpha. It's a really neat effect.
-
Well.... that sounds most intriguing. :nod:
Are there any current plans for how such a system would work at the modders level?
Ie, would stuff be defined in a separate table or the ships table on a per ship basis, and what sorta stuff would it give control over that can't be done or can't easily/efficiently be done at the moment? Particles/lightning/explosions sorta thing?
(Sorry, more questions - but you've gotten me interested now. ;) )
-
1) Can/will we be able to change textures mid-mission via SEXP?
I'm under the impression we can't yet, but I reckon this would be the best way to handle custom cloaking devices and the like. :)
That's an excellent optimization idea. You can already cloak/decloak ships by using change-ship-class for models with animated textures, but changing the map would be even better. Ask the WCSaga guys for the cloaktest zip I sent them awhile back. (I don't think I have it myself anymore.)
-
I remember that actually - back when I saw it I thought it was a bit of an odd way to go about it though, since I still thought the texture replacement worked mid mission. ;)
It is a clever way to do it with what we have now though. :)
-
Are there any current plans for how such a system would work at the modders level?
Ie, would stuff be defined in a separate table or the ships table on a per ship basis, and what sorta stuff would it give control over that can't be done or can't easily/efficiently be done at the moment? Particles/lightning/explosions sorta thing?
There aren't really any current plans (detailed anyway) for the new system, it's just something that we seriously need to work on at some point in the not too distant future. The idea would be to make it all more moddable, both from a tbl level and from a scripting level. We will at the very least need a good tie-in interface, at the tbl level, for shaders. If that is going to be done half-way decently then it needs to be done as the ground work an improved effects system for larger scale things. The particle system needs to then be scaled back for smaller and more speciallized effects (the two systems are mutually exclusive in principle from a design/optimization point of view).
Whether or not this will all be provided though existing tbls (other than the shader definition files themselves) or whether an additional tbl will be introduced, it totally up in the air. Obviously the intent will be to do as much as possible with the existing tbls though.
-
Not sure if anyone has said this before, taylor, but that is all excellent advice regardless of which game engine you use. If people perfect those skills here, the principles are relevant to other game engines, which in turn makes for a real skill that could be valuable to that artist later down the line.
Anyway, back to your fetaure discussion :)
-
Great thread :yes:
I should really make DDS version of my textures...sigh...
-
Necroing this for a quick question...
Considering the first post and advances in both the rendering resolutions and PC internals, giving us more power for $$$, what would be the general.. well, reccomendations on texture size and ship polycounts?
-
I am using Rhino as my main modeling program to export as 3DS or OBJ, then 3D Exploration for doing my export to COB, then PCS2 to create a POF.
I have MAX 4 at my disposal but find it way too clumsy and over complicated as a modeling platform.
I have a question concerning overall capital ship construction.
When I create groups of objects:
Turrets:
do I group the cannon arm components, then group the cannon arms to the base of the turret?
create a separate layer and group for each turret?
Then group the turrets as one large group, then group them to the main hull group of poly's?
Do I group each level of the model details in any specific order to be read?
Do I name the layers, objects, and groups with the Detail-01 , Detail-02, Turret-01, Turret-02, etc.. tags by level of detail?
Or do I have more freedom in naming my objects, Groups, Layers, etc...?
-
To be honest it probably won't matter at all what you call your objects and groups - you'll most likely lose them when going through 3d exploration.
This is what a healthy hierarchy will look like in TS:
(http://i5.photobucket.com/albums/y184/VA--Twisted_Infinities/Misc/HealthyHierarchy.jpg)
(The names of the lights and meshes are not important - the names of the objects are)
3d exploration won't keep those lights intact, meaning that you'll often find your turret arms will merge with the turret base as one object, and single part turrets will merge with the hull object, because object groups can only be formed with two or more objects.
You might be able to get around this though, by using dummy objects - and by that I mean just really tiny tetrahedrons placed inside other geometry to create the object groups. Then you might be able to set it up to export from 3d exploration like on the right hand side of this:
(http://i5.photobucket.com/albums/y184/VA--Twisted_Infinities/Misc/VariousHierarchies.jpg)
Oh, and finally - have a look at this thread (http://www.hard-light.net/forums/index.php/topic,54757.0.html) and also check to see if you are able to export Collada .DAE objects from Rhino. If so, then your conversion life is probably going to become a whole lot easier soon. :)
-
Awesome post sir, I truly thank you! ;)
-
VA, I'd say sticky this after a post like that but it already is. Hope everyone notices that who needs to. Maybe it should be split and stickied somewhere? Or put on the wiki?
-
Well the collada implementation should soon be doing away with the need to so clearly understand the TS hierarchy I'm hoping. ;)
And to be honest my description there is kinda all over the place. :nervous: I really should write up a better one for the wiki sometime.
-
[q], anisotropic filtering level just became a registry option for instance[/q]
What's the flag? I would like to try this out.
It's a string value, "OGL_AnisotroicFilter", but it only takes numbers (a float value). Minimum value is 1.0, which is to not enable the filter, max value is determined by the video card (will likely be 16.0 on current cards). It will cap the value if you go out of that range. Also, if using a debug build, you can change the setting in game by bringing up the debug console with "Shift-Enter" and using the "ogl_anisotropy" command. Use "help ogl_anisotropy" to get a general idea of it, "ogl_anisotropy" with no options will reset to the default/registry value, "ogl_anisotropy x" will set the filter to "x" value. The default value is 1.0.
The anisotropic filter did work before but not properly, and it was always set to the max.
What about DirectX's anisotropic filter? Did it still not work properly as OpenGL's equilavent?
And also, as you never heard before, the geometry instancing is a practice of rendering multiple copies of the same model in a scene at once. I believed it may be used as ships, asteroids, and other objects. It is supported starting from Direct3D version 9 which will be implemented in future releases. :)
-
So lemme get this straight: you bumped a thread that's been inactive for nearly a year, to respond to a post that was from over 3 years ago? You're lucky your post is relevant... ;)
-
Whoa whoa whoa, hold on. Nevermind that this was some pretty big necro, it's also his first post, so maybe he's been lurking for a while and just wants to set the record for most beams caused by one post. I dunno. But what do you mean D3D9 will be implemented in future releases? Of what, FSO? Are you volunteering to fix FSO's D3D support or something? Also, did you go to UMR? I knew of a Bryan See but I'm not sure how spelled his name.
-
Whoa whoa whoa, hold on. Nevermind that this was some pretty big necro, it's also his first post, so maybe he's been lurking for a while and just wants to set the record for most beams caused by one post. I dunno. But what do you mean D3D9 will be implemented in future releases? Of what, FSO? Are you volunteering to fix FSO's D3D support or something? Also, did you go to UMR? I knew of a Bryan See but I'm not sure how spelled his name.
Yes, I want to help out and volunteering to fix FSO's D3D support and anything I see which has gone wrong. I'd like to commit changes to the SVN respiratory. I've recently changed the asteroid limits from 256 to 32768, and created D3D9/10 (and possibly D3D11) support, because Direct3D 8 has since been unsupported in Windows Vista. Also, I've created some Game Definition File (GDF) files so that the game can be launched from the Windows Vista Games Explorer.
-
That's good news. Contrary to some beliefs, D3D support wasn't dropped in 3.6.10 because anyone was against, it just had no maintainer. Assuming you're working with the current trunk SVN, and your changes don't interfere with the new OpenGL code, that's really good news. Also, many limits are what they are due to breakage cause in seemingly unrelated areas by changing them, typically multiplayer. If you could break up your changes into the different features as patches, I'm sure some of the staff would love to take a look at it. I'm not in control of who gets SVN write access but seeing some code first would be helpful. There's a lot of undocumented coding practices around here that most coders just learn as they go, as it's somewhat rare for a coder to show up and just immediately have code to show off. Not that there's anything wrong with that :) I can only recall one other time it's happened, and that was the widescreen fix that showed up out of left field.
Anyhow, D3D9/10 is great. It was generally known the D3D stuff was going to have to move in that direction, which was another block to anyone working on it. Also, do you happen to have a test build with all/any of this stuff in place? Assuming any of it is in a functioning state already.
-
Thanks for your post. Since D3D support has no maintainer, I have to step in to gain my experience in programming. So who's in charge of SVN write access, so that I could commit my changes to it?
I have a test build ready, but I don't have much time to upload to the recent builds forum unlike the others.
-
So who's in charge of SVN write access, so that I could commit my changes to it?
That's not how it works. You'll need to join the SCP project first, "prove your worth" by submitting patch files for features or fixes, and then eventually (if you quality) you get write access. It's a meritocracy. :)
-
How to submit patch files for features or fixes? Through the recent builds forum? With 7zip files and a MD5 sum? How to upload like the rest of the other patches?
-
bryan, upload it so we can all take a look at it.
(i'm actually afraid of this whole d3d thing, since GLSL shaders are givin me enough headaches as is...)
-
But how?
-
You can post the patch as an attachment, or inline in a post using the code tags, or pastebin, or anywhere you can upload it to really. If it's a large set of patches I would 7z the patchset and post it as an attachment.
-
What about the url: swc.fs2downloads.com?
-
That's actually the Star Wars Conversions' own site, I just use it to host some other stuff. To be honest a lot of that stuff should be on www.fs2downloads.com but Hunter broke it :P But if you want that kind of upload you could try mediafire, rapidshare, etc.
-
While I am all for more people being interested in programming for the SCP, I honestly hate to think how badly bricked further development will get by trying to keep a dead horse alive.
D3D got resurrected practically last minute once already, just let it fade.
Further, all this should get split off, as it has nothing to do with making efficient models.
-
Further, all this should get split off, as it has nothing to do with making efficient models.
True dat. And Zacam, if it weren't for all those ****ty Intel chipsets out there, I'd have to agree. But they're making it hard to just completely ignore an offer to help fix D3D.
-
At last, I've got to show you something to prove my worth in order to join the SCP project: a D3D update, as well as an asteroid limit increase from 256 to 32768. You can find them at the attachments below. I tested this build and it is perfectly fine, but I still need to add some more features, including vertex/geometry/pixel shader support, geometry instancing, possibly D3D11, etc. With that, can I have SVN write access?
[attachment deleted by evil Tolwyn]
-
These might be too big to post as attachments, it looks like it attached twice and neither one is openable. I would use Rapidshare unless you want to create an account on Mediafire, or anywhere else someone can suggest.
-
Here's the links to these two files I intend to prove my worth before joining the SCP project:
http://rapidshare.com/files/216367014/fso-WIN-20090402_r5131-DirectX9-10.7z.html
http://rapidshare.com/files/216367018/fso-WIN-20090402_r5131-DirectX9-10.md5.html
-
Any chance we could get the code diff? Perhaps a debug build too?
-
I've got a debug build to get the code diff, too:
http://rapidshare.com/files/216479986/fso-WIN-20090402_r5131-DirectX9-10d.7z.html
http://rapidshare.com/files/216479988/fso-WIN-20090402_r5131-DirectX9-10d.md5.html
-
After some closer inspection, it looks like Direct3D isn't enabling itself the way it was in 3.6.9. When selected the D3D option in the launcher, it's still using OpenGL, so I'm not sure the changes have actually been turned on, or at least that they're not enabled in the way D3D used to be. Is there anything special required to use D3D instead of OpenGL? Also I was talking about the patches to apply to the code, that represent the changes you've made from what's in our SVN trunk. Could you put up the .diff file(s)?
-
The asteroid.h and freespace.cpp are the files that are changed, whereas the CPP files with a grd3d9/grd3d10 prefix are new. So, do I need to put up the .diff files if I want to join the SCP project?
-
So, new D3D files.... Did you remember to actually link them?
Are you aware that while you might be able to compile a build with DX files present, it is still disabled from actually using D3D and will still default to OpenGL, even if D3D is selected in the launcher?
Which is exactly what happens with both of the builds you posted. OpenGL is still being used, not DX.
In order for there to be _any_ provability that the code you have changed actually has any effect, you are going to have to post DIFF files, not exe files.
And no, generally, just posting a DIFF or two or three does not automagically grant access to writing in the SVN. You'll get more results by fixing existing issues and/or implimenting error free features and by establishing a personable presence here on the forums with the members.
-
I have to admit, it's strange that you seem so eager to join the SCP. Normally, people just join the community, contribute some stuff, hang around, etc.
Anyway, you need to provide us the code you've changed so we can see what's going on with it. If you're expecting to put some sort of stipulation on it, like if you become part of the team you'll provide the code, I'm sorry but that's not how it works around here. We're really not that desparate for it in the first place. People become part of the 'the team' when they act like part of 'the team'. This isn't the NFL.
So if you really just want to help out, we need to see the code so we can check it for any pitfalls it might have. As far as I can tell it's not actually functioning right now. This probably sounds like exactly what Zacam said but I still felt the need to say it my way :)
To clarify, Zacam also meant that by default, in all the project files the NO_DIRECT3D preprocessor define has been enabled, and would have to be removed in order to compile Direct3d builds. Otherwise it wouldn't have any effect. As far as I can tell, these builds were compiled with that define enabled. Again, that's why we need to see the actual code, since it appears something is going quite wrong at this point.
-
I see the problem, and I have to remove the NO_DIRECT3D preprocessor define which has been enabled, since I'm a new maintainer for D3D support.
Anyway, are there any issues related to efficient model rendering that I would like to fix?
-
I see the problem, and I have to remove the NO_DIRECT3D preprocessor define which has been enabled, since I'm a new maintainer for D3D support.
Anyway, are there any issues related to efficient model rendering that I would like to fix?
why dont you post your diff files first.
-
:necro:
If you even CAN necro a sticky... Can we have a 2015 update to the 2011 addendum?
I've learned that it's perfectly okay to have a unique texture for each subobject in the model. What should specifically be avoided is using multiple textures for one subobject. That means it's okay to use a shared texture for turrets, or a debris texture for debris chunks/destroyed submodels, or to put all your subobjects (turrets, radar dishes, rotating submodels, detail-boxed greebles) on a separate texture from the main hull.
With modern models it seems low-poly optimization is something of a lost art. (Kids these days, when I was your age we had 800 polies for a juggernaut, grumble grumble.) What kind of polycounts should we be shooting for, for each size of ship? I've seen fighters with many thousands of polies that do little more than slow down the wireframe 3D Ship Select animation, with fine details that should really be normal-mapped; but practically, how much difference does this make? Likewise, I'm working on a cruiser-sized ship now that can be optimized to less than half its current polycount with no loss of detail, but how important is it to actually put in the work? Is geometry processing so computationally cheap that I don't have to bother?
How does the engine handle non-manifold geometry nowadays? If I have intersecting polies, do lighting and collision detection work properly? Can I use intersecting geometry, such as uncapped tubes between hull elements, without having to boolean solids and clean up the joins?
Are quads recommended, acceptable, or discouraged?
Thanks for bringing me up to date.
-
As the recent BP rerelease has shown, the memory limitations of a 32-bit executable are perhaps more of a concern these days than the performance impact of multiple 40962 textures; if you have multiple ships with multiple sizable textures, FSO may well just run out of memory before you have a chance to worry about how many FPS the mission will give you.
Hopefully it won't be too long before 64-bit builds start being distributed on a regular basis for Windows, but it's going to be post-3.7.4 no matter what, so those memory concerns will still be, well, concerns for the next stable release, if not the stable release after that.
-
The advice I give myself when modelling for FreeSpace is, "Don't go nuts. Be smart."
Polycounts should be reasonable to what suits them. Does a fighter need 20k polies? Well I guess its possible. 10k? Probably if its a very prominent player-flown one. 5k? Why not. Modellers should ask themselves "do I need this level of detail? Will I notice it in game?" Do you need 32 sided cylinder on this small fighter, or would a 6 or 12 sided one do? I wouldn't put down any sort of hard-limit to when to stop adding polygons, but if my scout fighter has more polygons than a impressive cruiser, I might rethink some of my geometry. Also consider that collision detection is one of the more prominent bottlenecks the engine has. 20 fighters at 50k polys each close to each other is probably going to cause some slowdowns (as well as shadow map generation).
The biggest thought that keeps my polycount and greeble madness down? "I have to UVMap this, don't I?" If its easy to reduce detail while not cutting down on fidelity, I'd say go for it. If its really hard, eh, I wouldn't worry about it. Let it be a monument to "don't be doing that in the future." ;)
I'm adding these small powerups to JAD and I grabbed this one model from turbosquid's free section. It had 2000 polys, a little high for my taste for a small powerup that the player will barely see. Luckily cutting out the excess polygons was super easy, got it down to 1000. I would have liked around 500, but it needed those polys to keep this bumpy shape that was required.
And on the other side of things, I was trying to import a ship from another game where the poly count was too low! The ship was about 1km long and had around 6k polys total. It looks fine that in game (that game's camera was always far away and the lightning effects were super basic), but once in FreeSpace you could really see the edges up close and it was a really curvy ship, so it was really odd to see bad shine highlights shaped like squares appear. So I had to untriangulate it and carefully set everything up so a smooth operation wouldn't completely wreck the UV mapping. Model poly count jumped up to around 26k polys and the lighting looks a lot better on it now. (Kept the 6km model for LOD1 anyway!)
And with respect to the other more specific points:
Non-manifold geometry isn't an issue, the game will triangulate everything anyway. All the converters I use export triangulated anyway, though I keep my stuff in mostly quad form for easy of editing.
Intersecting geometry should be fine, might need to do the old reset xform trick to solve mysterious collision errors. Uncapped tubes and the like should be fine as well, but you take away UV space from stuff you could jam in there (though personally I'd rather waste the UV space than go through the pain of attaching it properly). You might run into some baking issues with intersecting geometry, but it should be fixable in Photoshop.
These are just my general attitudes to modelling anyway.
-
^ Solid. ^
-
My current workflow always results in fairly high poly models but I always bake detail into normal maps, so if I were to work on a fighter/bomber, I'd end up with about 20-45k tris, but after baking all possible details into the normal map, I'd have a model with between 5k and 10k tris and good looking normal map. From there I can easily make my textures because I can use the normal map as a guide for drawing detail into the diffuse map. The way I specifically do it is by creating my base model, UV mapping it, model the details, bake them, then remove the details.
Quads are perfectly fine to work with, however you should always triangulate the model before importing into the engine. In many cases, you will also need to triangulate by hand because faces may not triangulate in the way you hope they would, if you have any faces that aren't nearly or perfectly flat, you generally will need to triangulate them by hand.
-
Quality posts. Thanks guys!
-
I'd also like to note that when LODding a model, you might want to keep this feature (http://www.hard-light.net/wiki/index.php/Ships.tbl#.24Collision_LOD:) in mind. What I always do when LODding a model is that I make LOD1 compatible with LOD0 collision-wise; I just decrease the number of sides and segments round shapes use, remove tiny greebling, remove small recessions like windows and thruster interiors, and so on.
On fighters you can usually drastically decrease the polycount without perceptible difference. On bigger ships you might be able to decrease the polycount less, but they are usually subject to tons more mesh-level collision checks so it might still help slightly with collision performance. I haven't really ran very detailed benchmarks.
I've learned that it's perfectly okay to have a unique texture for each subobject in the model. What should specifically be avoided is using multiple textures for one subobject. That means it's okay to use a shared texture for turrets, or a debris texture for debris chunks/destroyed submodels, or to put all your subobjects (turrets, radar dishes, rotating submodels, detail-boxed greebles) on a separate texture from the main hull.
Excluding Swifty's new batched model rendering system, the game renders each subobject and texture separately. So if you have a submodel which uses two textures, it should pretty much behave the same way as if it had been split into two submodels, each one using one texture. So, it's good to use as few textures as possible, but if multiple textures are particularly convenient in some case then that's not something to particularly obsess over, as long as the numbers are within reason.
Quads are perfectly fine to work with, however you should always triangulate the model before importing into the engine. In many cases, you will also need to triangulate by hand because faces may not triangulate in the way you hope they would, if you have any faces that aren't nearly or perfectly flat, you generally will need to triangulate them by hand.
Maybe with some exporters? At least when using the 3ds Max OpenCollada exporter, things are triangulated correctly as they are so you don't need to triangulate anything manually.