Several years ago, taylor wrote a long and exhaustive guide on how to optimize your in-game models for performance
. As good as it was, and as applicable as most of the items in that article were and still are, times have changed: It is no longer 2005 (when it was originally written) or 2011 (when I last updated it) anymore. Modern gaming machines are several orders of magnitude more powerful than even the most beastly of machines back then, after all.
That being said: Speaking just from my personal experience going through tech support threads here or on Discord, there are a lot of people out there who use FSO on older or just less powerful hardware; do keep that in mind when creating mods and campaigns.
Anyway, here are some trends I've been noticing over the past few months and over the past few mediaVP updates in particular, and some lessons we learned from that.
1. PBR textures take a lot of memory
Almost uniformly, new models tend to have textures at sizes that we used to consider ludicrously large just a few years ago. The Aeolus, for example, uses 124 MB of textures spread across its various maps. The Deimos uses over 200 megs, the Hecate close to 300, the Ravana almost 500: A mission that uses all of these assets (and fighters, weapons, skyboxes and other effects besides) will easily exceed 2 or 3 GB of video memory use. Given that the Windows desktop itself reserves some video memory on its own, there may be performance impacts on any card that has 4GB of onboard RAM.
It is therefore recommended that you create lower-resolution textures for your general releases and put the higher-res versions into advanced map packs that users can optionally download.
A good rule of thumb to follow here is to use 2048^2 for large assets like stations or capships, 1024^2 for fighters and smaller vessels.
Another good trick to use is rectangular textures with a 2:1 aspect ratio. A single 8192x4096 map has a lot of UV space available and takes less memory than two 4096^2 textures.
2. PBR textures are noisy
Tools like ArmorPaint or Substance Painter can generate textures with a lot of high frequency elements in them. These look good up close and serve well to give ships a sense of scale, but they also have a drawback: The further away these models get from the camera, the more likely it is that textures start exhibiting artefacts: they start looking noisy and rough due to the way textures are being sampled, giving the models a highly aliased look.
There is a way around that: Mipmapping. When you convert your textures to dds (which you absolutely should do!), do remember to tell the tool you're using to also generate mipmaps for those textures. It will, by necessity, increase the texture size, but it is almost guaranteed that no matter what tool you use, it will be better at it than FSO itself.
3. Do not mismatch texture sizes
The FSO texturing system has, for a long time, supported multitexturing: Diffuse maps, glow maps, normal maps, and a lot more besides. It is strongly recommended (although strictly speaking not necessary) that all maps that make up a given texture be created using the same resolution.
4. Polycount does not matter that much...
This went unmentioned in taylor's post, but let me just mention it for completeness' sake: Polycount does not really matter. It used to; back in the bad old days, the engine had to create cache objects for every model, a process that got exponentially slower the larger the model was. The algorithm responsible has been fired and replaced with a better one.
5. ...except in this highly specific and very irritating way
However, there is a hard limitation in the pof model format itself: No single subobject may exceed 2^16 vertices. While there is no practical limit on the number of subobjects, you do need to take care to divide your model intelligently such that it can fit in this limitation.
6. The rendering performance of a model is almost entirely dictated by its texturecount
A point that taylor made as well, but that is worth reiterating: The fewer textures a model uses, the better. Textures that are shared between several models are also great: When rendering a scene, the engine divides all models into batches organized by shader flags and the textures used; the fewer of these batches there are, the better.
7. Be conservative when creating effects
Not every effect needs to be a 300 frame, 60fps animation at 512^2 pixels. It is possible to get away with less (much less); for everything else there are the aforementioned optional texture packs.