Everyone seems to be coming to me with something like this lately, but you are really just wasting your time. Remember that this has been in the works since last year, and in testing since March. You also have to remember that without a material system in place the shaders have to be one-size-fits-all. And all of the content has to work in a way that it's fully compatible whether you use shaders or not. You also have to deal with the fact that not every ship is going to need/want a normal map, much less a height map. This isn't a FPS, and you really don't get close enough to see that much detail in the models all that often. In most cases the models are better off just using better base maps and/or more geometry rather than wasting memory on normal/height maps that aren't going to be noticeable a majority of the time.
Everything you are thinking of has already been considered, and we went with the current method because it simply works the best all around. So, here is a quick Q&A about this options:
Put the height map data in an extra channel of another map (like glow)
Nothing but the normal map can be modified to contain extra data since all of the other maps are still used by the fixed render pipeline and it has no idea how to properly handle (ie, ignore) the extra data in the maps.
Put the height map data in the alpha channel of the normal map
We need to conserve as much memory as possible, so normal maps need to be compressed. During testing the compression artifacts were deemed too great in a regular DXT1 or DXT5 format. Because of that we went with DXT5nm, which uses the green and alpha channels for the normal map, to get the best quality possible. With the alpha channel in use, and the two remaining channels not having enough precision by themselves for the height map data.
Put the height map data across the red and blue channels in the DXT5nm normal map
Certainly possible from a technical point of view, but is problematic for artists and modders. No existing tools would be able to work with the files, which means that we would have to create a utility for converting between the formats. This is problematic on a multitude of levels, basically to the point of not being feasible, even though it is technically possible to do.
Use a extra file for the normal map, but make it 8-bit only
What is the point? It would actually require more memory than a DXT1c image, and it wouldn't perform as well render speed wise as DXT1c either. The only thing that you gain is a lossless format, but it isn't all that important in the case of the height map.
Everything needs normal and height maps, so just sacrifice the quality and save the memory
Matter of opinion, and not everyone is going to agree with that. The shaders are upgradeable by mods though, so they are certainly free to make their content however they choose (like using standard DXT5, with RGB for the normal map and A for the height map) and including modified shader files which work with the content. But by default, doing this isn't the best option.