i figure the best way to avoid quality loss would be to avoid putting team colors on areas of the hull with a lot of surface variation. essentially try to limit coloration to mostly flat areas. trouble spots are abrupt changes in normal direction over an area only a few texels wide, where full range needs to go to the green channel. also one axis of the normal is completely isolated from the color data, and variation on that channel shouldn't matter (not sure if alpha is the vertical or horizontal though, i think its horizontal but im not completely sure). so you can color areas in places where you have a lot of variation on the alpha channel , but little or none on the green channel. another thing you can do when colorizing is try to put your contain edges to colored spots within a single 4x4 block (put up a 4x4 pixel grid to be able to identify where the blocks are). you can soften your edges, so long as the transition stay withing the edge blocks. you can probibly also have surface variation in colored areas if the color masks are fairly uniform and completely contain any abrupt seams.
still its a lot of trouble for a pixel monkey. red and blue normal map channels are ballast, and should be treated as such. so thats a vote for a -misc map from me. one thing i thought about (for my own game engine at least), is using the diffuse and specular maps' alpha channels for u and v normals, so that no ballast channels are required. which would give equal performance to bc5, without the need of hardware support. probibly not practical for freespace's established paradigm though. i kinda hope that in the future map channel allocation could be handled with a material system proper, but that may be out of the scope of such a system.