Looks very interesting.
But... I'm wondering.
How is the glide done in BtRL? Scripting or in-engine code?
If it's in-engine code, isn't that a way to disable FS2_physics calculations (or, well, part of them anyway) right there? The problem seems to be that FS2 tries to make it's own non-newtonian calculations on the background and you're trying to override them in the script with changing the position vector manually. But isn't the glide system already a possibility to enable effectively newtonian physics? Without the speed limit the BtRL currently has, that is. It has the ability to change ship's vector in the glide mode with thrusters, up to the velocity limit.
If it's done by scripting as well... well, perhaps adding atmo flight to that script would be a good place to start?
You will need to add control surfaces to enforce stability in that case, though, so that yawing doesn't make your ship fly sideways, since the FS2 physics won't be helping with keeping the ship flying towards where it's pointed at any more. Or have you done something else already to avoid ships flying sideways? I'm not so good at reading the scripts but I didn't notice anything like that.
The thing is, to add stabilizers realistically you would need to already put control surfaces onto them, and that would force you to include a way to change their lift coefficients with input controls, and a way for them to change the ship's attitude. Also, you need to take into account that the ship can have sideways AOA as well as pitch-oriented, and it affects the hull and control surfaces... essentially it affects drag also, because flying sideways increases the cross-section of the ship in the airflow, causes turbulence and other nasties like that... And even though I don't really know theg scripting system I can see it's not going to be a very coder-friendly process...
Other thing I was wondering. Why don't you calculate the lift, drag and propulsion force vectors in relation to ship, bunch them all together and then convert that net force vector to velocity change vector (ie. acceleration) and only after you have the net force vector calculated, convert it to global reference frame co-ordinate system. That way you'd only need to do that conversion once. Gravity is done in general reference frame anyway so it doesn't need to be converted at any phase according to my understanding.
Of course, if the engine can get acceleration values from LUA directly, that would be the easiest way of doing this. Is that possible? It would make all the conversions from acceleration to velocity change (not to mention position change) redundant, the only thing you'd need to worry is changing the ship-related acceleration to global reference frame.
As to the physics of the equations, I don't see anything fundamentally wrong with them so far... apart from them being rough approximations of aerodynamics, that is. But in a game like this, calculating the airflow based on the model mesh is not really necessary, tabled aerodynamic values will do very well (MS Flight Simulator uses them as well, so you're in good company). If we want accurate aerodynamics we need to port the FS2 media to X-Plane.
But for example, drag coefficient changes somewhat when you enter sub-sonic, supersonic and hypersonic areas of velocity, what with compressed shockwaves going on around the ship bouncing to and from the surface, and changing the viscous properties of the air a bit. But I daresay that's a bit more advanced physics modelling and should perhaps wait until you get the fundamental stuff working properly with the rest of the game.
I might add that I know next to nothing about the scripting system and what it allows so I'm just kinda spewing ideas around. They might not be possible at all, but then again, they might have some value.