I think the main problem we have here is that we're not sure of the GTVA's rank structure, so its easier assume that destroyers are commanded exclusively by Admirals. If we knew more of the command structure, it'd be easier to determine the fleet composition and size of the GTVA fleets.
But I doubt every destroyer is commanded by rear admiral and above. Maybe the ones we see are all commanded by rear admirals and above, but there could be destroyers captained by .... well ... Captains.
If we assume every destroyer is captained by an Admiral, then the GTA's taskforce mentioned in Awakenings would have like ... 19 Admirals or something. 16 Reliant-class destroyers and 3 Orions. Also, the number of cannonical destroyers we see, if each was captained by an admiral, that's a very big command group.