According to Wookiepedia:
Empire: 1 Executor, 1 Communications ship, 50 Imperial Star Destroyers, 10 Victory Star Destroyers, about 5 smaller ships, and 2900+ fighters
Rebel: 20 Calamari Cruisers, 10+ Nebulon B Frigates, numerous other classes of ship (too many to list) and 500+ fighters
Though from the actual movie, I think one could achieve the same effect with a much smaller fleet on either side. Really when watching I only noticed maybe 3 calamari cruisers and perhaps 20 star destroyers. Obviously this was due to the lack of models, but even still, one gets a good sense of the size of the fleets from a couple shots of the Rebel fleet entering and exiting hyperspace, and from the long shots of the battle on the death star as Luke and the Emperor watch.