Are the ships using just one waypoint (i.e. just one point to travel to), or are they using multiple points (i.e. a patrol path)?
If they're using just one, I found the problem always is that I give the ship a "Waypoints" order, but not "Waypoints Once"; the ship therefore tries to do the waypoints again, though it's already at it (which results in spinning and weird dancing).
If they're using two, they might be too close together (that is, if the waypoints end up close together), or there's simply something wacky with the ship AI.
The problem I could see with yours is really only that the waypoints aren't connected. That, or something's blocking the other waypoint (though, in this case, the ship would just mindlessly beat its head into whatever the object is).
I would recommend checking the "Waypoints" --> "Waypoints Once" issue. See how that works. If all else fails, send me a version of the mission and I'll see what I can do with it.
