I took French in Junior High and my first 2 years of High School, and during that time, we took two trips to Canada. The first, we went to Hull. The second, to Montreal and Quebec City.
During the first, a rather amusing incident happened. As it was the first year we had taken French, we had a rather poor grasp of the language. About half way through the day, one of the members of my class took it upon himself to go speak to some kids from the area in French. If I remember correctly, they then took that opportunity to make fun of him in both French and English.
The second trip, I think I actually encountered more things in English than in French, especially in Montreal. The only problem I had on that trip was with an employee in a McDonald's in a mall near Montreal, but I think that was more of a matter of her lacking intelligence as opposed to communications problems (She didn't understand the word "Sprite").