One thing that, while not exactly annoying, is certainly awkward, is when a family member (say, my uncle) who has absolutely no knowledge of games or gaming, suddenly sees you playing and decides to find out what the fuss is about. It's a leap of imagination that cannot be explained, it has to be experienced.
How do you explain a game to someone who has never used a mouse, and whose most recent interaction with a computer involved using punch-cards? How do you explain slower-than-light lasers, weird aliens and massive destructo-fests of exploding ships to someone whose reaction to the phrase "Star Wars" is "Ah yes, that Reagan thing."? How do you explain fireballs, dragons, runes, swordplay and level-advancement to someone whose idea of roleplaying is that stuff psychologists inevitably do when they've got a group to work with? How do you explain that one does not need to be a violent person to like Duke Nukem 3D?
There are reasons people like Jack Thomson exist, and one of them is that these things cannot be adequately explained to an outsider. And when someone you care about suddenly shows up and wants to know, and you try your best to explain but know deep down that everything you say is going way over their heads, that's not really comfortable. If makes you feel bad, and makes them feel old, stupid, and perhaps a bit worried if they managed to catch you with Doom 3 running. I now make it a point to never be playing a video game whenever anyone from the more senior generations happen to stop by.