I get asked this a lot. I think I have a pretty good memory for names and faces, so I can usually provide an answer, but if I'm uncertain, I always have one answer, and it's usually right.
"Yes, you've seen me before. You probably spent about three hours studying me closely. You're not remembering because I was naked at the time."
My favorite occurrence of this happened in a Seattle restaurant years ago. An absolutely gorgeous man came up to ask me this, then before I could answer him, his eyes lit up and he said "OH! I didn't recognize you with your clothes on!" There just happened to be one of those weird silences in the restaurant, so this statement echoed nicely. His girlfriend looked perplexed, and I left the restaurant feeling like I had just earned a much more interesting reputation that I deserved.