How does the facial recognition work anyway?

In the past few years all sorts of consumer-marketed gadgets have already promising and doing some facial recognition. And of course your phone, computer, Facebook and where else you might be, will do their share. I’ve thought that part of it is cool and another part creepy, and have wanted to dig in much deeper.

I know the facial processing softwares rely on algorithms. What matter to your recognition is e.g. the distance between the eyes, the shape of the face in general, I guess some wrinkles formations and so on. But it’s all absurd to me. How does it work in practice? Say you use the facial recognition thing to check if a person’s face looks like the picture in their ID or passport at an airport. Does any of that check rely on using people as checkers any more? What if someone gains or loses a lot of weight, or has some cosmetic surgery on their face?

And how does the old-fashioned, analog facial recognition work? How do you recognize someone you know? How long do you need to know someone or how many times do you need to meet them to be able to recognize them (if you have no other cues such as voice)? How does any of it work?

My confession is it never really worked for me. I grew up sighted, but usually my glasses grew fast to be off enough to keep it in a blurrier world, sharp enough to see where the road was when walking on it. I would learn to recognize school and work friends slowly, but it would take ages. I probably already then used the techniques I use now: get to know the voice of the person, random trivia about what they do and like, what kind of humor they have, then associate them with the place or context you meet them in, and try to figure if they have any unique, easy to recognize features, such as being 3 or 9 feet tall, using a wheelchair or a guide dog and so on. The downsides of my techniques are pretty obvious: meeting people out of the location or context you usually meet them at, or they dye or cut their hair, or dress in different styles and colors of clothing. All of those things are enough to make me not recognize them – and those aren’t really limited to outsiders either. I’ve had trouble over the years in recognizing my own face in photos and videos too. Voice is the easiest way to recognize someone as a result. And even voice has downsides: trying to differentiate accents or recognize one of the many bland, not really unique voices in your third language is a lot more work than recognizing someone in the first or second language. Some people also change their accent or speech patterns moving from one setting to another – just please don’t do that, people.

If you can easily recognize someone, are you able to remember at will what the person looks like, and describe their features in detail?

I’ve never been good with that. Even basics like guessing the age or height can be quite bit of guessing. Also adults don’t typically ask others, especially in person, how old or tall they are, at least if in sighted or pretend-sighted company. Guessing people’s age from their voice and what they say can still be quite off for me. I can easier guess how tall someone is if they stand very close to me (even if I don’t see them – I can hear where their voice comes from and that means also higher or lower from the ground), but not so well if they stand further, or if they are sitting. If someone tells me their age it feels awesome – both as in sharing some personal detail and also in often understanding that I probably won’t be able to get that information visually. And there is nothing wrong with any age. It can often also help in figuring further chat topics, such as finding if you had similar things in the same age if around the same age and growing geographically apart, or asking or sharing about how it was when you were that age to do something that was obviously done differently than these days.

Makeup and haircut can definitely make people unrecognizable to me, especially if radically changing the amount of makeup typically worn or going with different style, color or length of the hair. Then again these days I rarely know if someone wears any, and usually feel too awkward to just ask. I always seemed to assume all ladies between the ages of 15 and 60 would wear some, and only recently have discovered (that is, been told) that that isn’t the case. That there are a lot of ladies that don’t wear any visible makeup. How much or little does the presence or absence of makeup or hair changes affect the facial recognition for others or in the automated systems? Automated systems should still measure the distances between key facial features on one’s face. So makeup or no makeup, unless you change the colors of the makeup to make the shape of the face appear different, it shouldn’t matter to the systems.

And I keep wondering how people do the recognition. I doubt that people would look at the distances between the features and memorize them like a machine. What makes a face recognizable? The still features? Like if you only have a photo but can’t see the person in live. Or the facial expressions uniqueness, the way someone smiles or cries and so on? Does the expressiveness or lack of make a difference in how easily someone new will be recognized? I remember something about the general uniqueness of the people in the past – if someone looked very different from the general population around them, they would be easier to recognize. And these days I wonder about the process and how that works in so many levels. Can you recognize someone you know decades ago from just the looks, without voice? What if you only met someone once?

Someone recognized me last week. They had met me once, at an airport, a number of years ago. I have as good facial recognition skill now as I had years ago – so a white, male American… no clue what the person looks or looked like. Weird and interesting when encounters like that happen, but also uncomfortable. Obviously not much benefit advertising too openly about the lack of facial recognition or low acuity, so occasionally I wonder of all these little details. As far as I’ve understood most people don’t recognize everyone they’ve met, especially if in random contexts. Sometimes it all feels like in these commercials against drunk driving, “they see you before you see them”. I see there is a person, persons, or groups of people, yet usually can’t really provide any useful input in what they look like. So pretty much everyone I run into can see much more details before me. And as always, keeping my eyes or ears open for input from the others about the same topic. Any insights, whether theoretical or from own experience…


2 thoughts on “How does the facial recognition work anyway?”

  1. Interesting topic. I have no idea how I recognize people’s faces. It wasn’t until I started following Autistic people and learning more about autism that I found out that not everyone does recognize faces. You’ve given me food for thought. I wish I had an answer for you.


  2. UnuhiNuiʻi, your blog is currently included on our Actually Autistic Blogs List ( Please click on the “How do you want your blog listed?” link at the top of that site to personalize your blog’s description.
    Thank you.
    Judy (An Autism Observer)


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s