A Global Database for Facial Recognition

1/26/2011

An article on Reddit recently (Unidentified hot girl stealing my Christmas presents) got me thinking: why hasn't a private company created a global database for facial recognition matching?

The pieces are all there: Crawl Facebook's publicly available data and index people's profile pictures. Then, provide a service where anybody can upload a picture and it will attempt to provide a match for the faces in the picture. It's sort of a DNA matching database for photos, if you will.

The implications are a bit frightening, but it's likely just a matter of time before someone does this exact thing, unless the law changes. IANAL, but it seems pretty clear that there is no expectation of privacy in public.

Like this post? Subscribe to this blog now.

John Titus

1/26/2011 09:00:22 am

Facial recognition isn't very good in the best case - subject looking straight at the camera, good lighting, not smiling - and most facebook photos don't come near this ideal. Even the best facial recognition algorithms can't deal with those types of photos when the gallery exceeds several thousand subjects.

David Rusenko link

1/26/2011 09:04:06 am

@John: Do you think these are fundamental limits, or just limits in today's algorithms? Also, even if you just used a subset of Facebook pictures that are ideal, it would still be a pretty powerful tool...

ogrisel link

1/26/2011 09:26:12 am

There are several issues to implement such a global face search engine: - first it is currently possible to identify the person with a high predictive accuracy only if the number of possible identities is constrained to a small number (let say a couple of thousands of identities). For social network it's working well since you can constrain the search to your social graph neighbors. For the general case it's not that easy... - second it's hard to build efficient indexing strategies for face pictures databases: fulltext search is scalable because text documents can be represented by very sparse vectors of word and n-grams (a.k.a. shingles) frequencies in a very high dimensional space. Naive faces representations (e.g. eigenface projection) do not share those feature, they are usually dense vectors (with a couple of hundreds of dimensions). To avoid having to do a sequential code on hundred of millions of encoded pictures you either need to: - find compact binary codes (semantic hashing) whose hamming distance preserve the cosine similarity of the eigenspace vector space - find sparse encoding in high dimensional spaces (similar to what the cortex does) to be able to reuse fulltext indexing strategies on those sparse representations. Both approaches (semantic hashing and sparse coding for multimedia indexing) are currently active areas of research.

Justin Mitchell link

1/26/2011 10:10:04 am

I worked on tag suggestions for Facebook[1], here are some problems with rolling out a global system like this. 1/ You need more than one source image to build a good representation of a user's face. Merely going through profile pictures is not sufficient (particularly given the number of users who do not feature their faces in a profile picture). In general, you're going to want as many source images as possible, with marginal returns becoming less important around 50ish good images. 2/ Face recognition algorithms just aren't that good. The false positive rate is high, particularly with a less constrained search space (e.g. millions of people instead of a few hundred) and suboptimal photos. We got around this problem by limit the search to your immediate social circle, a wider scan would not be feasible. 3/ As mentioned, efficiently indexing all the faces would be an additional challenge (although not entirely insurmountable). My guess is this is the most tractable piece of the puzzle. 4/ It'd probably only work for very ideally selected photos. For instance, a single frame from the hot girl video probably wouldn't be good enough to find a match. That said, including many video frames might yield a better result, but I haven't messed around with that much. Anyway, hope I didn't discourage anyone too much. -Justin Mitchell [1] http://blog.facebook.com/blog.php?post=467145887130 Disclaimer: The views expressed here are my own, and do not represent Facebook in any way.

Ido Rosen link

1/26/2011 10:18:50 am

@David: Humans are great at face recognition. If humans can do it, so can computers, just not yet. Modern machine learning techniques have improved facial recognition significantly, to the point that it's not actually all that bad, but none of the readily available open source facial recognition tools use some of these techniques and the code from academics testing ML algorithms on face recognition is widely dispersed across many publications and personal research pages. A practical solution would probably have an ML algorithm suggesting similar faces and identities and might crowdsource or have people checking those automatically generated matches to train the algorithm in some deformable model of what that face looks like in 3D, then let humans (through Facebook friends or even publicly available photos) find that person. Along the way, there'd probably need to be some sort of TinEye-like reverse image search involved to aid such a database in finding references to that uploaded image, if they exist. (TinEye itself is too simple of an implementation, most likely there'd be better similarity measures used than just "is this bitmap identical".)

Ted

1/26/2011 11:21:59 am

Using geographic location as a filter (or match confidence modifier) might make the problem easier. People probably spend most of their time around one area. I doubt that the gift thief lives very far away from the house she hit up. @Ido: Are humans really that good at this sort of facial recognition? If I had infinite time and tried to match the thief girl against hundreds of millions of photos, I'd probably find a bunch of possibilities (aka false positives) and be unable to narrow it down.

johndburger

1/26/2011 11:33:16 am

@Ido "Humans are great at face recognition.". Really? How many humans would be good at the task described here: look at a collection of millions of pictures of people you don't know, then match a new one against the collection. I couldn't do it.

David Jones link

1/26/2011 01:39:53 pm

Humans are not as good at face recognition as you might imagine.

Scott link

1/26/2011 02:40:28 pm

Its already been done. By private companies selling to the Highest bidder, mainly the Govt. How do I know, cause I have been watching my coworkers working on the project for the past year. Its just not public yet...

1/26/2011 05:39:16 pm

Didnt google pull facial recognition from their google goggles product a while back. Seems the ability is ready there, they are just afraid of the legal issues, ala the guy coming out of the strip club on streetview

Hrishi Mittal link

1/26/2011 10:28:21 pm

Dailybooth would probably be a good source to start with, since it has a lot of pictures of each user.

Ido Rosen link

1/27/2011 03:54:49 am

@johndburger, @DavidJones: I should've qualified that: Humans are wonderful at face recognition given some context. One bit of information humans have that machines don't is contextual information. Even just the context that they've met the person before, that the person is at a conference of some sort, or on the street in New York, is helpful. Similarly, if you've annotated the image with contextual information (what site it's from, what other sites feature the same image [this is where my reference to TinEye comes in], etc.), and you integrate that context somehow, you'll have significantly better results. Face recognition on CCTV systems like London's could work extremely accurately if they depended on tracking people's routes through the city, and given contextual information of where they've shopped and the ability to cross-reference that with other data sources. That context is similar to the context available to humans (i.e.: where was I when I last saw this face, does the most likely image match also like shopping for clothes at Gap, did I notice this person ever wearing Gap clothes, etc.). The reason you may think humans are bad at face recognition is that without any contextual information whatsoever (i.e. not even lighting or background information), there is an insanely high false positive rate. In a sense, you're also comparing apples to oranges in your analogy by conflating the scale and nature of the processing that a computer given a million (or even billion) images and perhaps some small amount of context for each image (the URL from which they came), to a human given a thousand images and millions of contextual clues (in the background of the image, in the content of the site the image came from, in being able to use Google to find people who work in the company logo in the background, etc.)... Humans are pretty ingenious at face recognition in a natural setting (i.e. not on a computer screen). This is why social context (Facebook reverse image search), information extraction from the source of the image (website, etc.), and other context is key in the artificial setting where you don't have access to motion/emotional cues, background setting cues, and millions of other potential contextual cues.

Ido Rosen link

1/27/2011 04:01:00 am

@johndburger, @DavidJones: By the way, this is an awkward place for me to respond. Feel free to reach out to me at the link by name. I'm always interested in good conversation, and it looks like you both have interesting backgrounds. :o)

dissertations link

1/31/2011 07:29:45 pm

The good thing about your information is that it is explicit enough for students to grasp. Thanks for your efforts in spreading academic knowledge.

Joel link

2/1/2011 06:23:25 pm

Thank your for the Information David. I enjoyed this article a lot! I would anyway like to ask why is it that Weebly.com doesn't have a new Design? It's been the same since i began working with Weebly. That was 2 Years ago. Thanks!

David Rusenko link

2/2/2011 11:54:18 am

@Joel, We're coming out with a new redesign soon, so stayed tuned! :)

Comments are closed.

David co-founded Weebly, an incredibly easy to use tool that helps millions of people create a professional web site, blog or online store.

He was named to Forbes' 30 under 30 list, is a part-time DJ and has traveled to over 20 countries.

Investments include Cue, Parse, Exec, Churchkey, Streak, Incident Technologies, Adioso and Zenefits.

RSS Feed

A Global Database for Facial Recognition

Categories

Blogroll

Flickr Photos