Great stuff! I especially like how the facial characteristics of the hybrids are mostly preserved. Could you have simply introduced more noise to increase variability?
I don't understand what you mean. Why would we introduce noise into the results?
Given a face to replace and a database to find candidates from, the process is fully deterministic. The database itself can of course grow, or be switched out. This allows one to create different databases for different applications (e.g., one with synthetically rendered faces, one with stock photos, one with personal photos, etc.).
It looks like the hybrid is produced by extracting the entire face from the second photo and fitting it onto the face frame of the first. I get a complete match with the resultant face by doing this in Photoshop.
If this is right, then the program is agnostic to individual facial features, as in it's everything from the eyebrows to the lips, or nothing.
Also, if this is right, would there be privacy implications to the person whom the face is extracted from? If you are protecting the privacy of the person whose face is swapped out, then it makes sense that only their frame remains, but the hybrid face is very similar to the original face (second photo).
That's correct (although our adjustment step changes the lighting and color of the replacement candidate to match the frame of the input image).
This is why I said earlier that one could create multiple databases. Since the replacement candidates might be recognizable, they should be from a set of people who've given their permission for their photos to be used (e.g., stock photo collections). Another option is to use computer graphics to render a large collection of synthetic faces. Modern techniques are good enough to generate fairly realistic faces.
Finally, note that if every face within some application has been replaced (e.g., if this were to be applied on all images on Google Streetview), it wouldn't even be a privacy issue for the replacement candidates -- since users would know that all faces are replaced, seeing a particular face only tells the user that that person was not there.
