A couple of decades ago, facial recognition could only be seen in spy movies. Today, it’s practically an integral part of the smartphone and even the infrastructure of major cities. What is this technology, how does it work, and where is it used?
The article from Юровский.
How facial recognition systems originated
The first research on facial identification dates back to the 1960s. Woody Bledsoe, a professor at the University of Texas, founded his small company, Panoramic Research Incorporated, where he and his colleagues tested all kinds of algorithms, including character recognition. Despite the fact that the company was not very successful, according to unconfirmed sources, the CIA allowed it to stay afloat.
Woody Bledsoe dreamed of creating an “intelligent machine” capable of recognizing faces. In 1963, he presented a project in which his system was to identify ten faces from photographs. This seems insignificant today, but in those days the computing machinery was massive cabinets with magnetic tapes and punched cards. There wasn’t even a universal method for basic photo digitization.
After four years of trial and error, it was decided to recognize a face based on a few key points: nose, lips, mouth width, and so on. The system created was eventually able to identify faces, based on the input data and suggesting the right version of the photo. However, there was still a problem with smiling, different angles and ages for the same person. Because of these variations, the algorithm could identify the same person in the photos as different people.
Eventually, by 1967, a better system was designed that already successfully identified faces on the basis of ordinary newspaper clippings. Most importantly, it proved highly effective. A human could match a subgroup of 100 people in three hours. The machine did it in three minutes, albeit with some flaws.
In 1973, an automated system was created that was able to extract facial features data from digital photographs. Previously, these parameters had to be entered manually. Despite all these achievements, the practical application of facial recognition began only in the 2010s.
There are several reasons:
- The growth of computing power. Only in recent years has the performance of computer equipment become sufficient to process such large volumes of data.
- Formed base. Decades ago, corporations and government agencies did not have photos of not just ordinary citizens, but many criminals. Today that problem has been solved thanks to social media and a digital database of documents.
- Development of cameras. Only in recent years have relatively inexpensive cameras with sufficient shooting quality appeared, allowing their mass use in both smartphones and surveillance systems.
How facial recognition works
Facial recognition is a multi-step process. First of all, the scanning systems are involved; they take a picture of a person’s face and transmit it to the data processing center.
Detection is the first stage. The cameras usually capture not only the face, but also many other objects in the environment. A person is immediately able to tell where the car is, where the background is, and where the person himself is. But for a computer, any photo is just a collection of pixels. The solution to this problem was the Viola-Jones method, developed in 2001.
It is based on the use of special patterns (masks) to determine the light and dark zones. Using a special formula, calculations are made from the dark and light pixels, on the basis of which the result is derived, whether the mask corresponds to the area of the image to be processed. Specific patterns can be identified in the human face. As soon as the algorithm finds a certain number of matches, it gives a verdict – this is the area where the human face is located. The algorithm is trained on other faces beforehand.
However, in the last few years, more and more systems are already using neural networks. They are more accurate, less sensitive to the shooting angle, and, with sufficient hardware, even faster.
Normalization is the second step after face detection. To further make it easier to determine the key parameters, the system tries to build an “ideal frame” – a face that looks strictly straight. Various transformations like rotation, zooming and other deformations can be performed here.
Constructing a “face print” is the next step. How exactly it is done depends on the algorithm used. The variety of methods can be divided into two large groups: geometric and machine methods.
Geometric methods analyze the distinctive features of facial images and form a certain array of data on their basis. The array is compared to a reference, and if the match is above a certain threshold – the face is found.
For analysis, key points are usually used, between which the distances are calculated. The number of required points also depends on each specific algorithm – from 68 to 2000 markers can be used.
Geometric algorithms include the method of flexible comparison on graphs, hidden Markov models, the method of principal components and others.
Machine methods are neural networks. They are trained on a huge database of images and, analyzing a set of certain features, eventually determine the coincidence of faces. How neural networks work is a topic for a separate paper. To simplify things as much as possible, a specific vector is formed for each photo. For example, for three photos (Angelina Jolie and two of Brad Pitt or my face) we get three different vectors. The difference between the photos of Pitt will be minimal, which allows us to conclude – there is one and the same person on the photo.
The largest companies have proprietary algorithms, which are gradually modified. For example, in 2018, more than ten algorithms from leading companies in the field of facial recognition were presented in the NIST test. The first place went to the Chinese company Megvii, the second to the Russian company VisionLabs, and the French firm OT-Morpho rounded out the top three. Other well-known projects also include DeepFace, FaceNet (Google) and Amazon Rekognition.
In addition to 2D face detection with an error rate of 0.1%, there is 3D facial recognition technology. Its error rate is as low as 0.0005%. Such systems use laser scanners with range estimation or scanners with structured surface illumination. The best known technology is Apple’s FaceID, but top Android smartphones also have 3D mapping recognition systems.
Using facial recognition systems
Facial recognition technology is used in a wide variety of applications, usually in one way or another related to security.
The first and one of the biggest areas is urban video surveillance systems. Almost all major developed countries already use or have approved biometric identification. City cameras recognize hundreds of thousands of faces in real time, comparing the results with huge databases. This makes it possible to quickly track down criminals and illegal migrants. Similar systems are installed in almost all airports, as well as many train stations.
However, the number of cameras is not an absolute indicator. For example, the area of Beijing is 16,410 km², and that is about 70 cameras per square kilometer. The area of Paris is only 105 km² and the density of cameras is much higher – 255 cameras per square kilometer! The locations should also be taken into account – many alleyways will still be in the “blind zone”.
One of the most modern is the facial recognition system in Moscow. The algorithms used are able to process about one billion images in just 0.5 seconds. Four different recognition algorithms from NtechLab, TevianFaceSDK, VisionLabsLuna Platform and Kipod are used simultaneously.
The second most important application of facial recognition systems is commercial. These can include:
- Banking. Facial recognition not only makes it possible to identify fraudsters and blacklisted people, but also makes it easier to obtain services.
- Access control systems. Ensuring the security of an enterprise and even an office is much easier with an advanced video surveillance system and facial recognition. It not only simplifies access, but also allows you to quickly identify any people who trespassed on the territory.
- The retail industry. Intelligent systems in stores can offer you, for example, clothes in the right size by recognizing you by your face. And that’s not to mention quick payment by face.
- Medicine. Advanced algorithms will be able to detect individual emotions, such as epileptic seizures or strokes.
- Mobile technology. Identification by face has already become as popular a method of authorization as a fingerprint.
What’s the threat of technology
Of course, in theory, the rapid detection of dangerous criminals with just a couple of facial images is a great prospect. But in practice, the ethics of using the technology can often go overboard.
For example, the FBI has a fairly large database called Next Generation Identification (NGI) – by 2014 it already had about 100 million photos. But it turned out that the database contains not only photos of criminals, but also of people who have never been prosecuted. Moreover, the algorithms used were only 80-85% accurate. Not only you can forget about anonymity on a global scale – the system’s mistakes can turn you into a criminal, if you look like one or just happened to be in the wrong frame of the camera.
Another problem is the increased surveillance by corporations. Many of them already have a comprehensive digital profile, including your photos, geolocation and fingerprints. But with facial recognition data, these corporations will be able to track your movements literally by the minute. It turns out, even if you have never posted your photos anywhere in social networks or even on a smartphone, your face can still get into someone’s database.
The fact that not only government agencies, but basically anyone can find you by your face is also important. After that, it will not be difficult for ill-wishers to find other data – address, phone number, and so on. All this can lead to various types of fraud, threats, and more.
In many countries, lawsuits are already being filed against corporations and government agencies, but in large cities, facial recognition systems will be inevitable. We can only hope that their use will be as lawful as possible. However, it is still possible to defend yourself. The algorithms are imperfect, so a cap, a mask and even glasses can introduce significant errors, which will make it difficult to read your “faceprint”.
