But, unlike the other Biometric technologies, being used today, facial recognition is very prone to privacy rights issues and claims of civil liberties violations. The primary reason for this is that facial recognition can be used very covertly, without the knowledge or the consent of the individuals of whom the system is trying to track down.
Also, facial recognition does have its fair share of technological flaws as well. For example, if a facial recognition system were to capture the image of an individual who is grossly overweight, and then capture another image of the same person who went through massive weight loss, the facial recognition system would not be able to make a positive match. In other words, the system can be very easily spoofed when these aspects of the hand are taken, and this includes the following variables:
But it is not just weight loss which can trick a facial recognition system, but such other things as the presence and the subsequent removal of facial hair, aging, as well as the presence and absence of other objects on the face, such as hats, sunglasses, as well as the switching from contact lenses to eyeglasses and vice versa. But, one of the key advantages to facial recognition is that it can be used for both verification and identification scenarios, and for heavy duty usage as well.
For instance, facial recognition is used quite frequently in the e-Passport infrastructures of many nations around the world, and is also used for large scale identification applications at the major international airports, especially in hunting down those suspects on the terrorist watch lists.
Facial Recognition: How It Works
Facial recognition technology relies upon the physical features of the face (see Figure #5) of which are determined by genetics. Also, this technology can either be deployed either as a fully automated system, or as a semi-automated system. With the latter, no human interaction is needed, all of the verification and identification decisions are made by the technology itself. With the latter, human intervention to a certain degree is required, and this is actually the preferred method for deploying a facial recognition system.
Given some of the serious obstacles it still does face, it is always better to err on the side of caution and have an actual human being have involvement as well in rendering a verification or identification decision.
Facial recognition systems of today focus upon on those parts of the face which are not as easily prone to the hurdles just described. These regions of the face include:
- The ridges between the eyebrows;
- The cheekbones;
- The mouth edges;
- The distances between the eyes;
- The width of the nose;
- The contour and the profile of the jawline;
- The chin.
The methodology to capture the raw images of the face is much different when compared to the other Biometric technologies. Although facial recognition is a non contactless technology, the image capture processes are much more complex, and more cooperation is required on part of the end user. To start the process of raw image collection, the individual must first either stand before a camera, or unknowingly, have their face captured with covert surveillance methods, such as using a CCTV camera system (with the technology that is available today, facial recognition can literally be implanted in a CCTV).
Once the raw images are collected by the camera, the data is then either aligned or normalized to help refine the raw images at much more granular level. The refinement techniques involved include adjusting the face to be in the middle of the pictures which have been taken, and adjusting the size and the angle of the face so that the best unique features can be extracted and later converted over to the appropriate verification and enrollment templates.
All of this is done via mathematical algorithms. As mentioned previously, facial recognition is countered by a number of major obstacles, but even more so at the raw image acquisition phase. These include a lack of subtle differentiation between the faces and other obstructive variables in the external environment, various different facial expressions and poses in subsequent raw image captures, and capturing a landmark orienting feature such as the eyes.
To help compensate for these obstacles, much research and development has been done in the in the area of what is known as 3-Dimensional imaging. In this technique, a shape is formed and created, and using an existing 2-Dimensional image, various features are created, and the resultant is a model which can be applied to any 3-Dimensional surface, and can also be used to help compensate for the above mentioned differences.
However, it should be noted that these types of 3-Dimensional facial recognition systems are not widely deployed in commercial applications yet, because this technique is still considered to be under the research and development phases. Right now, it is primarily 2-Dimensional facial recognition systems which are used on the commercial market. 3-Dimensional facial recognition systems are only used as a complement to the 2-Dimensional ones, in which higher imaging requirements are dictated, and the capture environment is much more challenging.
According to the International Committee for Information Technology Standards, also known as INCITS, the raw image of a face must meet certain stringent requirements in order to guarantee its effectiveness and reliability in a facial recognition system. These requirements are as follows:
- The facial raw image must include an entire composite of the head, the individual must possess a full head of hair, and the raw image should capture the neck and shoulders as well;
- The roll, pitch, and yaw of the facial raw images collected must possess a variance of at least +/- 5 degrees of rotation;
- Only plain and diffused lighting should be used to capture the facial raw images;
- In order for verification and/or identification to take place, no shadows whatsoever should be present in the raw images collected.
If a 3-Dimensional facial recognition system is used, the following properties must be observed:
- Stereo imaging must utilize at least two cameras, which are mounted at a fixed distance;
- If structured lighting is used, the facial recognition system flashes a defined, structured pattern at the face, which is used to capture and compute depth;
- Laser scanners are the most robust form of sensing, but are very costly to implement as well as very slow, it take as long as 30 seconds or even more to capture and process the raw image of a face;
- Hybrid sensors do exist, and can use both the stereo imaging and structured lighting techniques.
The entire process of facial recognition starts with the location of the actual image of a face within a set frame. The presence of the actual face can be sensed or detected from various cues or triggers, such as skin color, any type or kind of head rotation, the presence of the facial or even head shape, as well as as the detection and presence of both sets of eyes in the face.
Some of the challenges involved in locating the face in the frame include identifying the differentiation between the tonality of the skin color and the background, the various shapes of the face (depending of course on the angle in which the raw image is actually presented to a facial recognition system), or even multiple images of faces may be captured into a single frame, especially if the facial recognition system has been used in a covert fashion, in a very large crowd.
The Techniques of Facial Recognition
To help alleviate these obstacles and to provide a solution in which a single facial image can be detected in just one frame, various techniques have been developed and applied to facial recognition. These techniques fall under two categories:
- Appearance based;
- Model based.
With appearance based facial recognition techniques, a face can be represented in several object views, and it is based on one image only, and no 3-Dimensional models are every utilized. The specific methodologies here include Principal Component Analysis, and Linear Discriminant Analysis. Model based facial recognition techniques construct and create a 3-Dimensional model of the human face, and after that point onwards, the facial variations can be captured and computer. The specific methodology here includes Elastic Bunch Graph Mapping. All of these techniques will now be discussed in greater detail.
With Principal Component Analysis (this is linear based, also known as PCA), this technique dates all the way back to 1988, when it was first used for facial recognition. This technique primarily uses what is known as “Eigenfaces”. Simply put, Eigenfaces are just merely 2-Dimensional spectral facial images, which are composed of grayscale features.
There are literally hundreds of Eigenfaces which can be stored in the database of a facial recognition system. When facial images are collected by the system, this library of Eigenfaces is placed over the raw images, and are superimposed over one another. At this point, the level of variances between the Eigenfaces and the raw images are then subsequently computed, averaged together, and then different weights are assigned.
The end result is a 1-Dimensional image of the face, which is then processed by the facial recognition system. In terms of mathematics, PCA is merely a linear transformation in which the facial raw images get converted over into a geometrical coordinate system. Imagine if you will, a quadrant based system. With the PCA technique, the data set with the greatest variance lies upon the first coordinate of the quadrant system (this is also termed the first Principal Component Analysis), the next data set with the second largest variance falls onto the second coordinate, and so on, until the 1-Dimensional face is created.
The biggest disadvantages with this technique are that it requires a full frontal image, and as a result, a full image of the face is required. Thus, any changes in any facial feature requires a full recalculation of the entire Eigenface process. However, a refined approach has been developed, thus greatly reducing the calculating and processing time which is required.
With Linear Discriminant Analysis (this is linear based, also known as LDA), the is to project the face onto a vector space, with the primary objective being to speed up the verification and identification processes by cutting down drastically on the total number of features which need to be matched.
The mathematics behind LDA is to calculate the variations which occur between a single raw data point from a single raw data record. Based from these calculations, the linear relationships are then extrapolated and formulated. One of the advantages of the LDA technique is that it can actually take into account the lighting differences and the various types of facial expressions which can occur, but still, a full face image is required.
After the linear relationship is drawn from the variance calculations, the pixel values are captured, and statistically plotted. The resultant is a computed raw image, which is just simply a linear relationship of the various pixel values. This raw image is called a Fisher Face, which can be seen in. Despite the advantages a major drawback of the LDA technique is that it does require a large database.
With the Elastic Bunch Graph Matching (this is model based, also known as EBGM) technique, this looks at the non linear mathematical relationships of the face, which includes such factors as lighting differences, and the differences in the facial poses and expressions. This technique uses a similar technique which is used in iris recognition, known as Gabor Wavelet Mathematics.
With the EBGM technique, a facial map is created, and an example of this can be seen in. The facial image on the map is just a sequencing of graphs, with various nodes located at the landmark features of the face, which include the eyes, edges of the lips, tips of the nose, etc. These edge features become 2- Dimensional distance vectors, and during the identification and verification processes, various Gabor mathematical filters are used to measure and calculate the variances of each node on the facial image.
Then, Gabor mathematical wavelets are used to capture up to five spatial frequencies, and up to eight different facial orientations. Although the EBGM technique does not at all require a full facial image, the main drawback with this technique is that the landmarks of the facial map must be marked extremely accurately, with great precision.
Facial Recognition: The Advantages & The Disadvantages
Facial recognition systems can also be evaluated against the same set of criterion. In this regard, there is a primary difference between this and the other Biometric technologies. And that is, while the face may not offer the most unique information and data like the iris and the retina, facial recognition can be very scalable, and like fingerprint recognition and hand geometry recognition, facial recognition can fit into a wide variety of application environments.
The evaluation of facial recognition can be broken down as follows:
- Universality: Unlike all of the other Biometric technologies, every individual possesses a face (no matter what the condition of the face is actually in), so at least theoretically, it is possible for all end users to be enrolled into a facial recognition system;
- Uniqueness: As mentioned, facial recognition is not distinctly unique at all, even members of the same family can genetically share the same types and kinds of facial features, as well identical twins (when it comes to the DNA code, it is the facial features which we inherit the most resembling characteristics from);
- Permanence: Given the strong effect of weight gain and weight loss (including the voluntary changes in appearance), as well as the aging process we all experience, permanence of the face is a huge problem. In other words, the face is not at all stable over time, and can possess a large amount of variance. As a result, end users may have to constantly be re-enrolled in a facial recognition system time after time, thus wasting critical resources and processing power;
- Collectability: The collection of unique features can be very difficult, because of the vast differences in the environment which can occur during the image acquisition phase. This includes the differences in lighting, lighting angles, and the distances at which the raw images are captured, and also including the extraneous variables such as sunglasses, contact lenses, eye glasses, and other types and kinds of facial clothing;
- Performance: In this regard, facial recognition has both positive and negative aspects, which are as follows:
- Accuracy: Facial recognition according to recent research, has a False Acceptance Rate (FAR) of .001, and a False Rejection Rate (FRR) of .001;
- Backward compatibility: Any type or kind of 2-Dimensional photograph can be added quite easily in the database of the facial recognition system, and subsequently utilized for identification and verification;
- Lack of standardization: Many facial recognition systems are in existence, but there is a severe lack of standards amongst the interoperability of these systems;
- Template size: The facial recognition biometric template can be very large, up to 3,000 bytes, and as a result, this can greatly increase the storage requirements as well as choke off the processing system of the facial recognition system;
- Degradation: The constant compression and decompression and recycling of the images can cause serious degradation to the facial images which are stored in the database over a period of time;
- Acceptability: In a broad sense, facial recognition can be widely accepted. However, when it is used for surveillance purposes, it is not at all accepted, because people believe that it is a sheer violation of privacy rights and civil liberties. Also, some cultures prohibit the use of facial recognition systems, such as the Islamic culture, where women are required to wear head scarves, and hide their faces.
- Resistance to circumvention: Facial recognition systems can be very easily spoofed and tricked by 2-Dimensional facial images.
Applications of Facial Recognition
Given the covert nature and the ability to deploy to deploy facial recognition easily into other non-biometric systems and technologies, it is no wonder that it has a wide range of market applications it can serve. Probably the biggest application for facial recognition has been that for the e-Passport, and to a certain degree, the National ID Card system, for those nations whom have adopted it.
For example, the International Civil Aviation Organization (also known as the ICAO) has made facial recognition templates the de facto standard for machine readable travel documents. Also, along with iris recognition, facial recognition is being used quite heavily at the major international airports, primarily for the covert surveillance purposes, in an effort to scan for individuals on the terrorist watch lists.
Also, contrary to public support of it, facial recognition can be used very covertly at venues where large crowds gather, such as sporting events, concerts, etc. Facial recognition systems can also be used in conjunction with CCTV cameras, and strategically placed in large metropolitan areas. For example, the city of London is a perfect example of this. At just about every street corner, there is a facial recognition/CCTV camera system deployed.
Because of this vast network of security over there, the London police were able to quickly apprehend and bring to justice the terrorist suspects whom were involved in the train bombings which took place. Another popular application for facial recognition is for border protection and control, especially widely used in the European countries.
Facial recognition is also heavily used in conducting real time market research studies. For instance, it can be used to gauge a potential customer’s reaction to certain advertising stimuli by merely recording that individual’s facial movements. Casinos are also a highly favored venue for facial recognition, as a means to identify and verify the welcomed guests versus the unwelcomed guests.
Facial recognition has also been used in both physical access entry as well as time and attendance scenarios, but nowhere near to the degree that hand geometry recognition and fingerprint recognition are currently being used in these types of market applications.