Personal Countersurveillance (I): Facial Recognition

(Please note some of the internal links are in Spanish)

Those of us who work in the cybersecurity sector are accustomed to hearing about threats and defense measures, but almost always referring to a virtual environment. However, there are other dimensions, such as physical security, that can affect us in different ways.

This series has been inspired by the paper by Adam Harvey of the Chaos Communication Congress of 2016: “Retail Surveillance / Retail Countersurveillance”. In it I will discuss some concepts concerning surveillance systems and counter-surveillance measures that can be used to avoid recognition by third parties.

This first article focuses on facial recognition, some of its most controversial applications to date and their implications.

Figure 1: Anonymous. Image taken from http://luisjimenez.com/wp-content/uploads/2016/05/faception.jpg [Posted on 15/05/2017]

Writer Susan Sontag once said that photographing someone turns that person into an object that can be symbolically possessed. Once obtained, this object (image) can be analyzed to extract information.

The first studies looking for a facial recognition machine date back to the 1960s. No major breakthroughs were made until the 1990s, when the US Department of Defense (DoD) initiated the Feret project, aimed at the acquisition, analysis and identification of facial features, collecting a database of about 8,000 images.

In comparison, FindFace currently has a database of half a million users of Vkontakte (Russian Facebook) that can be identified by uploading a photo to the application.

The technology that allowed this qualitative leap with the arrival of the new century was the Viola-Jones algorithm. Introduced in 2001, it is efficient, of good precision, requires little hardware, is economical and allows its use in light systems.

Since 2008, the combination of the Viola-Jones algorithm and Open CV (an artificial vision library) enables the use of cameras for the extraction of information from the images, greatly reducing human intervention in the process.

To have an order of magnitude of the amount of information that an image can contain:

• A 1 × 1 pixel box allows 256 different color combinations.
• A 7 × 6 pixel image already contains enough information to perform facial recognition with 95% reliability using the AT&T facial dataset.
• A region of 12 × 16 pixels allows the recognition of actions by training a neural network to interpret the extracted information.
• Open CV uses a format of 20×20 pixels (256^400 possible combinations), containing much more information than the cases previously exposed.
• 100 × 100 pixels ((100*100)^256)^3 possible combinations) is 2.5% of an Instagram photo. This gives a glimpse of the amount of information that an image uploaded to a social network can contain. That is why each image of a face uploaded to social networks like Instagram and Facebook is analyzed and knowledge is extracted from both the image and its associated metadata.
• In turn, analyzing a region of 100×100 pixels on the forehead of a person captured on video is possible to find out their pulse.

Figure 2: Richard McPherson, Reza Shokri, Vitaly Shmatikov, Region of 6×7 pixels used for facial recognition using the AT&T facial dataset, ElDiario.es. Defeating Image Obfuscation with Deep Learning. Image taken from: http://images.eldiario.es/fotos/informacion-suficiente-reconocimiento-facial_EDIIMA20170109_0587_19.jpg [Posted on 15/02/2017]

The question of the magnitude of the volume of information contained in an image allows techniques such as steganography, which allows to hide information within an image.

As for the information that can be extracted from the face of a person, there are several recent studies that affirm that it is possible the detection of criminals by the study of the facial features: “Automatic Inference on Criminality using Face Images” led by Xiaolin Wu and Xi Zhang of the Shanghai Jiao Tong University.

Researchers say that “they find some discriminant structural features for predicting criminality, such as lip curvature, the distance between the inner corners of the eye and the so-called nose-to-mouth angle. Above all, the most important discovery of this investigation is that the images of faces of criminals and non-criminals form two very different varieties“.

There are a lot of studies like this in which through the use of technologies such as neural networks, images of human faces are analyzed in order to answer various questions such as whether a person has leadership skills depending on the width of his mouth. In this area there is a company that is already making a name for the characteristics of its services: Faception.

The CEO of the Israeli company (Shai Gilboa) claims that there are facial features encoded in DNA and this allows them to automatically identify the personality, employment, IQ and even criminal inclinations of anyone in real time, claiming to have a ratio of success between 85 and 95%.

Its clients hope to use this technology for security applications, trying to identify individuals who might be criminals, pedophiles, terrorists, etc.

The company raises controversy and concerns about the possible indiscriminate arrest of innocent people. However, the company’s head of profiling (David Gavriel) argues that Faception is only a support tool that provides recommendations, identifying threats in potential terms, and making final decisions in the hands of customers.

Figure 3: Anonymous, Personality classifiers used by Faception, Faception. Image taken from: http://www.faception.com/our-technology [Posted on 15/02/2017] 

These statements invite reflection because the basic concept is not new. In the nineteenth century the pseudo-scientific theory of phrenology was widely disseminated, holding the possibility of character determination and personality traits, as well as criminal tendencies, based on the shape of the skull.

Many similar pseudoscientific disciplines had their moment in the past: personology, physiognomics, metoposcopy, chiromancy … All of them banished from science by being wrongly based on biological determinism to affirm that the destiny of a person is predetermined.

Figure 4: Anonymous, Diagram of the areas of the skull according to which attribute are related. Image taken from: http://omicrono.elespanol.com/wp-content/uploads/2016/01/PhrenologyPix.jpg [Posted on 15/02/2017]

Over the next few years, 50 billion devices are expected to be introduced to the internet, among which there are products such as Netatmo Welcome, an IoT camera with facial recognition that allows the identification of individuals and the visualization of video in streaming through the internet by using your mobile application.

The resurgence of ideas based on the psychological profile of people based on purely biological attributes, together with the creation of an environment where there are many devices with facial recognition capabilities, increases the risks that are already foreseen for the future IoT ecosystem that is currently brewing, as well as threats to the privacy of individuals.

New technologies allow us to solve a lot of problems, but above all allow us to answer a lot of questions. Machines do not discriminate, nor are they prejudiced, they simply do what they are programmed for. That is why, in order to make an appropriate and responsible use of emerging technologies, it is not only necessary to know how to ask the right questions, but also to correctly interpret the answers given to us, a task which is still the responsibility of humans who are part of the cycle.