The real dangers of facial recognition: mass obedience and online labor exploitation

Facial recognition is a scam, but that doesn’t mean we should underestimate the threat it poses to freedom and fundamental human rights.

Case in point: Russian media MBKh discovered that Moscow police officers illegally “monetize” footage of 175,000 surveillance cameras on forums and messenger groups. For the equivalent of 470$, anyone can have access to facial recognition lookup services that, when provided with the picture of an individual, match it with passerby from hundreds of cameras, along with a list of addresses and times they were caught on camera. 

Interestingly enough, face recognition tech *does not work*, and the journalist has to grudgingly admit its low accuracy:

“As for the accuracy of the results, none of the photos returned were of the investigator. However, the facial features were similar to the input and the system assessed a similarity of 67%.”

According to the journalist, the explanation for this suboptimal performance is “the limited number of cameras connected to the face recognition algorithm”. Apparently, the sample is too small, but the technology is not fundamentally called into question…

Another explanation (often ignored by journalists, ever ready to believe the AI hype and therefore disregard its actual dangers) is introduced by a 2018 New York Times coverage of China’s surveillance-industrial complex: mass surveillance systems are not automated per se and are largely based on the intervention of crowds of micro-workers who cherry-pick millions of videos, cut out silhouettes of individual and tag metadata, fill in databases and annotate information:

“The system remains more of a digital patchwork than an all-seeing technological network. Many files still aren’t digitized, and others are on mismatched spreadsheets that can’t be easily reconciled. Systems that police hope will someday be powered by A.I. [emphasis added] are currently run by teams of people sorting through photos and data the old-fashioned way.”

“The old-fashioned way” here means “by hand”… Data annotation, triage, enrichment, especially for the computer vision models underlying face recognition algorithms, is a blossoming market. Recent research by Bonnie Nardi, Hamid Ekbia, Mary Gray, Sid Suri, Janine Berg, Six Silberman, Florian Schmidt, Trevor Paglen, Kate Crawford, Paola Tubaro and myself witnesses its development in sectors as diverse as home automation, transportation, advertising, health, entertainment… and the military. It is based on a workforce of hundreds of million of online laborers, alternatively called microworkers or crowdworkers. They work long hours, with precarious contracts and exploitative working conditions, and are paid very low wages (in some cases less than a cent per micro-task). Although they are attested in the global North, they are predominantly based in developing and emerging economies—such as Russia and China. But the companies that recruit them to package their annotated data and resell it as surveillance technologies, are mainly located in so-called liberal democracies. Despite the Chinese market supremacy, US, French, Japanese, Israeli and Finnish corporations are spreading these technologies all over the world, according to the 2019 AI Global Surveillance Index.

Despite the importance of these “humans in the loop” that constitute the secret ingredients of AI-based technological innovation, the threats of facial recognition, smart cities and predictive policing must not be minimized. The glorification of AI turns it into a powerful psychological deterrent and a disciplinary device. “The whole point,” explains an expert interviewed by the New York Times, “is that people don’t know if they’re being monitored, and that uncertainty makes them more obedient.”

Any action aimed to fight against the alleged omnipotence of these technologies begins with the recognition of their fictitious nature. If automated surveillance is made up of men and women who train, control and impersonate “artificial artificial intelligence“, it is from the awareness of their role in a dystopian and inhuman system that a change is going to come.

Chinese media about “Qu’est-ce que le digital labor ?” (Oct. 3, 2015)

After a press release by Taiwanese agency CNA, several Chinese-speaking media outlets have been discussing the central theses of our book “Qu’est-ce que le digital labor ?” (INA, 2015).

臉書廣告賺2000億 義學者:用戶都是免費數位勞工

▲義大利學者卡西立Antonio Casilli。(圖/翻攝自Antonio Casilli推特)


社交媒體臉書(Facebook)已經成為多數人生活中不可或缺的一部分。義大利學者卡西立(Antonio Casilli)表示,網路的使用已經成為數位工作的一種;在臉書發文、按讚、分享都具有商業價值,讓業者荷包滿滿,用戶其實已淪為免費的數位勞工。





卡西立還說,不能因為大家樂意使用臉書,就否認這是工作,因為感覺到快樂也是促進生產力的誘因之一。他認為,網路工作是新型的認知資本主義(cognitive capitalism),全面滲透在日常生活,模糊了家庭、工作的界線,也引發隱私問題;因為用戶很難得到合理的報酬,卡西立主張,應該要向科技大公司課稅,然後提供每個人基本工資保障。

另一名社會學家胡斯(Ursula Huws)則指出,當前資本主義讓過去非商品的社會關係,也進入經濟範疇、有了利潤空間,科技也瓦解泰勒化生產模式,例如優步(Uber)帶來便利,但工 作業更不穩定;科技便利讓工作零碎化,甚至隱形化,新無產階級誕生,但生產者尚未自覺,仍以為自己是占便宜的消費者。



Source: 臉書廣告賺2000億 義學者:用戶都是免費數位勞工 | ETtoday國際新聞 | ETtoday 新聞雲

Tableau des équivalences web occidental / web chinois

Il y a quelques mois, lors de la conférence Lift10, l’auditoire a été capturé par le brillant exposé de Basile Zimmermann. Le jeune professeur de l’Université de Genève a expliqué – d’une manière extrêmement convaincante – comment la différence culturelle entre la Chine et les sociétés euro-étasuniennes soit encodée dans le langage et dans les pratiques d’écriture. Et quand les usages technologiques s’en mêlent, l’écart peut se creuser encore davantage. Les  claviers,  les écrans, et les conventions communicationnelles opposent radicalement la manière de lire des contenus en ligne en Chine et dans les pays “alphabetiques”.

Certes utile pour se repérer dans le web chinois, le tableau concocté par l’expert de médias sociaux Thomas Crampton, doit IMHO être lu à l’aune des commentaires de Basile Zimmermann – qui nous invite à ne pas réduire la différence culturelle à un simple jeu d’équivalences.


Summer readings, cultural revolutions, destructive designs

My interest in the topic of the Chinese Cultural Revolution was jump-started by Tsinghua sociologist Guo Yuhua. In the summer of 2008, in the aftermath of a month-long fieldwork conducted in Beijing and Shanghai, I came back to Paris to attend the conference La Chine et l’internationalisation de la sociologie. There, Guo Yuhua delivered a presentation about political rituals in rural China, emphasizing the role of “movements” as being instrumental in creating a certain form of emerging governance in remote provinces. By movements, Chinese authorities traditionally mean loosely-designed public campaigns promoting ever-changing (and often contradictory) policies: movements to “save the country through physical fitness”, movements to “chase away sparrows”, movements to “voice dissent”, movements to “repress dissent”, movements to “kill and bury stray dogs”, and so on. Something like western democracies national plans, but less clear as to scope, budget and timing, and more bottom-up and arbitrary in their application: “is one hour of exercise per day sufficient to stay healthy?”; “on what exactly should I voice my dissent?”; “how many dogs do we have to kill, overall?” All these questions are not answered by Chinese  policymakers. Rather, the answer is supposed to emerge consensually, after a period of collective negotiation sometimes leading to tensions, struggle and social criticism.

The idea that popped in more than one head that day, while listening to Guo Yuhua, was that maybe the long series of disruptive political events that we conventionally call the “Cultural Revolution” should not be regarded as a coherent political masterplan, but as the random combination of some of those campaigns – starting with the “Destruction of the Four Olds” in 1966, peaking with the “Down to the Countryside movement” in the early 1970s, and fading away after the “Criticize Lin Piao, Criticize Confucius” movement in the mid-1970s.