This is Who made our minds?, my free Thursday essay probing the greatest, cruellest and most beautiful minds of the past 5,000 years, inspired by my book, The Soul: A History of the Human Mind (Penguin 2024). Just insert your email address below and click on the free option. Or you can pay a little to help me help you discover ‘who made our minds’. Coming up: Theocracy versus Democracy; Leonardo da Vinci’s eye; Dinner with Judas
THE CARTESIAN separation of ‘mind’ and ‘body’ is happening, but not in the way Descartes imagined.
Billions of human minds or ‘digiselves’ now exist in the digital realm, detached from the plodding, earthly existence of their bodies, or ‘bioselves’.
Our minds are now treated as billions of data points, that can be sold, imitated, stolen, manipulated or hacked.
Techno giants and governments prefer to deal with your digiself because that is the active agent who does your buying, signing, engaging, filling in official forms, paying taxes and so on. Your bioself, your flesh and blood, is the passive recipient of commands initiated by your digiself, in accordance with the wishes of governments and corporations.
There is no physical ‘you’ in this chain of interactions. Techno firms and governments recognise only your digiself, your avatar, which used to be called your mind.
Many people live entirely in and for their digiselves, leaving their bioselves in spatial limbo. I know of some who felt they ceased to exist when social media companies like Meta closed their accounts. One said she felt as if she’d been murdered.
Aware that our bodies will grow old and die, many of us are hoping to perfect our digiselves, in the belief that we will live forever in the cyber realm. A kind of immortality is within reach if AI systems succeed in copying the conscious mind and creating an eternal ‘you’.
—
Artificial intelligence systems, troll farms, facial-recognition technology and microchip implants aggregate data selves and seek to control them, often in seemingly innocuous ways, by digitalising your financial and tax affairs, monitoring your voting patterns, cross-referencing your buying and ‘liking’ habits, sharing your medical records and so on. In a benign world, much of this is helpful and well-intentioned. In a malign world, it is a Kafkaesque nightmare.
In China, for example, government-run neuronal and facial-recognition systems collate and manipulate the lives of hundreds of millions of digiselves.
Shortly after assuming the presidency in 2013, Xi Jinping made a speech about data harvesting. ‘The vast ocean of data,’ he said, ‘just like oil resources during industrialization, contains immense productive power and opportunities. Whoever controls big data technologies will control the resources for development and have the upper hand.’
By the early 2020s, the Chinese Communist Party was close to fulfilling this goal. It had taken ‘possession’ of the data self of every Chinese citizen, and of countless foreign citizens. The Data Security Law and the Personal Information Protection Law, both enacted in 2021, gave the regime secret access to citizens’ private data and therefore complete control over their lives.
The Chinese state became a monstrous data vacuum, sucking up vast amounts of domestic (and, increasingly, overseas) personal data, while sealing off Chinese data from the rest of the world.
If Western countries tend to confine such powers to agencies tracking terrorists, spies and criminals, the Chinese regime uses them to track everyone. The Chinese Communist Party has thus become the world’s largest spy, and the target of its spying activities is its own people.
—
China’s creation of a nation of spooks follows a rich history of domestic espionage, which long pre-dated Mao’s police state.
The ancient custom of baojia, for example – an embedded system of ‘neighbourhood’ vigilantism – has produced regiments of local snitches throughout Chinese history.
The baojia system was invented during the Qin Dynasty (221–206 CE), was revived in the Song Dynasty (960–1279 CE) and was ‘perfected and used on a large scale’ during the Qing Dynasty (1636–1912), ‘to maintain control and effectively rule’ the people, writes Yi-Zheng Lian, a former chief editor of The Hong Kong Economic Journal. Today’s Communist Party ‘merely added the digital cameras’. A Chinese ‘Big Brother,’ he concludes, ‘has been watching for 2,000 years.’
What is new is the reach of China’s digital spies, as the scholars Anne Cheung and Yongxi Chen show in their astonishing 2022 survey of China’s Social Credit System (SCS). In the early 2000s, the Social Credit System was a standard financial credit system. Today it is a tentacular economic, social and moral surveillance system, a window into the mind of every Chinese citizen.
In China everyone has a unique data- or digiself, which the SCS continually monitors and evaluates. It does this by expanding the concept of financial credit to the moral and social standing of the people. China thus ‘ranks’ every citizen according to their social and political utility – or lack of it – rather as we might ‘star’ or ‘like’ movies, restaurants and books.
Citizens, for example, who engage in ‘antisocial’ behaviour, such as protesting and criticising the government, receive a low rank (‘one star’, as it were), quietly excluding them from job advancement or restricting their educational options, or worse. Self-censorship and drone-like obedience to the government are rewarded with higher ranks (more stars).
China has thereby become the world’s first ‘datatorship’, a novel kind of tyranny in which faceless officials or government trolls never meet their victims and are never held accountable for hobbling the lives of millions by manipulating their data selves. In this sense, one might call modern China a ‘Datatorship of the Trolletariat’.
There is no escaping this system. Every digital word, blog and image; every like, star, review and chat; every bid, purchase, sale; every digital crumb you leave behind will be hoovered up and tagged to your digital self. ‘Anonymity in the digital environment is almost impossible in China,’ write Cheung and Chen. You exist only insofar as your digital self exists. ‘The end result of the data self’s dominance is the bio-self’s growing dependence on the state.’
Even if a Chinese citizen were able to challenge an incorrect credit record or an unfair character assessment, Cheung and Chen report, ‘he or she remains unable to challenge the overall digital iron cage of subordination, particularly when that cage has been built by a state that monopolizes coercive power’.
Like some vast parasitic excrescence, then, the Beijing regime preys on the Chinese mind, forcing the people to conform to the state’s template of the ‘ideal citizen’.
Desperate to survive, and to ensure their children’s future, countless numbers of people try to anticipate the thoughts and actions of this ideal citizen, whose profile the state updates according to the political exigencies of the moment.
This process culminates in the spectacle of hundreds of millions of Chinese people trying to behave how they think the state wants them to, not how their own minds would like to behave.
As such, the Chinese state has achieved something truly historic. It has resolved the millennia-old mystery of free will (as defined by religions and philosophers): in China, most people have no free will, because they’re enslaved to the ideal digital profile imposed on them by the state.
Resistance is futile, because any sign of divergence from the ideal profile appears immediately on your data trail and will be tracked. Under Nazi Germany and Stalinist Russia, cells of secret resistance were possible. Not in modern China, the first datatorship to make secrecy impossible, and the hope of free will inconceivable.
—
Citizens of datatorships have nowhere to hide because everyone will be electronically tagged, with advanced forms of facial recognition and radio frequency identification (RFID) technology. The latter allows the state to identify a person, animal or object instantly, using electromagnetic radio waves emanating from an embedded microchip.
The embedded microchip is a rudimentary form of ‘biohacking’, to which not everyone is hostile. Many people outside China have volunteered to be biochipped, by carrying an RFID microchip – about the size of a rice grain – beneath the skin on the hand, a quick swipe of which unlocks doors, starts the car, pays a train fare, extracts cash and verifies their identity.
Microchipping the mind heralds the era of brain surveillance, warns Nita Farahany, a Duke University professor and author of The Battle for Your Brain (2023). Brain surveillance involves hacking the conscious mind with ‘wearable devices’ that measure electrical activity in the brain and musculature, producing real-time displays of your brain activity and bioelectric changes in your muscles.
Western companies are applying the technology in supposedly benign ways by integrating neural interfaces with employees’ watches, headphones, earbuds, hard hats, caps and VR headsets ‘to monitor fatigue, track attention, boost productivity, enhance safety, decrease stress, and create a more responsive working environment’, according to one study.
The technology is called ‘brain transparency’ because it exposes the mood and inclinations of the mind and how that affects the body. Brain sensors can assess how tired you are, whether you’re wandering on the job, or how long you spend in the bathroom or on a coffee break. Every moment in your working day is monitored. Whether you look attentive won’t help you if your brainwaves show your mind is slacking.
‘You can use those displays to “see” your emotions, your arousal, and your alertness,’ explains Farahany. ‘You can learn if you’re wired to be conservative or liberal; whether your insomnia is as bad as you think; whether you’re in love or just in lust.’ You can even track the onset of mental illnesses or dementia, and get advance warning of epileptic seizures.
But hacking the mind will also give employers access to ‘incidental information’, with which they can discriminate against, or weed out, ‘unproductive’ employees: perhaps those with early signs of cognitive decline, or mood and attention ‘disorders’, like daydreaming, which is a tendency common to all creative minds. Albert Einstein and Isaac Newton both recognised that the greatest creative ideas often arise from the wandering minds of daydreamers.
Daydreaming or not, mental downtime is essential because it also leads to greater productivity, according to recent research across 900 Boston Consulting Group teams in thirty countries. If workers know their minds are being monitored they become anxious to stay ‘on task’, to avoid losing their jobs. They start to crack up. This can only lead to a vicious cycle: mind-hacking = high stress = poorer long-run performance = early burnout.
—
All dictatorshiops (and datatorships) prey on the young, and China is the first to test-drive the new technology in the classroom. Young minds are being monitored by facial-recognition systems, electroencephalogram (EEG) headsets (that monitor the brain’s electrical activity), and cameras and microphones that track the moment-by-moment changes in a child’s expressions, emotions and attention levels.
The point is to track a child’s brainwaves, to test whether they’re ‘on-task’ or ‘off-task’. Colour codes or symbols designate the child’s attention level. Drowsy or distracted children are given a poor rank, and are reported and reprimanded.
These ‘artificially intelligent’ classrooms alert the teacher to the moment a child loses focus (a natural enough thing for a child to do).
Computer scientist Sharon Oviatt has completed a penetrating analysis of China’s AI classrooms. She finds children ‘being punished for independent thinking [and] creative thought’. Such ‘invasive surveillance’ will undermine ‘children’s autonomous control, self-regulation, motivation to learn in the future, trust in authority, independent ideas, ability to think creatively, and lifelong learning’. Nor is there any privacy protection: everyone has access to a child’s level of ‘on or off-task’ attentiveness.
Good parents, we naively suppose, would reject neural monitoring as an outrageous intrusion on a child’s freedom to think, play and develop naturally. Not so: the AI software systems are ubiquitous in Chinese schools, and Chinese companies are exporting them to the West, a grim reflection of what many parents and educators seem to care about most: their school’s ranking in the league tables.
The technology, however, has failed on its own terms, as Oviatt concludes: there is no evidence that measuring the brain’s attentiveness during real-time learning is ‘a reliable science’, and there is ‘no single brain pattern associated with optimal learning’.
Hacking a child’s brain is, in sum, the latest monstrous intrusion on human biological integrity. Sold to parents and schools as a means of improving a child’s intellectual performance, it merely denies the young brain’s freedom to imagine and think and creatively wander.
The brain is the last unconquered frontier of the human anatomy, and China, Russia and emerging datatorships are leading the invasion.
Neural interfaces will give them absolute control over their citizens, by placing the mind in the crosshairs of the state and creating nations of human robots or ‘humbots’.
Of what use are re-education camps, torture and inquisitions if you have the technology to read a person’s mind? The human thought police may lose their jobs in the coming datatorships, but that’s moot consolation if a sensor or chip can do the job better.
Next Thursday, 24th October 2024: Theocracy v Democracy
Selected sources and further reading:
Backer, L.C. (2019) ‘China’s Social Credit System: Data-Driven Governance for a “New Era”’, Current History, 118(809), pp. 209–14.
Backer, L.C. (11 October 2019) ‘Blacklists and Social Credit Regimes in China’, paper presented at the Interdisciplinary Conference ‘Super-Scoring? Data-driven Societal Technologies in China and Western-style Democracies As a New Challenge for Education’, Cologne.
Barron, D. (2021) Reading Our Minds: The Rise of Big Data Psychiatry, New York: Columbia Global Reports.
Cheung, A.S.Y. and Chen, Y. (9 December 2021) ‘From Datafication to Data State: Making Sense of China’s Social Credit System and Its Implications’, Law and Social Inquiry, 47(4), pp. 1137–71.
Dai, X (2018) ‘Toward a Reputation State: The Social Credit System Project of China’, Social Science Research Network.
Farahany, N. (2023) The Battle for Your Brain: Defending the Right to Think Freely in the Age of Neurotechnology, New York: St Martin’s Press.
Lian, Y-Z. (1 July 2021) ‘The Chinese Communist Party Is 100. It’s Not Going Anywhere’, The New York Times.
McCay A. and Sharpe, M. (12 April 2023) ‘What Is Neurotechnology and Why Are Lawyers Getting Involved?’, LSJ Online.
Montag, C., Elhai, J.D. and Davis, K.L. (June 2021) ‘A Comprehensive Review of Studies Using the Affective Neuroscience Personality Scales in the Psychological and Psychiatric Sciences’, Neuroscience and Biobehavioral Reviews, 125, pp. 160–7.
Mozur, P. and Krolik, A. (17 December 2019) ‘A Surveillance Net Blankets China’s Cities, Giving Police Vast Powers’, The New York Times.
Oviatt, S. (October 2021) ‘Technology As Infrastructure for Dehumanization: Three Hundred Million People with the Same Face’, ICMI ’21: Proceedings of the 2021 International Conference on Multimodal Interaction, pp. 282–3.
Pottinger, M. and Feith, D. (30 November 2021) ‘The Most Powerful Data Broker in the World Is Winning the War against the U.S.’, The New York Times.
Sellers, B.G. (2021) ‘Global Surveillance: The Emerging Role of Radio Frequency Identification (RFID) Technology’, in Arrigo, B.A. and Sellers, B.G. (2021) The Pre- Crime Society: Crime, Culture and Control in the Ultramodern Age, Bristol: Bristol University Press.
Spielkamp, M. and Brigitte, A. (eds.) (2019) ‘Automating Society 2019’, AlgorithmWatch.
The Wall Street Journal, AI in Chinese classrooms, video: https://www.wsj.com/video/series/in-depth-features/under-ais-watchful-eye-china-wants-to-raise-smarter-students/C4294BAB-A76B-4569-8D09-32E9F2B62D19
Zhu, H., Huang, X. and Chen, Y. (2019), ‘The Index System of Personal Credit Scoring in Chinese Cities and Its Application Scenarios’, Credit Reference, 37(4), pp. 9–15.
Zuboff, S. (2019) ‘The Age of Surveillance Capitalism’, Public Affairs.