In the introductory pages of The Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence, Kate Crawford describes artificial intelligence as a political “registry of power” (p. 8) that is simultaneously symbolic and material in nature.
Stemming from the fifteenth-century term for describing an official account, “registry” implies an essence of formality and authorization, as something recorded that should be seemingly taken as true. Yet perhaps a more accurate description of the term lies in its sociocultural and political implications. On one hand, a registry is a fundamental reflection of that deemed worthy enough to record at the time of its making. On the other hand—and as Crawford’s own examples later illuminate—“worthiness” itself depends on sociohistorical constructions of identity and heritage and the ways these categories have shifted over time.
The motivations behind such categorization practices are the subject of Crawford’s fourth chapter, “Classification.” As its opening reflections on nineteenth-century craniometry remind us, scientific instruments and methods are far from objective; rather, the samples, tools, and methods chosen in a given experiment inevitably reflect the dominant viewpoint and prejudices of their time. Samuel Morton’s personal endorsement of polygenism, for example, inherently shaped his presumably objective findings. After collecting, measuring, and, importantly, selectively classifying over a thousand human skulls, Morton proposed a direct correlation between skull volume and intelligence, as well as a top-down hierarchy of skull size that categorized the skulls of white subjects above all others.
As later assessments have shown, while Morton’s individual calculations were shoddy at best, the classification practices he employed were themselves established by—and consequently (re)legitimized—the racial logics of the era, and many of today’s AI practices are built upon these archaically perpetuated tendencies. Therefore, the very practices of classification merit more scrutiny than the current debates on AI bias allow. As the statistical definition of “bias” implies some level of variance or error, Crawford stresses, critical narratives that focus on AI bias alone are far too narrow and ultimately cyclical in scope: the “bug” in the system is, more often than not, an integral and “self-reinforcing” feature of classification that ends up “[amplifying] social inequalities under the guise of technical neutrality” (p. 131).
In other words, much like Morton’s experiments, AI data sets will always yield political perspectives similar to those that underpin their respective technical systems. Machine learning presumably constitutes a kind of baseline of truth, supposed ‘outliers’ are read and subsequently treated as bias, results are integrated into future systems, and so the cycle goes.
Crawford invokes some of the companies explored in earlier chapters to further explore these patterns and circular logic. Amazon’s early attempts to automate its hiring process, for instance, inadvertently exposed the company’s patterns of gender bias. Because Amazon used a decade’s worth of résumés from its own engineer employees to develop its system’s dataset, the hiring recommendations that followed reflected what had previously constituted a successful Amazon engineer across the company’s history, and all female candidates were thus passed over. Though Amazon eventually determined this iteration of AI an overly biased failure, Crawford argues that this system was in fact an “intensification of the [company’s] existing dynamics” that has clearly continued into the present. As recent headlines suggest, gender bias remains an issue across Amazon’s various job sectors and pay scales.
Chapter 4 also returns to ImageNet, the AI visual recognition training set that was first introduced in the text’s previous chapter on data, and places it in conversation with other visual datasets produced by IBM (Diversity in Faces) and the University of Tennessee at Knoxville (UTKFace). As Crawford us, training datasets are similar to encyclopedias in that they do not replace but rather add to or overlay older classifications and collections, and this is clearly seen through the binaristic ways machine learning systems approach race and gender: while “male” and “female” bodies are implicitly nested under ImageNet’s “natural” categories (e.g., “Natural Object → Body → Human → male body”), for instance, images of those who do not identify as one or the other are either classified under a category of sexuality (e.g., “Person → Sensualist → Bisexual → . . . ) or are absent altogether (p. 138). UTKFace, a visual database geared toward automated face detection and age recognition, (re)produces similar fixed reductions: gender is classified into two categories (male or female), and race is neatly arranged into five (White, Black, Asian, Indian, and Others).
Though some of ImageNet’s more offensive categories have been removed over the years—and this process is documented in the 2019 paper entitled “Towards Fairer Datasets”—Crawford insightfully argues that, much like Morton’s craniometry practices, the “gaps and contradictions” of classification lie in not the indisputable outdatedness of some of the terms chosen but in the processes themselves. She writes:
The focus on making training sets ‘fairer’ by deleting offensive terms fails to contend with the power dynamics of classification and precludes a more thorough investment of the underlying logics. Even if the worse examples are fixed, the approach is still fundamentally built on an extractive relationship with data that is divorced from the people and places from whence it came. Then it is rendered through a technical worldview that seeks to fuse together a form of singular objectivity from what are complex and varied cultural materials.
Kate Crawford, Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence, 143
This passage encapsulates the strengths of both this chapter and the rest of the text in a number of ways. In some sense, The Atlas of AI is a kind of digital media prehistory that evokes the methodological work of Tung-Hui Hu, among others. Hu’s own prehistory takes up the digital cloud as both a historical object and a “resource-intensive extractive technology” (Hu 2015, 146), and Crawford’s literary atlas similarly examines AI’s extractive and all-consuming planetary infrastructure.
At the same time, this chapter employs interdisciplinary discourse deriving from science and technology studies (Geoffrey Bowker and Susan Leigh Star), linguistics (Gary Lakoff) and surveillance studies (Simone Browne) to account for the ways that historical and extractive logics concretely emerge in the ‘real,’ or offline, world.
By drawing a through line from Morton’s craniometry to Browne’s notion of “digital epidermalization” in particular (which accounts for the digital “imposition of race on the body), Crawford demonstrates how imperial logics and the “affordances of tools”—analog, digital, or otherwise—merge together to create larger classificatory schemas and what is eventually taken as the universal “horizon of truth” (p. 133). And by revealing the hidden, political, and ofttimes harmful motivations behind these past and present classification schemes, Crawford ultimately challenges us to critically consider how such practices will inevitably dictate the future(s) of, and relationship between, global companies, users-turned-biometric-profiles, and the ‘natural’ order of society itself.
Peer Reviewed by Sharon Musa
References
Downey, Anthony. “Trevor Paglen: On ‘From Apple to Anomaly,’” Barbican, September 26, 2019, https://sites.barbican.org.uk/trevorpaglen/.
Harvey, Adam and Jules LaPlace. “IBM Diversity in Faces,” Exposing.ai (2021): https://exposing.ai/ibm_dif/.
Hu, Tung-Hui. A Prehistory of the Cloud (Cambridge: MIT Press, 2015).
“Morton Cranial Collection,” Penn Museum, accessed March 20, 2024, https://www.penn.museum/sites/morton/life.php#_ftn79.
Noel, Melissa. “Penn Museum Faces Criticism After Burying Remains of 19 Black People Used in Racist Scientific Research Without Community Input,” Essence, February 5, 2024, https://www.essence.com/news/penn-museum-criticism-burial-remains-black-people/.
Palmer, Annie. “Amazon Sued by Three Employees who Allege Gender Discrimination and ‘Chronic’ Pay Inequity,” CNBC, November 20, 2023. https://www.cnbc.com/2023/11/20/amazon-sued-by-three-employees-who-allege-gender-discrimination.html#:~:text=Three%20staffers%20in%20Amazon%27s%20corporate,with%20higher%20titles%20and%20salaries.
“UTKFace: Large Scale Face Dataset,” GitHub Pages, accessed March 20, 2024, https://susanqq.github.io/UTKFace/.
Yang, Kaiyu et al., “Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy,” (FAT* ’20): Proceedings of the Conference on Fairness, Accountability, and Transparency (January 2020): 547–558, https://doi.org/10.1145/3351095.3375709.