Lessons from Joy Buolamwini

Joy Buolamwini is a computer scientist and founder of the Algorithmic Justice League who exposed severe racial and gender biases in commercial facial recognition systems. She coined the "coded gaze" to describe how human prejudices get built into artificial intelligence. This profile covers her research and advocacy to show how data reflects societal power dynamics and why civil rights must extend to the digital realm.

Part 1: The Coded Gaze and Algorithmic Bias

  1. On defining the coded gaze: "The coded gaze is my term for algorithmic bias, describing the ways in which the priorities, preferences, and prejudices of those who have the power to shape technology can create exclusion and discrimination." — Source: [Unmasking AI]
  2. On the illusion of neutrality: "Artificial intelligence has as much racial and gender bias as the people who create it." — Source: [Doha Debates]
  3. On who builds tech: "Who codes matters. The lack of diversity in the tech industry directly translates to a lack of inclusivity in the systems they build." — Source: [Algorithmic Justice League]
  4. On training data: "If we train machine learning models on datasets that primarily feature lighter-skinned, male faces, the system will naturally fail when presented with anyone outside that narrow demographic." — Source: [TED Talk]
  5. On the spread of bias: "The dangerous thing about algorithmic bias is that it scales prejudice. It takes a single human flaw and hardcodes it into systems affecting millions." — Source: [Unmasking AI]
  6. On fixing the problem early: "We cannot wait for AI systems to cause harm and then try to patch them. We have to audit the data and the assumptions before the code is ever written." — Source: [Algorithmic Justice League]
  7. On hidden discrimination: "Bias in algorithms is often invisible to the people being harmed. A human discriminator might show their prejudice, but an algorithm rejects you silently." — Source: [Coded Bias]
  8. On the limits of technical fixes: "AI will not solve discrimination, because the cultural patterns that say one group of people is better than another because of their gender, their skin color, the way they speak, their height, or their wealth are not technical." — Source: [Unmasking AI]
  9. On systemic inequality: "We cannot use AI to sidestep conversations about patriarchy, white supremacy, ableism, or who holds power and who doesn't." — Source: [Unmasking AI]
  10. On representation: "You cannot have a fair system if the very data representing the world completely ignores large segments of the population." — Source: [TED Talk]

Part 2: Facial Recognition and the Color of Surveillance

  1. On failing systems: "I discovered the hard way that some of the most advanced facial recognition software in the world couldn't see me until I put on a white mask." — Source: [TED Talk]
  2. On police use of facial recognition: "Deploying flawed facial recognition systems in law enforcement directly threatens the freedom and safety of Black and brown communities." — Source: [Congressional Testimony]
  3. On false arrests: "When an algorithm falsely matches a Black man's face to a crime he didn't commit, it is a life-altering violation of his rights." — Source: [Algorithmic Justice League]
  4. On surveillance as control: "We have to stop thinking of surveillance technologies as mere tools and start recognizing them as instruments of social control." — Source: [Coded Bias]
  5. On the burden of proof: "Citizens should not bear the burden of proving that an algorithm misidentified them; the creators of the algorithm must prove it is safe before deployment." — Source: [Unmasking AI]
  6. On mass data collection: "Harvesting millions of faces from the internet without consent to build a facial recognition empire is a fundamental violation of privacy." — Source: [TED AI]
  7. On biometric tracking: "Our faces are not public property. The increasing normalization of biometric tracking strips away our ability to move through the world anonymously." — Source: [Unmasking AI]
  8. On the urgency of regulation: "If we do not ban or strictly regulate law enforcement use of facial recognition, we are accepting a future of automated racial profiling." — Source: [Algorithmic Justice League]
  9. On precision versus justice: "Even if facial recognition becomes perfectly accurate for all skin tones, we still must ask if a perfectly precise surveillance state is a just society." — Source: [Unmasking AI]

Part 3: Intersection of Tech and Social Justice

  1. On defining algorithmic justice: "The rising frontier for civil rights will require algorithmic justice. AI should be for the people and by the people, excluding no one." — Source: [CBC]
  2. On power dynamics: "Technology is not objective; it is a manifestation of the power dynamics of the society in which it was created." — Source: [Unmasking AI]
  3. On the cost of efficiency: "We are often told that AI will make systems more efficient, but we must always ask: efficient for whom, and at whose expense?" — Source: [Coded Bias]
  4. On poverty and tech: "AI will not solve poverty, because the conditions that lead to societies that pursue profit over people are not technical." — Source: [Unmasking AI]
  5. On the illusion of a tech utopia: "Believing that more data and more compute will automatically result in a more equitable world is a dangerous fantasy." — Source: [Algorithmic Justice League]
  6. On community impact: "The communities most harmed by predatory AI are the ones least likely to have a seat at the table where the algorithms are designed." — Source: [TED Talk]
  7. On automation and labor: "When we automate decision-making in hiring and lending, we risk reinforcing historic redlining and employment discrimination under the guise of math." — Source: [Unmasking AI]
  8. On intersectionality: "Algorithmic harm often operates at the intersections of identity, disproportionately affecting dark-skinned women who fall outside the norm of the white male default." — Source: [Gender Shades Project]
  9. On fighting back: "We do not have to accept that if companies have already created a product it is a forgone conclusion that the product will be used." — Source: [Unmasking AI]

Part 4: The Concept of the "Excoded"

  1. On defining the excoded: "The excoded are those who are historically marginalized and find themselves excluded, erased, or harmed by the automated systems built without them in mind." — Source: [Unmasking AI]
  2. On erasure: "When a system fails to recognize your face, it is a literal erasure of your existence in the digital realm." — Source: [TED Talk]
  3. On inclusion vs. justice: "It is insufficient to simply add more diverse faces to a training dataset; we must ask if the system itself is inherently predatory toward the excoded." — Source: [Algorithmic Justice League]
  4. On automated rejection: "For the excoded, AI is the automated rejection letter for a job, the denied loan, and the falsely triggered fraud alert." — Source: [Unmasking AI]
  5. On vulnerability: "Those who are already vulnerable in the physical world are almost always the most vulnerable to algorithmic harm in the digital world." — Source: [Coded Bias]
  6. On demanding visibility: "We must demand that technology recognizes our humanity without simultaneously weaponizing our identities against us." — Source: [TED AI]
  7. On the digital divide: "The gap is no longer strictly about who has access to the internet; it is about who has the power to define the rules of the algorithms that govern the internet." — Source: [Unmasking AI]
  8. On centering the marginalized: "If we build systems that protect the most vulnerable among us, we inherently build safer systems for everyone." — Source: [Algorithmic Justice League]
  9. On digital redlining: "The excoded face a modern form of redlining, where their ZIP codes and demographics are used by algorithms to deny them opportunities." — Source: [Coded Bias]

Part 5: Corporate Accountability and Big Tech

  1. On self-regulation: "We cannot rely on Big Tech to self-regulate. When profit motives clash with human rights, profit inevitably wins unless there is external accountability." — Source: [Algorithmic Justice League]
  2. On meaningful transparency: "The public has a right to understand the processes behind AI creation and deployment. Black box systems are incompatible with democratic oversight." — Source: [Algorithmic Justice League]
  3. On the rush to deploy: "Companies are treating the public sphere as a beta-testing ground, deploying flawed systems and waiting for marginalized people to report the damage." — Source: [Unmasking AI]
  4. On auditing algorithms: "Independent audits of AI systems should be standard practice, never a public relations exercise conducted only after a crisis." — Source: [Gender Shades Project]
  5. On actionable critique: "Actionable critique means we must move beyond simply identifying the flaws in Big Tech; we must provide the frameworks for them to dismantle those flaws." — Source: [Algorithmic Justice League]
  6. On the role of the whistleblower: "It should not require a researcher to risk their career to prove that a billion-dollar company's algorithm is racist." — Source: [Coded Bias]
  7. On tech leadership: "The leaders of major tech companies often possess a blind spot regarding algorithmic harm because they have never experienced systemic discrimination themselves." — Source: [Unmasking AI]
  8. On data harvesting: "The business model of extracting our data to train models that are then sold back to us to police us is fundamentally exploitative." — Source: [TED AI]
  9. On algorithmic hygiene: "Tech companies must practice algorithmic hygiene by regularly checking their models for bias the same way they check their code for security vulnerabilities." — Source: [Algorithmic Justice League]
  10. On corporate pushback: "When you expose the flaws in a powerful company's algorithm, their first instinct is often to discredit the researcher rather than fix the code." — Source: [Unmasking AI]

Part 6: Policy, Regulation, and Biometric Rights

  1. On affirmative consent: "Individuals should have real choices regarding their interaction with AI systems. Opting out should not mean opting out of society." — Source: [Algorithmic Justice League]
  2. On biometric rights: "We need a comprehensive bill of biometric rights to protect our physical and behavioral data from being harvested and weaponized." — Source: [TED AI]
  3. On the role of government: "Lawmakers must step up to govern AI. We cannot let the speed of technological innovation outpace the speed of our civil rights protections." — Source: [Congressional Testimony]
  4. On vague definitions: "The vagueness of 'working for everyone' actually does a disservice if you're not willing to do the work to define that humanity and how we expand our understanding of whose stories are worthy." — Source: [Business Insider]
  5. On regulatory frameworks: "We need federal legislation that establishes clear boundaries on what constitutes an acceptable use of facial recognition technology." — Source: [Coded Bias]
  6. On continuous oversight: "AI systems must be subject to rigorous mechanisms that protect people and hold institutions accountable." — Source: [Algorithmic Justice League]
  7. On the dangers of deepfakes: "The proliferation of deepfakes threatens our shared sense of reality and requires urgent policy interventions to protect individuals from synthetic exploitation." — Source: [SXSW Keynote]
  8. On banning specific use cases: "There are certain applications of AI, such as predicting criminality based on facial features, that are so inherently flawed they should be outright banned." — Source: [Unmasking AI]
  9. On global standards: "Algorithmic justice is a global issue. We need international coalitions to establish human-rights-based standards for artificial intelligence." — Source: [Center for Humane Technology]

Part 7: Art, Poetry, and Humanizing Technology

  1. On the poet of code: "I call myself a poet of code because art and poetry allow us to communicate the emotional truth of algorithmic harm in a way that data tables cannot." — Source: [Dare to Lead]
  2. On Ain't I a Woman: "By invoking Sojourner Truth, I wanted to show that the historical erasure of Black women is now being codified into the algorithms of the future." — Source: [AI, Ain't I a Woman?]
  3. On the power of storytelling: "Statistics rarely change minds on their own. It is the human stories behind the data that force society to confront algorithmic injustice." — Source: [Coded Bias]
  4. On bridging disciplines: "We need artists, sociologists, and philosophers in the room when AI is being built alongside computer scientists and engineers." — Source: [Unmasking AI]
  5. On the emotional toll: "Discovering that a machine does not see you is a uniquely modern form of alienation that cuts to the core of your sense of belonging." — Source: [TED Talk]
  6. On the white mask: "Putting on a white mask so a robot could see my face was a visceral, theatrical demonstration of the absurdity and racism embedded in the code." — Source: [TED Talk]
  7. On reclaiming narratives: "Through spoken word and digital art, we can reclaim our narratives from the machines that seek to classify and diminish us." — Source: [Algorithmic Justice League]
  8. On human-centric design: "Technology should adapt to the diversity of humanity, rather than forcing humanity to conform to the narrow parameters of the technology." — Source: [Unmasking AI]
  9. On creative resistance: "Art is a reflection of reality, and in the face of algorithmic oppression, art becomes a vital form of resistance." — Source: [Vision & Justice Convening]

Part 8: The Future of AI and Human Agency

  1. On the path forward: "We can code a better future. It requires intention, accountability, and the courage to dismantle systems that do not serve us." — Source: [Algorithmic Justice League]
  2. On the illusion of alignment: "We cannot assume that AI is aligned with human values when the tech industry has not yet aligned itself with the basic principles of equity and civil rights." — Source: [Center for Humane Technology]
  3. On preserving human agency: "No matter how advanced AI becomes, the final decision over a person's liberty, livelihood, or life must always rest in human hands." — Source: [Unmasking AI]
  4. On redefining progress: "True technological progress is not measured by the speed of the processor, but by the equitable distribution of the technology's benefits." — Source: [Coded Bias]
  5. On youth and the future: "The next generation of computer scientists must be taught ethics as rigorously as they are taught calculus and programming." — Source: [Unmasking AI]
  6. On collective action: "Fighting algorithmic bias is a shared endeavor. It requires a coalition of researchers, policymakers, activists, and everyday citizens demanding change." — Source: [Algorithmic Justice League]
  7. On hope versus optimism: "I am not always optimistic about the trajectory of Big Tech, but I am profoundly hopeful about the power of everyday people to demand algorithmic justice." — Source: [Dare to Lead]
  8. On the legacy of our code: "The algorithms we write today are the digital infrastructure of tomorrow. We must ensure we are not building a foundation of prejudice." — Source: [TED Talk]
  9. On rejecting inevitability: "We do not have to accept a dystopian future. Technology is a human creation, and humans have the power to change its course." — Source: [Unmasking AI]
  10. On the ultimate goal: "Fixing the code is secondary; the primary aim is to create a society where technology is a tool for liberation, never a weapon of oppression." — Source: [Unmasking AI]