Computer scientist Amy X. Zhang is an associate professor at the University of Washington’s Paul G. Allen School of Computer Science & Engineering, where she heads the Social Futures Lab and builds tools that give internet users more agency. Through systems like Squadbox and PolicyKit, her research allows communities to handle their own content moderation and governance. This profile covers her approach to fixing online discourse and creating democratic digital spaces. Reference: PolicyKit: Building Governance in Online Communities

Visual summary of operating lessons from Amy Zhang.

Part 1: Content Moderation and Online Harassment

  1. Limits of Centralized Moderation: Platform-wide, centralized moderation rules fail to capture the nuanced contexts and vulnerabilities of marginalized communities, necessitating user-empowered and personalized filtering controls. — Personalizing Content Moderation on Social Media / CSCW 2023.
  2. Friendsourced Email Defense: Crowdsourcing inbox moderation to a trusted personal support network allows individuals facing harassment campaigns to shield themselves from toxic content without forfeiting access to vital communications. — Squadbox: A Tool to Combat Email Harassment / CHI 2018.
  3. Distributing the Emotional Burden of Moderation: Content moderation imposes severe psychological strain; collaborative tooling must distribute this emotional labor across peer defenders while mitigating secondary trauma. — Squadbox: A Tool to Combat Email Harassment / CHI 2018.
  4. Relational Context in Harassment Detection: Automated filters routinely miss abusive behavior because harassment is deeply contextual, frequently turning on interpersonal history and subtle linguistic cues invisible to global classifiers. — Survey of Multi-Platform Online Harm and Interventions.
  5. Balancing Exposure with Safety: Designers face an inherent tension between fostering ideological discovery and safeguarding users from harassment, demanding granular controls that let individuals set their own boundaries. — Personalizing Content Moderation on Social Media / CSCW 2023.
  6. Collective Defense Against Coordinated Harassment: When targeted abuse escalates into organized brigading, single-user block buttons collapse under the volume, necessitating coordinated, multi-user defense workflows. — Squadbox Project.
  7. Algorithmic Blind Spots in Slang and In-Group Nuance: Global machine-learning classifiers trained on generic corpora routinely misinterpret community-specific argot, missing subtle microaggressions while erroneously penalizing benign in-group speech. — Personalizing Content Moderation on Social Media / CSCW 2023.
  8. Proactive Architecture Against Networked Harm: Platform architectures must anticipate coordinated brigading and harassment surges structurally, rather than relying exclusively on post-incident reporting that leaves users harmed. — Survey of Multi-Platform Online Harm and Interventions.

Part 2: Community Governance and PolicyKit

  1. Community-Authored Governance Systems: Online communities require expressive software tools to author custom governance rules rather than being confined to the static, top-down administrative hierarchies imposed by platform hosts. — PolicyKit: Building Governance in Online Communities / UIST 2020.
  2. Making Software Governance Malleable: Because platform code acts as de facto law, governance frameworks must make software rules malleable, enabling communities to evolve policies through democratic deliberation. — PolicyKit Project.
  3. Escaping Administrative Monoculture: Most social platforms default to a binary monarchical structure of all-powerful administrators and passive users, ignoring the rich diversity of human civic and cooperative traditions. — Modular Politics: Toward a Governance Layer for Online Communities / CSCW 2021.
  4. Executable Community Policies: Translating informal community norms into executable procedures ensures that policy triggers, reviews, and sanctions execute transparently and predictably across platform actions. — PolicyKit: Building Governance in Online Communities / UIST 2020.
  5. Dynamic Evolutionary Permission Models: Healthy online collectives must be able to alter their constitution over time through built-in amendment procedures that adjust permissions without crashing community operations. — Modular Politics: Toward a Governance Layer for Online Communities / CSCW 2021.
  6. Civic Ownership Through Participatory Rulemaking: Involving members directly in authoring community guidelines fosters civic responsibility and collective stewardship, substantially reducing compliance friction. — What Makes Online Communities 'Better'? / ICWSM 2022.
  7. Procedural Transparency in Sanctions: Democratic moderation systems must generate explicit, auditable records explaining why specific actions were approved or restricted, diminishing perceptions of administrative bias. — PolicyKit Project.
  8. Grounding Digital Governance in Offline Institutions: Digital governance tools achieve lasting adoption when their technical primitives mirror proven offline democratic mechanisms, such as juries, town halls, and separation of powers. — Comparing Perceived Legitimacy of Content Moderation Processes / CSCW 2022.
  9. Modular Primitives for Democratic Experimentation: Decoupling governance logic into composable modules allows communities to experiment with diverse democratic models, ranging from randomly selected juries to supermajority consensus. — PolicyKit: Building Governance in Online Communities / UIST 2020.

Part 3: Decentralization and User Control

  1. Decentralized Autonomy and Pluralism: Decentralized architectures grant communities the autonomy to define bespoke moderation policies and selection algorithms free from centralized corporate mandates. — Personalizing Content Moderation on Social Media / CSCW 2023.
  2. Administrative Burdens in the Fediverse: While federated social networks prevent monopolistic control, they frequently shift an unsustainable burden of legal, technical, and emotional triage onto volunteer instance administrators. — Modular Politics: Toward a Governance Layer for Online Communities / CSCW 2021.
  3. Data and Graph Portability Across Platforms: Effective decentralization requires seamless data and identity portability, ensuring users can migrate social graphs and personal archives without sacrificing community ties. — Survey of Multi-Platform Online Harm and Interventions.
  4. Marketplaces for Pluralistic Feed Algorithms: Users should have access to diverse algorithm marketplaces, allowing them to choose recommendation policies tailored to learning, deliberation, or relaxation rather than passive engagement. — Personalizing Content Moderation on Social Media / CSCW 2023.
  5. Client-Side Middleware and User Overlays: Client-side middleware and browser overlays provide vital functionality—such as custom filters, collaborative annotations, and moderation lenses—without requiring platform permission. — Eyebrowse: Social Web Activity Tracking / CSCW 2014.
  6. Preserving In-Group Norms via Local Context: Federated networks enable localized conversational norms to flourish, protecting benign in-group communication from context collapse and hostility in broader public arenas. — What Makes Online Communities 'Better'? / ICWSM 2022.
  7. Accountability Through the Power of Exit: Low migration barriers and protocol-level exit mechanisms hold instance administrators accountable to community consensus far more effectively than platform monopolistic lock-in. — Modular Politics: Toward a Governance Layer for Online Communities / CSCW 2021.
  8. Sustainable Non-Surveillance Infrastructure: Sustaining democratic digital public spaces requires viable collective financing models that break reliance on behavioral surveillance advertising and speculative token mechanics. — AI and the Future of Digital Public Squares.
  9. Bridging Federated Architectures and Usability: Federated protocols demonstrate the viability of decentralized social architecture, but mainstream adoption hinges on developing intuitive onboarding, discovery, and governance interfaces. — Modular Politics: Toward a Governance Layer for Online Communities / CSCW 2021.

Part 4: Making Sense of Large Discussions

  1. Recursive Synthesis for Scale (Wikum): Massive online discussion threads suffer from extreme redundancy; recursive summarization tools enable readers to collaboratively condense thousands of comments into structured artifacts. — Wikum: Bridging Discussion Forums and Wikis / CSCW 2017.
  2. Micro-Tasking for Community Knowledge Distillation: Community members will voluntarily contribute to synthesizing large discussions when interfaces decompose summarization into lightweight, bite-sized micro-tasks. — Wikum: Bridging Discussion Forums and Wikis / CSCW 2017.
  3. Distilling Ephemeral Chat Decisions (Tilda): While group chat excels at immediate back-and-forth conversation, it obscures important conclusions; lightweight tagging tools capture agreements and action items before they scroll out of view. — Tilda: Making Sense of Group Chat / CSCW 2018.
  4. Visualizing Multi-Branching Discourse Trees: Replacing simple indented text trees with interactive visualizations helps participants track competing debate subthreads without cognitive overload. — Wikum: Bridging Discussion Forums and Wikis / CSCW 2017.
  5. Reducing Communication Catch-Up Overhead: Catching up on hundreds of unread messages imposes heavy cognitive and time penalties; structured summaries allow returning teammates to quickly grasp key outcomes. — Tilda: Making Sense of Group Chat / CSCW 2018.
  6. Hybrid Human-AI Debate Summarization: Fully automated text summarizers tend to smooth away minority dissent; hybrid workflows where human editors steer algorithmic distillation preserve essential nuances and conflicting stances. — Wikum: Bridging Discussion Forums and Wikis / CSCW 2017.
  7. Curating Collective Intelligence Across Forums: Without structured distillation frameworks, the collective problem-solving and deliberation generated across online forums remains trapped in inaccessible archives. — Mailing Lists: Why Are They Still Here? / CHI 2015.

Part 5: Algorithmic Curation and AI in Social Media

  1. Explainability in Algorithmic Feed Curation: Demystifying opaque social feeds requires inspectable interfaces that articulate why a specific item was prioritized, surfacing underlying scoring heuristics to the user. — Personalizing Content Moderation on Social Media / CSCW 2023.
  2. Fragility and Bias in Fully Autonomous Moderation: Delegating moderation entirely to autonomous classifiers creates brittle defenses easily bypassed by adversarial actors while systematically penalizing dialectal nuances of marginalized speakers. — Comparing Perceived Legitimacy of Content Moderation Processes / CSCW 2022.
  3. Centering Human Judgment in Triaged Moderation: Machine learning achieves the greatest efficacy when used to flag and cluster potential harms, reserving ultimate evaluative judgment for accountable human moderators. — Comparing Perceived Legitimacy of Content Moderation Processes / CSCW 2022.
  4. Optimizing Feeds for Consensus Over Outrage: Algorithmic objectives focused strictly on clicks and shares inevitably elevate sensationalism; ranking models must be re-engineered to optimize for cross-group consensus and understanding. — AI and the Future of Digital Public Squares.
  5. Community-Trained Local Classifiers: Rather than submitting to monolithic commercial foundation models, communities should have tools to fine-tune lightweight classifiers that reflect their distinct normative standards. — Personalizing Content Moderation on Social Media / CSCW 2023.
  6. Controllable Value Levers for Feed Algorithms: True alignment between users and recommendation algorithms requires transparent control dials that let individuals directly calibrate the diversity, depth, and topical focus of their feeds. — What Makes Online Communities 'Better'? / ICWSM 2022.
  7. Preventing Dissent Erasure in LLM Summarization: When large language models summarize sprawling civic debates, rigorous human oversight is essential to prevent majority consensus algorithms from erasing minority dissent. — Wikum: Bridging Discussion Forums and Wikis / CSCW 2017.
  8. Correcting Inherited Training Corpora Toxicity: Automated systems trained uncritically on historical web corpora replicate systemic toxicity, requiring community-led data curation and explicit value calibration. — Penalizing Transparency? / CHIWORK 2025.

Part 6: The Design of Social Platforms

  1. Interface Affordances Shape Collective Norms: The core interaction affordances of a platform—such as retweet mechanisms or vote counters—profoundly shape collective psychology, rewarding either performative escalation or reflective dialogue. — What Makes Online Communities 'Better'? / ICWSM 2022.
  2. Intentional Friction to Curb Impulse Virality: Introducing intentional cognitive and interface friction into content resharing significantly dampens reflexive spreading of falsehoods and abusive pile-ons. — Survey of Multi-Platform Online Harm and Interventions.
  3. Granular Privacy in Social Browsing (Eyebrowse): Transforming web navigation into a shared social experience requires granular privacy domains and explicit opt-in controls to prevent accidental exposure of sensitive activity. — Eyebrowse: Social Web Activity Tracking / CSCW 2014.
  4. Mitigating Interface Context Collapse: Collapsing divergent personal, familial, and professional spheres into a single consolidated timeline sparks acute social tension, requiring segmented interface boundaries. — Mailing Lists: Why Are They Still Here? / CHI 2015.
  5. Rich Expressive Feedback Over Binary Reactions: Flattening discourse into binary likes or upvotes degrades conversation; interfaces must support rich communicative signals like conditional agreement, confusion, and qualitative annotations. — What Makes Online Communities 'Better'? / ICWSM 2022.
  6. Balancing Anonymity with Pseudonymous Accountability: While anonymity protects vulnerable individuals reporting sensitive truths, sustainable platforms must balance open expression with pseudonymous reputation systems to curb unaccountable harassment. — Survey of Multi-Platform Online Harm and Interventions.
  7. Taming Notification Fatigue in Synchronous Streams: Constant, unranked notification streams drain attention and induce burnout; communication systems must intelligently batch and prioritize events according to collaborative urgency. — Tilda: Making Sense of Group Chat / CSCW 2018.
  8. Permeable Yet Defensible Community Boundaries: Resilient online communities maintain clear, defensible social perimeters that welcome genuine newcomers while shielding internal culture against coordinated trolling raids. — Modular Politics: Toward a Governance Layer for Online Communities / CSCW 2021.
  9. The Structural Power of Platform Defaults: Because the vast majority of participants never alter out-of-the-box configurations, setting pro-social, privacy-first system defaults is a software architect's most potent intervention. — Eyebrowse: Social Web Activity Tracking / CSCW 2014.

Part 7: Human-Computer Interaction and Social Computing

  1. Ethnographic Grounding in HCI Research: Engineering effective social systems requires rigorous ethnographic fieldwork to understand how communities naturally negotiate norms, resolve conflict, and manage power dynamics. — Modular Politics: Toward a Governance Layer for Online Communities / CSCW 2021.
  2. Measuring Social Computing by Trust and Well-Being: Success in social computing systems must be evaluated by improvements in interpersonal trust, psychological safety, and volunteer retention rather than raw clicks or daily active users. — What Makes Online Communities 'Better'? / ICWSM 2022.
  3. Bridging Technical Architecture and Political Sociology: Tackling toxic polarization and moderation collapse requires bridging technical computing capabilities with institutional political science, law, and behavioral sociology. — AI and the Future of Digital Public Squares.
  4. Participatory Design with Frontline Moderators: Tool designers cannot construct viable moderation interfaces in isolation; systems must be co-designed alongside the frontline practitioners bearing the daily burden of crisis response. — Personalizing Content Moderation on Social Media / CSCW 2023.
  5. Empirical In-Situ Prototype Deployment: Deploying working functional prototypes directly into functioning online communities uncovers behavioral friction, social edge cases, and unexpected governance breakdowns that lab studies overlook. — Squadbox: A Tool to Combat Email Harassment / CHI 2018.
  6. Accessible Tooling for Non-Technical Lawmakers: Democratic software governance succeeds only when visual interfaces empower non-technical organizers to draft, inspect, and enact executable community rules without writing raw code. — PolicyKit: Building Governance in Online Communities / UIST 2020.
  7. Translating Academic Research into Public Software: Social computing scholarship carries an imperative to deliver maintainable open-source artifacts to real communities rather than confining insights to theoretical academic venues. — PolicyKit Project.
  8. Recognizing and Supporting Unpaid Volunteer Labor: Volunteer community moderators provide essential public infrastructure through uncompensated labor; platforms must build specialized workflow tooling that honors and economizes their time. — Squadbox: A Tool to Combat Email Harassment / CHI 2018.
  9. Stress-Testing Systems Against Adversarial Extremes: The true test of a social architecture lies not in how it handles routine socialization, but in its structural resilience against coordinated harassment campaigns and bad-faith manipulation. — Survey of Multi-Platform Online Harm and Interventions.
  10. Unequal Costs in Mandatory AI Disclosure: Mandating disclosure of AI assistance in creative or academic writing creates unequal burdens when marginalized authors face steeper evaluation penalties upon transparently revealing tooling assistance. — Penalizing Transparency? / CHIWORK 2025.
  11. Iterative Human-AI Workflow Co-Execution (Cocoa): Human-AI collaboration systems must support dynamic replanning during partial execution rather than rigidly isolating upfront planning from downstream automated execution. — Cocoa: Mixed-Initiative Co-Execution / CHI 2026.

Part 8: The Future of Online Communities

  1. Human Governance Over Financialized Web3 Primitives: True community self-determination requires expressive institutions for collective human deliberation, which cannot be solved merely through tokenomics or financialized smart contracts. — Modular Politics: Toward a Governance Layer for Online Communities / CSCW 2021.
  2. Pluralistic Governance Paradigms for the Social Web: A resilient future for digital spaces demands pluralism, where many diverse community organizational models coexist rather than conforming to a single corporate standard. — What Makes Online Communities 'Better'? / ICWSM 2022.
  3. Rebuilding Trust Through Community-Auditable Systems: Restoring public confidence in digital institutions requires migrating away from commercial black-box algorithms toward community-auditable and transparent moderation architectures. — Comparing Perceived Legitimacy of Content Moderation Processes / CSCW 2022.
  4. Cultivating Digital Civic Parks: Democracies must invest in cultivating digital public parks—online spaces architected to sustain civic deliberation and common goods rather than maximizing user ad clicks. — AI and the Future of Digital Public Squares.
  5. Cross-Platform Middleware as Independent Lenses: The future of social software lies in platform-agnostic tools that operate as user-controlled lenses across disparate services, unifying moderation and identity independently. — Squadbox: A Tool to Combat Email Harassment / CHI 2018.
  6. Scaling Civic Empathy in Mass Deliberation: The central challenge for large-scale digital discourse is designing interfaces that scale mutual understanding and perspective-taking without flattening distinct community viewpoints. — Wikum: Bridging Discussion Forums and Wikis / CSCW 2017.
  7. Valuing the Maintenance of Digital Infrastructure: Sustaining democratic digital alternatives requires shifting institutional recognition and funding toward the labor of maintaining software infrastructure, not solely rewarding novelty. — PolicyKit Project.
  8. Human Agency and Collaborative Online Resilience: Despite the systemic dysfunctions of commercial feeds, online groups repeatedly demonstrate remarkable capacity to build mutual aid networks and self-governing spaces when equipped with cooperative tools. — What Makes Online Communities 'Better'? / ICWSM 2022.
  9. Opportunities and Hazards of AI Public Squares: While foundation models could assist deliberation and scaffold participatory public spaces at scale, they simultaneously risk amplifying polarization unless governed through democratic public-interest frameworks. — AI and the Future of Digital Public Squares.