Illia Polosukhin is a co-creator of the NEAR Protocol and a co-author of the 2017 paper "Attention Is All You Need," which introduced the Transformer architecture. His engineering background spans early generative AI research at Google to building decentralized payment rails for global workforces. This compilation documents his mechanics for system design, his practical arguments for cryptographic infrastructure, and his roadmap for an internet operated by user-controlled software agents.

Part 1: The Transformer Architecture

  1. On the origin of Transformers: "The architecture emerged from a need to process sequences more efficiently than RNNs, allowing parallelization across words in a sentence." — Source: [a16z crypto]
  2. On naming the architecture: "The paper's title 'Attention Is All You Need' was a direct reflection of stripping away complex recurrent layers to focus entirely on attention mechanisms." — Source: [The Cognitive Revolution]
  3. On scaling laws: "Early on, it became clear that throwing more compute and data at the Transformer architecture reliably produced better results." — Source: [TWIML AI Podcast]
  4. On unexpected impact: "We built it to solve translation at Google, not necessarily anticipating it would become the foundation for general-purpose language models." — Source: [a16z crypto]
  5. On simplicity in design: "A core advantage of the Transformer is its mathematical simplicity compared to the recurrent neural networks that preceded it." — Source: [The Neuron]
  6. On hardware utilization: "Transformers run efficiently on modern GPUs because matrix multiplication is exactly what that hardware was designed to do." — Source: [The Smart Economy Podcast]
  7. On the 'T' in ChatGPT: "The generative pre-trained transformer relies entirely on the architecture we detailed in 2017, proving the durability of the initial concept." — Source: [Business Insider]
  8. On open research: "Publishing the paper openly was a standard Google research practice at the time, which inadvertently catalyzed the current AI boom." — Source: [TWIML AI Podcast]
  9. On architecture lifespan: "While new architectures like state space models emerge, the Transformer remains the baseline everyone is trying to beat." — Source: [The Cognitive Revolution]

Part 2: User-Owned AI

  1. On data ownership: "If a model learns from your actions, you should have cryptographic proof of ownership over that personalized model." — Source: [Encrypted Podcast]
  2. On systemic alignment: "A single centralized AI cannot align with the diverse values of billions of humans. Alignment must happen at the individual user level." — Source: [The Neuron]
  3. On local execution: "Running inference locally or within trusted execution environments ensures that your personal AI doesn't leak context to a central server." — Source: [Uneasy Money]
  4. On platform lock-in: "User-owned AI prevents a future where a single corporation can revoke your access to your digital intelligence." — Source: [a16z crypto]
  5. On monetization: "When users control their AI, they can choose to monetize the underlying data or keep it strictly private." — Source: [Encrypted Podcast]
  6. On the billionaire chief of staff: "Every internet user should have an AI agent that operates with the competence of a dedicated chief of staff, acting entirely in their interest." — Source: [Business Insider]
  7. On open weights: "Releasing model weights is necessary but not sufficient for user sovereignty; you also need permissionless infrastructure to run them." — Source: [TWIML AI Podcast]
  8. On sovereign identity: "An AI agent needs a wallet and a cryptographic identity to transact on your behalf without requiring your continuous input." — Source: [The Smart Economy Podcast]
  9. On competitive dynamics: "Open-source, user-owned models force centralized incumbents to compete on service quality rather than data monopolies." — Source: [The Blockcrunch Podcast]

Part 3: Blockchain as Infrastructure

  1. On the origins of NEAR: "We started building AI models and quickly realized we couldn't pay our global network of data labelers using traditional banking." — Source: [The Blockcrunch Podcast]
  2. On ledgers and agents: "Blockchains act as the backend for AI agents, providing a secure, permissionless environment for them to interact." — Source: [The Smart Economy Podcast]
  3. On verifiable execution: "If an AI gives you an answer, you should be able to verify the computation and the data provenance on a public ledger." — Source: [TWIML AI Podcast]
  4. On transaction speeds: "To support programmatic AI interactions, the underlying network must have high throughput and near-instant finality." — Source: [House of ZK Radio]
  5. On data provenance: "Blockchain solves the deepfake problem by providing a cryptographic paper trail for authentic human-generated content." — Source: [The Cognitive Revolution]
  6. On global coordination: "Cryptographic networks are the only reliable way to coordinate asynchronous work across borders without a central arbiter." — Source: [The Blockcrunch Podcast]
  7. On confidential computing: "Combining secure enclaves with blockchain allows us to run proprietary AI models without exposing the weights or user data." — Source: [Uneasy Money]
  8. On sharding: "Horizontal scaling via sharding is a requirement if we expect blockchain to handle web-scale traffic generated by automated agents." — Source: [House of ZK Radio]
  9. On infrastructure visibility: "Ideally, the blockchain operates entirely in the background. Users shouldn't know they are interacting with a decentralized ledger." — Source: [The Smart Economy Podcast]
  10. On Web3 utility: "The ultimate utility of Web3 is providing a trustless substrate for machine-to-machine commerce." — Source: [The Rollup]

Part 4: Chain Abstraction

  1. On fragmented ecosystems: "Users do not care which blockchain an application is built on. They just want the application to work quickly and securely." — Source: [The Blockcrunch Podcast]
  2. On wallet friction: "Managing multiple seed phrases and bridging assets across chains is an unacceptable user experience for mainstream adoption." — Source: [a16z crypto]
  3. On unified interfaces: "Chain abstraction allows developers to build applications where the user interacts with a single interface while multiple networks settle in the background." — Source: [The Rollup]
  4. On agentic routing: "AI agents will eventually handle the complexity of routing transactions to the cheapest or fastest chain." — Source: [The Smart Economy Podcast]
  5. On interoperability: "True interoperability requires protocols that can communicate state changes without requiring the user to manually intervene." — Source: [House of ZK Radio]
  6. On account models: "Flexible account structures allow users to sign transactions using biometrics or web2 credentials instead of raw cryptographic keys." — Source: [The Blockcrunch Podcast]
  7. On protocol tribalism: "Maximalism slows down the industry. The focus should be on abstracting away the infrastructure to grow the total user base." — Source: [The Rollup]
  8. On developer experience: "If developers have to rewrite their smart contracts for every new layer one, the ecosystem cannot scale efficiently." — Source: [House of ZK Radio]
  9. On the end state: "In a fully abstracted environment, the underlying network becomes as invisible as the server routing an email." — Source: [a16z crypto]

Part 5: Startup Building and Focus

  1. On finding mentors: "Find a mentor who's a few years ahead of you, like two or three years ahead, because they still remember what they were doing in your position." — Source: [20 Minute Playbook]
  2. On shipping speed: "Velocity is the most important metric for a young startup. You learn by shipping, not by planning." — Source: [20 Minute Playbook]
  3. On adapting to friction: "When we couldn't pay our international workforce, we didn't wait for banks to catch up; we built our own payment infrastructure." — Source: [The Blockcrunch Podcast]
  4. On extreme ownership: "Taking complete responsibility for both successes and failures within your team prevents the blame-shifting that kills startups." — Source: [20 Minute Playbook]
  5. On identifying problems: "The best products originate from internal bottlenecks that you are forced to solve for yourself." — Source: [The Blockcrunch Podcast]
  6. On early hiring: "Hire engineers who are comfortable operating in deep ambiguity and who prefer writing code over attending meetings." — Source: [20 Minute Playbook]
  7. On pivoting: "The transition from an AI startup to a layer-one blockchain required throwing away months of work to focus on the more immediate structural problem." — Source: [Constantin Kogan Podcast]
  8. On technical debt: "You have to accept a degree of technical debt early on to find product-market fit, but you must aggressively refactor once you find it." — Source: [20 Minute Playbook]
  9. On building protocols: "Launching a decentralized network is fundamentally harder than launching an app because you have to bootstrap a community of validators simultaneously." — Source: [Constantin Kogan Podcast]
  10. On founder resilience: "The hardest part of building a company is managing your own psychology during prolonged periods of uncertainty." — Source: [20 Minute Playbook]

Part 6: Decentralization and Trust

  1. On decentralization as a tool: "Decentralization is a tool and not a goal. We use it to solve specific problems related to trust and verification." — Source: [Uneasy Money]
  2. On formal verification: "As AI handles more capital, we need mathematical guarantees that the software executing the trades will behave exactly as intended." — Source: [TWIML AI Podcast]
  3. On training data bias: "A decentralized network of contributors is less likely to produce a culturally biased model than a homogenous team in a single geographic location." — Source: [The Neuron]
  4. On trustless execution: "You shouldn't have to trust the server hosting the model; you should be able to verify the computation yourself." — Source: [TWIML AI Podcast]
  5. On network security: "A blockchain is only as secure as its economic incentives. If attacking the network is profitable, it will eventually be attacked." — Source: [House of ZK Radio]
  6. On zero-knowledge proofs: "ZK technology allows us to verify that an AI model ran correctly without forcing the creator to reveal the proprietary weights." — Source: [House of ZK Radio]
  7. On single points of failure: "Relying on a single API endpoint for machine intelligence creates an unacceptable risk for applications built on top of it." — Source: [The Neuron]
  8. On censorship resistance: "A decentralized network guarantees that AI agents can communicate and transact regardless of arbitrary platform restrictions." — Source: [The Rollup]
  9. On auditing AI: "Public ledgers allow researchers to audit the sequence of data inputs that led a model to a specific conclusion." — Source: [The Cognitive Revolution]

Part 7: The Agentic Internet

  1. On the end of traditional apps: "In the future, you won't navigate a website. You will tell your agent what you want, and it will interact with the service's API." — Source: [a16z crypto]
  2. On automated commerce: "Most transactions on the internet will eventually be negotiated and executed by machines talking to machines." — Source: [The Smart Economy Podcast]
  3. On information filtering: "Agents will serve as our primary defense against the infinite supply of low-quality, synthetically generated content." — Source: [The Cognitive Revolution]
  4. On cross-platform tasks: "A truly capable agent needs permission to read your emails, monitor your calendar, and execute payments simultaneously." — Source: [a16z crypto]
  5. On API design: "Web developers will shift from building graphical user interfaces for humans to building clean, predictable APIs for AI agents." — Source: [The Rollup]
  6. On continuous learning: "Your personal agent will update its understanding of your preferences in real-time, functioning as an extension of your own memory." — Source: [Encrypted Podcast]
  7. On agent economics: "We will see micro-economies form where specialized AI models pay other specialized AI models for data processing." — Source: [The Smart Economy Podcast]
  8. On intention resolution: "The core function of an agent is taking a vague human intention and breaking it down into a deterministic sequence of API calls." — Source: [a16z crypto]
  9. On digital representation: "Your agent will act as your digital twin in online spaces, attending meetings or filtering communications when you are unavailable." — Source: [The Cognitive Revolution]

Part 8: Future of Work and Governance

  1. On human labor: "AI will commoditize repetitive cognitive tasks, forcing human labor to move toward higher-variance, creative problem solving." — Source: [The Cognitive Revolution]
  2. On network governance: "Decentralized systems require governance models that allow for rapid technological upgrades without sacrificing community consensus." — Source: [House of ZK Radio]
  3. On AI delegates: "Individuals might soon elect AI delegates to vote on their behalf in digital governance systems, based on a predefined set of values." — Source: [The Cognitive Revolution]
  4. On economic displacement: "We need robust mechanisms for distributing the wealth generated by automation, potentially facilitated by crypto networks." — Source: [The Cognitive Revolution]
  5. On regulatory approaches: "Regulating the math behind AI is impossible; regulation should focus on the application layer and the consequences of deployment." — Source: [Uneasy Money]
  6. On open-source security: "Open-source AI development is safer than closed systems because it allows thousands of independent researchers to patch vulnerabilities." — Source: [TWIML AI Podcast]
  7. On developer productivity: "Engineers will transition from writing boilerplate code to acting as reviewers and orchestrators of code generated by models." — Source: [The Blockcrunch Podcast]
  8. On community ownership: "The users who generate the data and test the protocols should have a financial stake in the success of the underlying network." — Source: [Constantin Kogan Podcast]
  9. On algorithmic transparency: "Citizens have a right to know how the algorithms that determine their credit scores or feed recommendations are weighted." — Source: [The Neuron]
  10. On technological optimism: "Despite the risks, deploying intelligence at scale is the most direct path to solving hard constraints in medicine, energy, and education." — Source: [The Cognitive Revolution]