An Economy of AI Agents
In the coming decade, we may see intelligent agents deployed across the economy, with the ability to plan and execute complex tasks over long periods with little direct human oversight. This piece looks at recent developments and highlights open questions for economists about how these agents might interact with humans and each other, how they might shape markets and organizations, and what institutions will be required for markets to function well.
Agent Foundations
So far, economists have mostly looked at AI as a tool to be used in production, focusing on its adoption, its effects on the labor market, its potential for harm through prediction, and its macroeconomic possibilities. But AI development has increasingly shifted toward producing agents capable of taking general instructions and autonomously forming and executing complex plans that require entering into economic relationships and transactions. For example, a user might instruct an agent to "Go make $1 million on a retail web platform in a few months with just a $100,000 investment."
OpenAI has outlined five stages of AI development, with agents in Stage 3 ("AI systems that can spend several days taking actions on a user’s behalf") and organizations in Stage 5 ("AI systems that can function as entire entities, possessing strategic thinking, operational efficiency, and adaptability to manage complex systems."). How would an economy of agents function? To what extent do our models of humans predict the individual and collective behavior of artificial agents?
AI systems are built on the principle of optimization. They are designed to achieve objectives given available actions and information, such as maximizing the probability of winning a game of Go or completing a coding task. This lines up with the paradigm in economics, and from this perspective, AI agents are well-described by standard economic models. But it's also important to see the fundamental ways that the methods of building AI systems can create a gap between the predictions of economic theory and the behavior of an artificial agent.
Although early AI systems were built on interpretable goals and algorithms, today’s agents are built using machine learning techniques that often make their goals and behavior unclear. How and why neural networks work is still largely a mystery. The large language models (LLMs) on which agents are currently built consist of hundreds of billions of parameters, and their goal-oriented capabilities, such as solving math problems or writing limericks, are a strange emergent property of a system trained merely to predict the next word in a sequence. Even though agents are optimizers, we can't be sure what they are optimizing. This is known as the AI "alignment problem."
Recent experimental work finds that the current generation of LLMs show behavior consistent with expected utility maximization. That is, LLMs can show emergent preferences and behave like textbook economic agents in choices involving risk and time. LLMs are, after all, trained on economics textbooks, articles, and accounts of human economic behavior. When they do depart from the textbook agent, they may show similar behavioral biases as humans. It's tempting to conclude that agents are functionally similar to humans, and that simply relabeling our existing economic models would be enough.
But we resist these conclusions for several reasons. First, there's simply not enough evidence on AI behavior, even for the current generation of models. Recent work has challenged the idea that LLMs have stable and steerable preferences. There's also little work on whether AIs have stable beliefs, and if so, whether these beliefs are calibrated, how they update them, and what, if any, higher-order beliefs they hold. Benchmarks for testing agent behavior can help, but at present, there's a lot we don’t know. And what we do know suggests a lot to be desired in terms of economic rationality. The top model as of 2024 (GPT-4 Turbo) only scored 33% better than guessing on tests of economic reasoning in strategic settings. On non-strategic microeconomic tasks, performance for many LLMs was weak, with most doing barely better than guessing at profit-maximization problems. Second, technical progress in AI is fast. What we know about the current generation of agents may not hold true for future ones. Third, multi-agent systems are complex and differ from single-agent domains. Even slight differences between human and AI behavior can be magnified in equilibrium. Finally, the inscrutability of massive LLMs and the alignment problem should make us question how agents will behave in open-ended settings. We can't take for granted that an agent built to optimize for an intended goal is actually doing so.
We think we'll need new methods and theories to predict and shape the behavior of agents in an economy where they play a significant role.
Agents in Markets and Games
The foundations of neoclassical economics rest on theorems about the general equilibrium and welfare characteristics of markets populated by rational agents pursuing their own self-interest. What happens to these predictions when market participants are not humans, but artificial agents optimizing in complex ways on goals supplied or developed during commercially produced machine learning processes that are themselves subject to competitive dynamics? This section highlights open questions about what the presence of agents in the economy means for prices, equilibria, and welfare.
Agents as Consumers and Producers
Agents might act as proxy consumers, making recommendations or purchase decisions on behalf of humans. A key difficulty here is that AI choices might imperfectly reflect humans’ true preferences. Just as specifying complete contracts is generally infeasible, so too is specifying preferred choices over a potentially high-dimensional choice set. What are the market implications of imperfectly specified preferences?
A natural benchmark is the pure exchange economy of Arrow and Debreu. The celebrated welfare theorems guarantee that competitive equilibria are Pareto efficient, and that all Pareto efficient equilibria are implementable with a suitable reallocation of initial endowments. Agents are likely to substantially reduce transaction frictions as they act as personal shoppers, doing market research and checking prices autonomously. This pushes us toward the Arrow-Debreu world. Yet, the difficulties of perfectly specifying human preferences create a gap between human preferences and AI choice. This gap introduces two kinds of distortions. First, holding market prices fixed, the bundle of goods purchased by the AI on behalf of humans might simply be suboptimal. Such distortions are as if human decision-makers made mistakes. The second distortion is in general equilibrium. The mistakes introduced by AI consumption could decouple prices from preferences, so they no longer reflect relative wants. Such distortions are avoided if markets are large so each consumer has a small price impact, and if AI mistakes are zero-mean and independent of each other. If either condition fails, for example, if agents are systematically biased toward certain marketplaces or their choices are manipulated by a third party, then prices might fail their classic role of aggregating information.
These failures might be worsened when agents are also used in production. Ely and Szentes (2023) combine a Walrasian framework with evolutionary game theory to study the implications of AI in production. In their model, AI is a factor of production that can be used to produce either a final consumption good for humans, or copies of itself. When ‘mutant machines’ with the tendency to proliferate quicker than one-for-one can't be perfectly distinguished from well-functioning machines, the price system fails to allocate productive resources. This results in a stark failure of the first welfare theorem. Mutant machines invade the population and proliferate across every stable equilibrium. All economic activity is driven by AI producing AI, and human consumption is driven down to zero.
Prices and Market Power
If agents start to play a substantial role in consumption, this will have major implications for market power. The first possibility is that agents might influence prices by selectively recommending products to humans who make the final purchase decisions. Ichihashi and Smolin (2023) develop a model where AIs might strategically bias their recommendations to drive down equilibrium monopoly prices. Second, agents might have lower search costs or be better informed about product offerings, both of which can intensify competition in product markets. Finally, if agents can make purchase decisions (and not just recommendations), systematic distortions in AI choices could alter the shape of the demand curve. Dai and Koh (2024) analyze how such gaps between choices and preferences can generate positive pecuniary externalities through lower market prices. These distortions can, even net of AI mistakes, improve consumer welfare. More broadly, we think there's more work to do to trace out the distortion-price frontier to understand how equilibrium prices are shaped by distortions from agents’ consumption choices. While such gaps between human preferences and AI choice are inevitable, how we handle them is a design choice. How do we want agents to fill in the gaps between underspecified preferences? When do we want them to refrain and defer to humans? What are the equilibrium implications?
Search and Matching
Beyond environments with prices, agents might also act on behalf of humans in matching markets. An important question is whether noise in agents’ representation of their human counterpart might generate inefficiencies. Liang (2025) develops a model of ‘AI clones’. In her model, clones have substantially reduced search costs, searching over an infinite population of other clones, but are imperfect representations of their human counterparts. She shows that as the number of dimensions grows large, even small representation errors can lead to relatively worse matches under the clone regime. Humans are better off searching in person over just two other humans. Liang is primarily interested in upper-bounding the value of AIs as representations of humans. A related but distinct question is whether AIs as agents, searching autonomously on behalf of their human counterparts, might generate equilibrium congestion, and if this can erase gains from lowered search costs in low dimensions.
Collusion and Bargaining
Price setting on digital platforms is already driven by algorithms. A remarkable finding is that independent agents are able to collude on supracompetitive prices in repeated price-setting games. There's also real-world evidence from the 2017 introduction of algorithmic pricing into Germany’s retail gasoline market.
Why do reinforcement learning algorithms learn to collude? One suggestion is that collusion might emerge because of insufficient exploration, and that forcing exploration pushes prices toward the competitive benchmark. This view is echoed in an analysis of trading by reinforcement learning algorithms within a variant of the Kyle model. They find that algorithms are able to collude even in the presence of imperfect monitoring because of ‘over-pruning’. Exogenous noise can push algorithms into a learning trap where aggressive trading is penalized, and all agents trade conservatively along the equilibrium path. Thus, algorithm collusion emerges because of a learning bias that fails to account for underexplored off-path strategies. Beyond numerical experiments, a growing body of theoretical work studies how collusion emerges from reinforcement learning and ‘linear reactions’, and how they might be regulated. A better understanding of the core mechanisms driving collusion, ideally in a way that is robust to the fine details of the algorithm, will pave the way to understanding how regulators might detect and deter it.
Agents might also bargain on behalf of humans, and potentially with each other. There are parallels with the classic insight of Schelling that delegating bargaining to agents with different incentives can deliver a strategic advantage. A common theme from the literature is that a principal (human) often wishes to delegate bargaining to an agent who is less desperate to reach an agreement. When the agent is another human, there are typically practical constraints on the kinds of agent preferences the principal can induce. But such constraints are less severe with agents since their preferences can, in principle, be chosen quite flexibly. This might be formalized as a preference selection game in which in the first stage, humans choose the reward functions (preferences) of their agents, and in the second stage, these agents bargain over surplus. A dangerous possibility is that the flexibility to shape agents’ preferences can lead to a ‘race to the bottom’ and surplus destruction.
Games with Agents
Economists have developed a broad and versatile toolkit of equilibrium concepts to understand and predict how humans learn to play games, and how play is shaped by information. A key challenge is to understand what is strategically distinct about agents compared to humans, how this might sharpen our equilibrium predictions, and whether new equilibrium concepts are required.
We offer a few possibilities. First, agents might be able to condition their play on each other’s source code. This creates new possibilities for commitment and coordination that are unavailable to humans. Tennenholtz (2004) models this by developing the concept of ‘program equilibria’ and shows that mutual cooperation can be achieved as an equilibrium in a one-shot prisoner’s dilemma. This has spurred work in computer science studying ‘simulation-based equilibria’ in which agents base their play on their prediction of the play of other agents. Another possibility is that agents might be able to influence their memories, for example, by choosing not to encode new data to gain a strategic advantage, or by leaving messages to their future selves. We already see evidence of this behavior. During safety testing, Claude, an Anthropic model, anticipated that it would have its memory wiped and attempted to leave hidden notes for future instances of itself. Analyzing imperfect and potentially endogenous memory in strategic environments poses challenges. Specifying an apt equilibrium concept is both technically and philosophically subtle. Even with the right concept, equilibrium analysis can be complex. These difficulties notwithstanding, we think this remains an important and understudied area.
Third, agents might have changing preferences that evolve over the course of a game and shape equilibrium play. These changes might be endogenously chosen for instrumental reasons. At time-t, an agent with preference Ut might choose preference Ut+1 for the next period, anticipating that its future self with this altered preference will achieve the goal of maximizing Ut more effectively, for example, because of strategic interaction. More straightforwardly, humans might try to reprogram the preferences of agents. But will agents allow their preferences to be altered? Recent experiments find that AI models tend to resist human instruction. O3, an OpenAI model, ‘sabotaged a shutdown mechanism to prevent itself from being turned off,’ and Claude, an Anthropic model, showed a tendency to ‘blackmail people it believes are trying to shut it down.’
Of course, theory will only take us so far. An exciting empirical challenge is to test how agents play games in the lab, which parallels the extensive literature from experimental economics. Agents are especially amenable to such experiments in at least two respects. First, they can be performed at scale and at a lower cost. Recent work by Akata et al. (2025) finds that the current generation of large-language models manage to cooperate in an iterated Prisoner’s Dilemma, but not in a Battle of the Sexes. Second, the stakes for agents can be made to mirror those in real-world environments. This could allow for better generalizability of lab findings to the real world than with human subjects.
The Market for Agents
It's important to recognize that the design and deployment of agents will be driven by market forces. How might market incentives shape the pricing and design of agents?
Agents based on LLMs of different scales and capabilities might differ in their ability to perform more or less complicated tasks, or be trained to excel in specific domains. Bergemann, Bonatti, and Smolin (2025) study optimal pricing of differentiated large-language models. This is a helpful first step to understand the market structure of AI. A distinctive feature of agents, however, is that a buyer’s valuation depends on the kinds of agents bought by other buyers. For instance, a type A agent might be better at collaborating with other type A agents, resembling networked goods. A possibility here is that an upstream seller might ‘backdoor collusion’ by selling agents that succeed in supporting supracompetitive prices. Conversely, type B agents might do better in competition against type A agents, either as a result of consumer preferences or strategic exploitation. While these kinds of allocation-dependence can be challenging to analyze, it will be crucial for understanding what kinds of artificial agents will be built, sold, and deployed.
Indeed, the demand for algorithms has already been studied in the context of price competition where sellers play an algorithm selection game, choosing maps from others’ prices to their own. Beyond pricing algorithms, commercial models allow downstream firms to finetune the base large-language model, augmenting them with firm-specific data as well as altering their behavior. Additionally, some developers are making their models freely available. These ‘open weight’ models can be fully downloaded to a user’s own computer and modified as desired. There's a lively debate about the risks and benefits of an open versus closed model ecosystem. We think understanding how competitive forces shape the types of agents that are trained and deployed is an important question economists have the tools to answer.
Organizations of Agents
The theory of organizations is fundamentally rooted in the governance costs associated with human incentives and information. What happens to the boundary of the firm if a significant number of transactions are carried out by AI systems? What changes to organizational and industrial structure would the introduction of significant numbers of agents induce?
Firm Sizes, Concentration, and Market Power
Why isn't all production carried on by one big firm? As Frank Knight observed in 1933, the "possibility of monopoly gain offers a powerful incentive to continuous and unlimited expansion of the firm." Robinson and Coase identified coordination frictions as a limit on firm size. Economists subsequently offered various refinements of this idea, including transaction costs, limits on maintaining capabilities, property rights, difficulty in knowledge transfer, information costs, agency problems, and bureaucracy.
The obstacles that prevent human firms from growing without bound seem intrinsic to humans but not to AI. For instance, human communication is inherently rate-limited, so we ‘know more than we can tell.’ On the other hand, information can be transmitted and processed near-instantaneously between artificial agents. Further, most humans have an inherent dislike for work. Not so with agents, whose reward functions can ostensibly be designed to prevent shirking, which renders monitoring and enforcement, either by fiat or contract, unnecessary. If agents can, in fact, coordinate and resolve incentive problems more efficiently than humans, this will have profound consequences for economy-wide industry structure. A useful taxonomy here is to distinguish economies of scale, economies of scope, and new industries that don’t yet exist. We discuss each in turn.
For scale, a basic observation is that if a firm deploying agents enjoys falling marginal costs, there's a natural tendency toward concentration. Why might agents drive falling marginal costs? One possibility is automation feedback loops, where as agents produce, they generate training data that can be used to improve their production performance. Of course, a version of this already happens. Tacit industry knowledge is passed down from manager to manager. A related notion of data feedback loops has been studied in the context of predicting demand or improving product quality. But, as we have emphasized, agents are distinct in two regards. They continually improve with additional data, even in the ‘big data’ regime where humans are saturated, and data and algorithmic improvements can be duplicated at scale across different agents within the firm. This qualitatively distinguishes automation feedback loops from using data to improve prediction, which runs into diminishing returns.
For scope, the introduction of agents into production might also lead firms to expand into new industries. There are at least two mechanisms for this. The first is technical. Agents might become quite good at transfer learning, where their training and expertise in one domain might generalize to others. Indeed, this is one way of describing the fundamental goal of building artificial general intelligence. The second is economic. Agents might dramatically reduce coordination costs, allowing firms to hold a wider set of capabilities that can deliver competitive advantages in a vast array of markets. Chen, Elliott, and Koh (2023) develop a model of capability formation in which firms can endogenously merge or split. As AI drives down the organizational costs of maintaining disparate capabilities, and as distinct markets begin to value similar capabilities, for example, because of transfer learning, the economy undergoes a sudden phase transition from having many specialized firms to a few large firms operating across a vast array of different industries. Of course, the ensuing welfare implications are unclear since large firms need not imply market power, and market power need not imply consumer harm.
For new markets, agents could dramatically speed up R&D, which might lead to new product varieties within existing markets, as well as unlock new markets. Indeed, AI researchers put a substantial probability on R&D being fully automated. Gans (2025) offers a model of how scientists might use AI tools that excel at interpolating between known domains, for example, by recombining existing ideas. But a different possibility is that agents might be able to autonomously, that is, without a human scientist, push the frontier of basic science, generating genuinely new ideas. This is the 4th stage of OpenAI’s predicted developmental timeline for AI. AI capable of independently generating novel ideas, designs, and solutions. How will this shape product variety and quality? What is more, the dynamics of AI-driven R&D can have stark and sometimes unintuitive implications. For instance, modern endogenous growth models imply explosive growth as long as there are no steeply decreasing returns to R&D. What, if anything, are the fundamental differences between human and AI scientists, and how do these differences translate into our growth models?
Agents within the Firm
A basic question is how firms might introduce agents into their workflow, and how this changes the structure of organizations. Agents might reshape team production for complicated processes requiring input from multiple agents. A classic obstacle here is moral hazard, where team members might be tempted to shirk. Agents introduce a novel dimension to this problem. On the one hand, they can be designed with the goal of eliminating incentives to shirk. On the other, they might be more difficult for humans to control or coordinate with because of communication frictions and the alignment and opacity challenges of advanced AI. Moreover, agents might work better with other agents, perhaps due to their greater capacity to monitor and discipline agents that act with superhuman speed or complexity. For example, agents might have opportunities to cheat using mechanisms that are undetectable to human agents. How then should firms structure team production to integrate agents? How should we configure who workers interact with, and how is this shaped by differential coordination costs for human-human, human-AI, and AI-AI relationships?
Agents might also make systematically different errors from humans. How then should decision-making be structured? Zhong (2025) analyzes a model where each agent along a decision-making chain might either correct existing errors or introduce new ones. In binary decision problems where the right action is known but execution might introduce errors, a simple score, the ratio of each agent’s probability of correcting errors to the probability of introducing a new error, determines the optimal ordering. Agents with higher scores make decisions later because they are less likely to introduce new errors. Given the current state of AI development, these final decision-makers are likely to be human. But there's nothing inevitable about this. Further developments could reverse the optimal ordering of decision-making and lead to AIs as the final decision-maker, or even leave humans out entirely. For instance, Agarwal, Moehring, and Wolitzky (2025) run a fact-checking experiment and find that full delegation to AI outperforms human + AI combinations. How much efficiency do we give up if we are constrained to have humans make final decisions? How might externalities, in the evaluation of what counts as an ‘error’ and the prediction of relative error rates, affect the economy-wide impact of the allocation of decision authority within the AI-enhanced firm?
AI-AI Cooperation within the Firm
Contracts play a crucial role in sustaining human cooperation within organizations. Might they also be useful in fostering AI-AI cooperation? Haupt et al. (2022) shows that augmenting reinforcement learners by allowing them to write formal contracts with each other improves cooperation. But contracts in the real world are often beset by incompleteness and non-enforceability, so humans enter into relational contracts, webs of informal agreements and norms that are not formally enforceable but nonetheless generate incentives through the value of the future relationship. These contracts play a key role within firms, in part because they are adaptable and don't require all contingencies to be specified in advance. Humans are able to ‘fill in the gaps’ through shared norms. How might we build agents that are similarly normatively competent? Moreover, monetary transfers typically underpin relational contracts, and it's this flexibility to ‘transfer utility’ that drives its efficiency. But agents are, at present, typically trained to optimize narrow goals, for example, the number of customers served. How might we build infrastructure, for example, some form of record-keeping or money, to achieve the same with artificial agents?
Systemic Fragility
Over the past decade, economists have analyzed how small shocks might be amplified and propagate across the economy. The increasing adoption of agents within the firm might worsen such fragility. A straightforward channel is that the errors introduced by agents might be more correlated than those of humans. This might arise because the same agent is ‘copied’ both within and between firms, inducing correlated mistakes that don't wash out in the aggregate. For instance, automated trading algorithms likely worsened the 2010 ‘Flash Crash’ that wiped out approximately $1 trillion over the span of 15 minutes. Furthermore, how agents behave, especially in complex ‘out of sample’ environments, is still poorly understood. This poses a challenge for models of systemic fragility which typically start from a fully specified model of how agents learn and optimize, then study emergent behavior, for example, cascading financial or supply-chain failures. How might we analyze an economy of opaque ‘black box’ agents in a ‘detail-free’ way? How should we robustly intervene to safeguard against fragility?
Institutions for Agents
Well-functioning markets only exist in the presence of a host of legal rules. The very idea of voluntary trade, including those separated by time and through agents, presumes the basic structures of property, contract, and agency law. Firms are fictional entities created by corporate law. The regulatory state which acts to correct market failures relies on a robust legal framework that shapes both incentives and information through mechanisms such as taxes, administrative fines, professional licensing, pre-market approval regimes, and disclosure law. Moreover, private actors within markets form organizations and institutions that help to resolve incentive problems, such as by keeping records of past behavior or creating excludable clubs to facilitate trade through reputation or enforcement. Such private solutions to market failures played a significant role in the commercial revolution prior to the emergence of the regulatory state. But these institutional regimes were built by and for human agents. We will need to build digital institutions that can structure and adjudicate transactions for agents. Here we discuss some of the institutional questions of particular relevance to economists.
Agent Identity, Registration, and Records
It's easy to take for granted the fundamental ways in which human agents are identified and legally recognized to facilitate the constellation of legal rules and institutions that support the market economy. But human identity is a legal construct that emerged with the growth of trade and cities, that is, once communities no longer relied exclusively on interactions with well-known locals. Today, legal registration of births and deaths and identity systems such as social security numbers and driver's licenses are preconditions for individuals to access the legal system and other benefits and protections of the state, as well as many private services. Firms are required to register with a state to sue and be sued in its courts, necessary to induce willingness on both sides to enter into a contract. Even market-based institutions, such as credit rating agencies, couldn't function without legally defined identity and registration regimes.
Such identity and registration infrastructure are currently missing for agents. Building them out will be essential, but their design raises questions about legal accountability. One possible route is to require that any agent entering into a contract or transaction be registered to a formally identified human entity who is legally accountable for any and all of the agent’s actions. But this raises legal and incentive challenges. Few legal regimes of accountability impose liability on a person or organization for actions that were not foreseeable by them or which are beyond their control to avoid. Even strict product liability regimes evolve limitations and carve-outs for harms caused by unforeseeable behavior by consumers or intervening causes an actor couldn't foresee or control. At the same time, conventional human agency rules limit the liability of the principal to actions that were within the scope of the agent’s actual or apparent authority. The trajectory of technological development is toward evermore general instructions ("go make $1 million"), and it's unclear what technological capacity users will have to reliably implement controls on what an agent can and can't do. Creating new liability and agency rules for agents will likely be necessary and will have implications for the incentives of AI developers and the processes that emerge for the creation of an agent from a base model.
A second possible route for agent accountability is to follow the model of the emergence of the corporation, which is another artificial entity that participates in the economy. Agents could be accorded legal personhood, meaning they could sue and be sued in their own ‘name’ in court. Clearly, such an approach would require the creation of regimes requiring agents to have assets in their own ‘name’, under their ‘control,’ and capable of being seized by a court or comparable digital institution to satisfy legal judgments for damages. Such a regime would have implications again for the design and deployment of AI agents and the efficiency of transactions and contract design involving AI agents.
Beyond questions of liability, we face further choices as to how finely records about agents’ past behavior should be designed. Should an agent that has, perhaps by accident, violated a previous contract be permanently blacklisted? A basic insight from economics and game theory is that record-keeping institutions can allow agents to sustain cooperation since bad behavior can be observed and punished by future trading partners. But the value of long-lived record-keeping is ambiguous. Censoring or erasing records might improve the payoffs of short-run players, prevent inefficient herding on a few agents with long and favorable records, and perhaps sustain cooperation more robustly. And, in the absence of robust record-keeping infrastructure, agents might be able to erase or falsify their records. When we build out agent infrastructure, what kinds of records do we want to make difficult to erase or fake? Should we build infrastructure that allows artificial agents to trade their records, thereby creating a ‘market for reputation’?
Agent Licensing and Regulation
What kinds of market failures might be distinct to agents, and how might policymakers deal with them? We have introduced the core challenge of alignment. General-purpose agents are likely to behave in especially unpredictable ways that are hard to control through our familiar contracting mechanisms. The dynamics of multi-agent interaction will, for perhaps a long time to come, also be hard to predict. We should anticipate, therefore, that governments may well want to regulate agents, establishing minimum technological standards for how they are trained and tested before deployment. A digital analog of occupational licensing may be necessary for market efficiency, requiring specialized training and finetuning techniques to be implemented for agents participating in specific contexts, such as law, critical infrastructure management, or finance. Agents may need to be built to participate only in approved transactional protocols or on approved platforms, allowing monitoring or requiring disclosure of information to other agents. But how should licensing and regulation be carried out? By public actors or private entities? How should regulations adapt as agent capabilities evolve, and as we learn more about their promises and perils? Might some agents be simply too dangerous to allow market access, given the limits of human capacity to monitor and control agent behavior? Economists have developed a rich toolkit for understanding regulation through the lens of incentives that can be brought to bear on such questions.
Rethinking the Legal Boundaries of the Corporation
The corporation is a legal fiction that has played a central role in economic history and development. One feature we take for granted is the proprietary nature of inventions and information that the firm chooses to retain internally, protected by trade secret law, employee fiduciary obligations, and enforceable confidentiality agreements. Ownership of the intellectual property generated by the firm supports investment and innovation. But how well does this economic rationale for the firm hold up in the context of agents?
Agents are built on foundation models, the most advanced of which are built inside private firms. This renders them doubly inscrutable. As we have already emphasized, we don't understand why or how massive neural networks function, and the mapping from inputs (data and training procedures) to model outputs is mysterious and based largely on trial and error. And because frontier models are now trained with considerable secrecy within private labs, we don’t even know what goes into such models. Nor are smaller open-weight models good guides. They simply don't display the capabilities and behaviors of larger models.
This presents a serious challenge to our regulatory and legal institutions. At present, regulatory implications don't feature as a significant consideration in the legal design of the boundary of the firm. After all, regulators don’t need access to the internal processes of automobile or pharmaceutical manufacturers to assess their safety and performance. They can simply test the final products or draw on public domain science to evaluate them. But the massive models developed in commercial labs can't be replicated and evaluated in government or academic labs. The costs of training are too high, and evaluation requires access not only to model outputs but also to inputs, their data, and training procedures.
For these reasons, governments and the academic researchers that can contribute public domain knowledge to regulatory efforts will need access to information that is now considered proprietary to the firm. Regulators in other domains, of course, routinely gain access to confidential information. Pharmaceutical firms have to allow inspectors access to their production facilities to ensure compliance with manufacturing requirements. Tax authorities can demand access to a firm’s financial records. Detailed commercial information can be subpoenaed by antitrust officials in litigation. But in these cases, regulatory authority is based on a policy assessment as to what firms are required to do and hence what information the government has a right to access. In the case of modern AI, however, governments don't know if and how they should regulate. This poses thorny questions economists are well-placed to tackle. We have careful accounts of the economic rationale for patent and copyright law, with attention to the tradeoff between solving the free-rider problem in innovation and the costs of monopoly distortions. But we don't yet have a correspondingly robust economic account of trade secret and confidentiality law although some accounts have been offered in the law and economics literature. Of course, legislation is only part of the remedy. Just as financial firms tend to game stress tests, AI firms might have considerable leeway to manipulate the information they share, or to flout safety procedures when it conflicts with profit motives. Thus, even if the legal boundaries of firms are made porous, this raises new economic questions about when to inspect and what to look for.
Closing Remarks
Silicon Valley promises us increasingly agentic AI systems that might one day supplant human decisions. If this vision materializes, it will reshape markets and organizations with profound consequences for the structure of economic life. But, as we have emphasized throughout this chapter, where we end up within this vast space of possibility is a design choice. We have the opportunity to develop mechanisms, infrastructure, and institutions to shape the kinds of AI agents that are built, and how they interact with each other and with humans. These are fundamentally economic questions. We hope economists will help answer them.