New Approaches to Characterize Industries: AI as a Framework and a Use Case | American Enterprise Institute

About the Compendium

New Approaches to Characterize Industries: AI as a Framework and a Use Case brings together leading economists, data scientists, and policy experts to tackle one of the biggest challenges in the AI era: how to measure its real impact on industries, jobs, and skills. Traditional classification systems are failing to capture the rapid diffusion of generative AI, leaving policymakers and practitioners without the data needed to respond effectively. This volume highlights innovative strategies—from “industries of ideas” models that track talent flows to new workforce data linkages and skill-taxonomy frameworks—that can guide smarter education, training, and economic policy.

Introduction 

The newest artificial intelligence technologies, especially generative AI systems, could fundamentally transform how firms do business and how Americans work. Still, there is little data and evidence to understand how AI is reshaping the economy. 

This is coming just as the politics of work are changing. The automation and trade shocks of the early 2000s and the resulting “deaths of despair” have brought political attention to how technological and economic shifts can transform communities and livelihoods. Business leaders, workers, educational and training institutions, and governments need local, timely, and actionable data to help the workforce respond to shocks that are likely to be even greater than those of two decades ago. 

Unfortunately, the visibility of the workforce’s transformation has also made it clear that traditional data sources are inadequate to inform that response. Even current scientific and industrial classification systems are not fit for this purpose. AI, like many other new and emerging technologies, is neither a well-defined scientific field nor a distinct industry.¹ Moreover, national governments’ increasing focus on industrial policy, which is fundamentally reshaping the economy and society, suggests that entirely new data and frameworks may need to be developed to understand how workers and firms interact. 

On March 18, 2024, the American Enterprise Institute, Stanford University’s Digital Economy Lab, and New York University convened a daylong seminar titled “New Approaches to Characterize Industries: AI as a Framework and a Use Case.” Its goal was to begin exploring elements of a theoretical framework that could help meet this data challenge, and seminar participants were tasked with identifying what data, definitions, models, and tools exist or could be developed. 

Participants were invited to the workshop for their expertise in data, measurement, and analysis. But more importantly, each presenter had a history of designing, deploying, and using new data systems— such as the Longitudinal Employer-Household Dynamics program at the US Census Bureau, the Institute for Research on Innovation & Science (IRIS) at the University of Michigan, the Texas Workforce Commission, and the New Jersey Department of Labor & Workforce Development—and private job market data. As intended, workshop participants drew on their experiences to identify empirically implementable, dynamic, and flexible approaches for understanding this critical and emerging space. 

The following three key takeaways emerged from the formal and informal discussions: 

1. An innovative institution should be established to implement a new vision and framework. This independent, nonpartisan institution could be dedicated to producing bottom-up, demand-driven tools and insights for businesses, workers, and governments by connecting advances in AI and other critical technologies to changes in the nature of new and existing jobs, skills, and economic opportunities.
2. Policymakers and practitioners should support the institution in establishing partnerships that directly serve the needs of businesses, workers, and researchers. These partnerships would build an understanding of how AI and other emerging technologies affect local and regional
   economies and labor markets. An initial focus might include
   
   1. Providing data and insights to firms as businesses’ AI capabilities move from experimental
      use to enterprise use at scale,
   2. Prototyping and producing customizable tools so current and future workers can acquire skills in response to changing demands, and
   3. Producing tools and analyses that federal, state, and local government agencies would
      use to allocate programmatic education and training resources.
3. Finally, the new framework and resulting classifications should be designed to inform and
   be complemented by the federal statistical system’s operations; federal, state, and local
   program and service providers; and scientific research. This can be achieved by bringing
   together the best minds from each key sector through focused fellowship, training, and competitions.

In this introduction, we analyze and integrate the papers produced for the workshop and the day-of exchanges among the invited experts. The chapter is divided into six key sections that move from the central challenges and opportunities in AI measurement through perspectives informed by research and federal and state agencies, concluding with an analysis of how to better represent AI data through a “follow-the-people” approach to talent flows—rather than using firm surveys to measure AI’s implementation. Workshop participants outlined a number of opportunities to strengthen data collection. 

The Problem and Promise of Measuring AI 

Erik Brynjolfsson, who directs the Stanford Digital Economy Lab, and Susan Athey, a senior fellow at the Stanford Institute for Economic Policy Research, delivered keynotes to begin the workshop. Brynjolfsson started by retelling how Dutch microbiologist Antonie van Leeuwenhoek became the first person to see a microorganism through a microscope. His microscope helped drive other discoveries, igniting a revolution in science that lasted decades. Similarly, Brynjolfsson thought generative AI and other advanced AI models are just at the beginning of their development and will take decades to be fully realized. 

New digital tools will require enormous investments and complementary intangible capital to be productive, but with this transition also comes the possibility of better understanding how firms implement technologies. With the right kinds of data collection in place, economists and researchers could better understand how many software engineers are at each company and what skills they have. Starting at this level of analysis would then allow for skills to be connected to firm output and productivity. In turn, this information could be used to better understand job-posting data and create skill and task taxonomies that would be more granular than the current O*NET system. Moreover, data of this type could help locate nascent superstar firms and illuminate markups for AI companies—a critical question for antitrust authorities. 

Following Brynjolfsson, Athey offered a complementary perspective by focusing on the practical hurdles all collection systems face. It is often assumed that companies have reliable information about their own processes and customers. But in Athey’s experience as the chief economist at the Federal Trade Commission, even the most advanced companies in Silicon Valley face challenges with quantification.

Economists are especially attuned to measurement, estimation, and inference issues. Revealed preference methods, such as surveys, are not always a reliable measure of true preferences or well-being. For example, impulse purchases and the dip in happiness that comes with having children both undermine the connection between respondents’ revealed preference and well-being.

Likewise, measuring digital skills and the use of digital tools presents its own challenges. For one, there is a chicken-and-egg problem when it comes to measuring how interventions influence people’s digital skills. In one study Athey conducted, an AI system gave book recommendations to students that seemingly helped students learn to read. But Athey and her coauthors discovered that a contest of who can read the most actually produced the biggest lift in reading. In other words, it wasn’t the recommendations that mattered but the incentives.

Athey’s keynote offered a warning: Even the most careful researchers and the best-paid tech workers are uncertain of the best and most effective ways to engage users. To solve the problem of mismeasurement, researchers and scholars should be looking beyond current survey tools.

Research Perspectives on AI Measurement, Session 1

In the workshop’s first session, Jason Owen-Smith of IRIS emphasized the high stakes involved, especially with new legislative requirements like the Creating Helpful Incentives to Produce Semiconductors (CHIPS) and Science Act, which mandates measuring science investments’ effects on job creation. Simply put, we lack methodologies, measures, and systems to achieve this goal. Owen-Smith described AI as the “poster child” for these challenges, given the complexities of defining and tracking its impact.

He focused on the measurement challenges AI poses, contrasting traditional classification methods—such as clustering AI-related documents by topic, which he described as largely arbitrary—with the “industries of ideas” approach, which starts with people. This method—pioneered by Julia Lane, a professor in New York University’s Wagner Graduate School of Public Service who helped organize the event and presented later in the day—involves identifying industries by tracking domain experts’ activities and movements in the workforce.

Owen-Smith suggested that state administrative data could be leveraged to follow AI researchers, thereby defining AI industries based on where these researchers are employed. This method could provide insights into the flow of talent and the translation of government research investments into economic and workforce development. He emphasized the importance of following AI researchers and using state administrative data to build a more accurate picture of AI’s impact on regional economies.

Lee Branstetter, the James M. Walton Professor of Economics and Public Policy at Carnegie Mellon University, emphasized the importance of firm-level data in understanding AI’s impacts at a granular level. He discussed how machine learning algorithms can be employed to analyze patent texts, enabling AI-related innovations to be identified and quantified. Branstetter highlighted the proliferation of AI patents as a promising metric for measuring AI-driven innovation, noting that firms filing these patents often significantly increase their productivity, employment, and output.

Branstetter also discussed the global landscape of AI innovation, noting that Japan is the closest peer to the United States. Contrary to popular belief, China lags behind in terms of innovation. Reiterating the power of the industries-of-ideas model, he proposed that tracking leading AI researchers and their movements—particularly as they transition from academia to industry—and the patents they produce could be instrumental in understanding AI’s evolving landscape. By combining patent analysis with public data, such as wage information, it could become possible to identify top AI scientists and follow their career trajectories. 

Branstetter also pointed to potentially using LinkedIn data to track the career paths of students and postdoctoral researchers, who play critical roles in AI development. This approach could further elucidate the economic benefits that arise when a critical mass of AI researchers converges within a startup or other firm. 

After Branstetter’s presentation, Diane Coyle, who codirects the Bennett Institute, provided critical commentary for Owen-Smith and Branstetter, noting that relying on the Elsevier Corpus, which is a large open-access database of Elsevier’s journals, may introduce biases. She suggested that alternative measures, such as the knowledge complexity index, could provide different insights into conceptual distances in the AI field. Coyle also raised concerns about the scalability of both Owen-Smith’s and Branstetter’s resource-intensive methods, stressing the need for not only accurate but also timely and widespread measurements to fully capture AI’s impact. In other words, researchers should be mindful that timely data can often trump more precise data that come months later. 

Coyle ended by asking how these measurements could inform our understanding of macroeconomic and long-term effects. She also touched on the current trends in computing power, ending on a positive note. While Moore’s law may be decelerating, the rapid decline in computing costs could have significant positive implications for the continued advancement of AI technologies, potentially countering the pessimism surrounding AI’s future. 

Research Perspectives on AI Measurement, Session 2 

Prasanna “Sonny” Tambe, an associate professor of operations, information, and decisions at the Wharton School at the University of Pennsylvania, opened the second session by discussing the significant gaps in measuring AI’s impacts, particularly at the firm level. He highlighted the need for better data to understand why some firms succeed in implementing AI while others struggle. AI investment, he noted, remains highly concentrated in a few firms, more so than investments in other digital technologies at so-called frontier firms. But much like other frontier firms, AI-focused firms are investing in infrastructure, workforce training, and high-skill talent acquisition, all of which are unevenly distributed across the economy. 

Tambe pointed out that AI infrastructure is often proprietary and rapidly evolving, making it difficult to identify and measure both tangible and intangible AI assets. He called for a consensus on how to track these assets effectively. Additionally, tracking individual AI professionals is a challenge due to a lack of reliable, up-to-date data. Tambe underscored the difficulties of measuring AI’s impacts due to the vast amount of data and their rapid depreciation over time. 

He also raised questions about AI’s effects on the labor market, particularly for low-skilled workers. There is still significant uncertainty around which skills will remain relevant or become obsolete as a result of AI. He highlighted that workforce training was a key factor in helping workers adapt, but the best approaches to prepare workers in such a dynamic environment remain unclear. 

Nestor Maslej, research manager at Stanford’s Institute for Human-Centered Artificial Intelligence (Stanford HAI), added to the discussion by focusing on the challenges of finding reliable metrics for AI innovation for Stanford HAI’s AI Index. He pointed out that traditional bibliometric data, which track the number of AI-related publications and are central to Stanford HAI’s AI Index, can be misleading. Paradoxically, while the number of AI publications has decreased recently, innovation in the field appears to be increasing, underscoring the difficulty of using bibliometric data to track AI advancements. 

Maslej also highlighted geographic challenges in measuring AI innovation, as bibliometric data do not account for differences in publications’ quality or impact. He stressed the importance of identifying the most cutting-edge research rather than treating all publications equally. Despite these limitations, bibliometric data can still play a role in understanding global AI innovation. Like Branstetter, Maslej offered evidence that China significantly lags behind the United States in AI innovation.

Tracking AI legislation is one of the newest developments for Stanford HAI’s AI Index, and all evidence points to a tidal wave of new regulations. The Organisation for Economic Co-operation and Development has an AI policy tracker of its own, but it lacks a consistent standard for what constitutes regulation or legislation, and it misses state and local regulations. Like the other presenters, Maslej called for better measurement of government spending on AI research and development (R&D) to gauge public-sector involvement and underscored the need for accurate measures.

Will Rinehart provided commentary on the two presentations, arguing that tracking AI legislation is vital, as regulation can create uncertainty and affect innovation. He also pointed to the need for more granular measurements, including tracking state and local AI regulations such as the California Consumer Privacy Act, which significantly affects AI development.

Perspectives from the National Artificial Intelligence Research Resource

Following the panels, Lane presented on the work of the National Artificial Intelligence Research Resource (NAIRR) and the importance of measuring AI’s impact on innovation, diversity, ethics, productivity, and employment. Lane emphasized that Congress is eager to evaluate the returns on its investments in AI, particularly through the CHIPS and Science Act. Effective industrial policy requires timely and accurate data. As much as there is a need for data at the national level, policymakers and researchers need to assess how AI influences labor markets at the state and local levels.

Lane critiqued the use of new PhD graduates as a measure of AI investment, suggesting that while many new AI PhDs are produced each year, their contributions to the field’s innovation and advancement are uncertain. To better track the most impactful researchers, she recommended monitoring AI experts who attend conferences, as they are likely to be driving the most significant innovations. This approach supports a shift from traditional sector-based classifications of industries to an industries-of-ideas model, which emphasizes following researchers as they move through academia and into the labor market.

Following Lane’s presentation, Manish Parashar—who directs the Scientific Computing and Imaging Institute—focused on the barriers that prevent some companies from adopting AI, despite its potential to enhance their operations. Parashar highlighted that identifying companies that have yet to implement AI and understanding the obstacles they face are key to democratizing AI technology. He emphasized the need for greater access to AI infrastructure, computing power, data, and software—especially for firms that lack the resources to integrate these technologies.

Parashar also stressed the importance of educating firms about AI’s capabilities, noting that many companies are unaware of how they can benefit from adopting AI. He advocated for creating a flexible AI R&D ecosystem that fosters collaboration while addressing concerns related to privacy, civil liberties, and national security. By democratizing access to AI R&D, NAIRR could help promote competition, cooperation, and innovation.

He concluded by identifying three critical components of AI development: infrastructure, policy, and measurement. He underscored the importance of providing policymakers with accurate and timely data to ensure informed decision-making. Exploring initiatives such as the national data platform pilot could be a step toward creating a collaborative environment that fosters AI innovation.

Suzette Kent, the former federal chief information officer of the Office of Management and Budget, followed Parashar’s presentation. She focused on the complexity of measuring AI and the lack of clear applicability of these measurements to key stakeholders, including state and local governments. Kent stressed the need for accurate data to inform policy decisions and guide AI investments, highlighting the current gaps in understanding AI supply and demand at the national level. 

She expressed concern about the uncertainty surrounding whether AI investments are translating into real-world innovation, suggesting that without clear metrics, government and public investment in AI may be reduced. Kent called for the development of a more cohesive and comprehensive national view of AI development and deployment to address these uncertainties. 

Perspectives from Data Collection Agencies and Organizations 

Adam Leonard, who directs the Texas Workforce Commission (TWC), then spoke on the underutilization of wage records, highlighting how these datasets can offer deeper insights than can individual data points by identifying broader trends. He provided an example of testing media narratives, such as whether Texas truly faces a teacher shortage, using wage and employment data from the TWC. 

The TWC has advanced the use of administrative and wage data to better understand labor market dynamics, tracking employment patterns and even helping monitor pandemic-related insurance claims. Leonard argued that wage records could be pivotal in tracing individuals’ movement from academia into the private sector, thus illustrating how ideas and innovations disseminate through the economy. Moreover, these records can help identify firm-level investments in AI, particularly through hiring trends. 

Lesley Hirsch, the assistant commissioner of research and information at the New Jersey Department of Labor and Workforce Development, discussed AI’s potential impacts on the workforce, particularly for low- and mid-skilled workers, who were significantly affected by previous rounds of automation. While AI excels at cognitive nonroutine tasks, Hirsch suggested that jobs requiring employees to work on-site may be less vulnerable to displacement. 

Nonetheless, there is great uncertainty surrounding AI’s overall effect on the labor market. It could displace jobs but also create new ones, just as computers have led to the creation of entire industries. Hirsch pointed to initiatives like New Jersey’s career navigator—which uses machine learning to guide workers toward optimal career choices based on their education, skills, and interests—as an example of AI’s potential to assist workers. 

Nela Richardson—ADP’s chief economist and environmental, social, and governance officer—explored two divergent strategies for measuring AI’s impact: following the money or following the people. ADP, which works with 25 million US workers, has focused on matching workers with employers based on relevant skills, with a particular emphasis on AI-related skills. Richardson shared that 85 percent of workers globally expect AI to affect their jobs, but she noted that AI’s greatest impact is likely to be at the task level rather than the job level. She advocated for a national task report that breaks down specific tasks AI could replace or enhance. Such a report could provide a granular understanding of how AI interacts with jobs and tasks, complementing other data sources such as job postings. 

Karin Kimbrough, LinkedIn’s chief economist, shared data on the growing excitement and demand for AI talent. According to LinkedIn’s data, discussions about AI have increased by 70 percent in recent months, with millennials and women particularly engaged in the topic. Kimbrough highlighted the rapid 22 percent growth in AI adoption over 2024, emphasizing that AI’s impact extends far beyond the tech sector. She divided AI’s impact into three categories: augmented, disrupted, and insulated tasks. While many knowledge industries, including economics, sit on the edge of disruption, other industries are poised for augmentation. Kimbrough’s data also suggested that women and Gen Z workers are among the most exposed to AI-related changes. 

Josh Hawley of Ohio State University presented a critical analysis of LinkedIn data, cautioning that it is not always reliable for tracking workforce trends. He underscored the need for more comprehensive data that go beyond individual states and are pooled across regions. Hawley advocated for developing broader datasets that can provide a fuller picture of AI’s labor market impact.

Vipin Arora, the director of the US Department of Commerce’s Bureau of Economic Analysis (BEA), argued for a recalibration of how AI is measured, comparing it to knowing the temperature outside without needing exact degrees to decide how to dress a child. While accurate AI measurement is resource intensive, Arora questioned how precise the measurements need to be to make them actionable. He emphasized the importance of communicating AI measurement findings clearly to policymakers and the public, calling this an ongoing challenge that requires significant attention.

BEA Associate Director David Wasshausen focused on how AI is reflected in economic accounts, comparing AI with the internet in terms of its need for vast infrastructure and wide-reaching impacts. Identifying what qualifies as AI and measuring its effects remains a major challenge, especially given the variance in AI’s application across different cases. Wasshausen noted that AI could generate enormous economic benefits for firms by enabling custom software development and other innovations, which in turn could boost profits and reduce costs.

In their coauthored chapter, Arora and Wasshausen argued for adding further details to current accounting methods. The “supply and use” tables framework that calculates gross domestic product–style measures can already record information about AI investments, whether these are denominated as intermediate purchases of software and consulting services or final expenditures. Arora and Wasshausen note that “a prerequisite for identifying new appr-oaches to characterize hard-to-measure industries is a comprehensive and accurate understanding of how those industries are currently measured.”

Emilda Rivers of the National Center for Science and Engineering Statistics addressed the complexities of measuring AI and its innovation potential, stressing that accurate measurement is critical because it shapes the narrative around AI and influences policy and cultural responses. Rivers advocated for more granular state and local data, highlighting the importance of collaboration and data sharing to capture the full scope of AI’s impact.

Nancy A. Potok, who served as the chief statistician of the United States, concluded the session with a discussion on the evolving role of federal statistics in measuring AI’s workforce impact. She described the decentralized nature of data collection within federal agencies, which creates challenges for cross-agency analysis. Potok emphasized the need for more granular “microdata” at the local level to enable companies, private individuals, and policymakers to make decisions more effectively. AI’s complexity and the lack of coordination across agencies pose significant barriers to understanding and acting on the data currently being collected.

In the chapter she presented on, Potok examined incorporating intellectual property as a category into classification systems, like the North American Industry Classification System and the North American Product Classification System, and more recent attempts to define “factory-less goods” as potential case studies. She pointed out that the BEA uses “satellite accounts” as models for linked sub-accounting of expenditures, which could be applied to AI as well. Potok also suggested using America’s Datahub Consortium for a pilot project linking the federal statistical system datasets together. Such an AI-consortium arrangement would require the Office of Management and Budget to establish terms and standards.

Improving AI Representation in Data

Throughout the sessions, participants stressed the need to move away from current survey methods and move toward a “follow the people” approach—what Lane has been calling an industries-of-ideas approach. It involves tracing the movement of grant-funded engineers from academic labs to industry as a way of understanding how ideas shape business. By linking administrative and wage records collected from companies and industries, we can better understand the flow of technological innovation and talent into the economy of goods and services.

The IRIS Universities Measuring the Effects of Research on Innovation, Competitiveness, and Science dataset—consisting of grant administration records for research awards processed by more than 100 US universities—puts the industries-of-ideas model into practice. Securely linking these grant records with individual- and company-level workforce data produces a map of job demand and tech development. There is already a corresponding National Science Foundation pilot project underway in Ohio, which will be crucial to analyze.² And to top this off, Leonard details in his chapter how he is testing this idea using data from the University of Texas at Austin. 

The follow-the-people approach also aggregates up to the level of companies and industries. Owen- Smith has developed techniques for using “seed sets” of talented AI researchers to identify and map their moves into the economy, and in his chapter, he begins to estimate how employers are using their AI skills. He proposes a threshold for a company’s “AI-ness” that could be generalized to companies with similar AI profiles. 

“Following the people” means following their companies’ records. This raises the question of how to better align terms and timelines with the federal statistical agencies responsible for collecting these records. It also raises the question: Given that the agencies have established frameworks, how can these be remodeled to house new windows on AI development? 

Conclusion 

Reviewing the contributors’ chapters leads to an overwhelming conclusion that an operational definition, or definitions, of AI must be determined from a place of authority. To designate AI engineers, companies, products and services, or related skills, a classification system is required. The United States already has several. But without a definitive statement of terms set via collaboration between government and industry and implemented through statistical agencies and researchers, we risk data and analytical chaos that will leave all segments of the economy and society without sufficient information to make meaningful choices about education, training, and workforce development investments.