Does Giving Employees Generative AI Transform How They Work?

Dillon et al. (2026) examined whether providing knowledge workers with an integrated generative AI tool changed how they spent their time and completed their work. They analyzed telemetry data from a randomized field experiment involving 7,137 employees across 66 large firms who were assigned either access to Microsoft 365 Copilot or a control group over six months. They found that workers who regularly used Copilot spent about two fewer hours per week on email (a 17% reduction), worked 9% less outside normal business hours, and read 6% fewer emails. However, they found little evidence that AI changed workers' overall tasks or workload.

Why This Article Was Selected for The Policy Scientist

This article addresses an important policy question because generative artificial intelligence is rapidly becoming embedded in workplaces, with implications for productivity, labor markets, organizational performance, and the future design of work. Understanding whether AI changes how employees allocate their time, rather than simply whether it increases output on isolated tasks, has broad relevance for employers and policymakers. The authors have made substantial contributions to the growing literature on generative AI and workplace outcomes, and this study extends earlier productivity research by examining real-world adoption over an extended period. Published in the American Economic Review, one of the most influential journals in economics, the article represents a significant contribution to the field. The randomized controlled trial involving more than 7,100 workers across 66 firms provides unusually strong causal evidence supported by a large, high-quality dataset. Although the participating firms were large early adopters, the findings are likely informative for many knowledge-intensive organizations.

Full Citation and Link to Article

Dillon, E. W., Jaffe, S., Immorlica, N., & Stanton, C. T. (Forthcoming). Shifting work patterns with generative AI. American Economic Review: Insights. https://doi.org/10.1257/aeri.20250275

Central Research Question

This study examines whether providing knowledge workers with access to an integrated generative artificial intelligence tool changes how they perform their jobs in everyday organizational settings. Rather than focusing on narrowly defined productivity tasks, the researchers ask whether AI alters workers’ allocation of time, the quantity of work they complete, and the composition of their daily activities. They are particularly interested in understanding whether the benefits documented in laboratory experiments and short-term field studies translate into meaningful changes in real workplaces after employees have had time to learn how to use the technology.

The authors also investigate whether generative AI produces broader organizational changes. They explore whether workers assume different responsibilities, complete more work, reduce time spent outside normal business hours, or create spillover effects for coworkers. These questions recognize that technological innovations often require organizational adaptation before their full effects become visible.

Previous Literature

The study builds upon two related bodies of research. The first examines the effects of generative AI on worker productivity. Previous experiments have consistently shown that AI can improve performance on specific tasks such as writing, software development, customer service, and other knowledge-intensive activities. These studies generally report that AI enables workers to complete individual assignments more quickly or with higher quality. However, most have examined isolated tasks rather than the broader pattern of work over extended periods.

The second literature examines how organizations adopt new technologies. Decades of research have shown that major technological innovations rarely transform workplaces immediately. Instead, firms often experience gradual adoption followed by complementary organizational changes, revised work practices, and managerial innovations that eventually generate larger productivity gains. The authors position generative AI within this broader framework by asking whether workers begin reorganizing their jobs soon after receiving access to integrated AI tools.

Unlike many previous studies, this research follows workers over six months in natural workplace environments. It therefore bridges the gap between controlled experimental evidence and real-world organizational behavior during the early stages of AI adoption.

Data

The analysis relies on an unusually large and detailed dataset collected through a randomized field experiment conducted between September 2023 and October 2024. Microsoft partnered with 66 large organizations representing multiple industries. A total of 7,137 knowledge workers participated in the study, with 3,684 randomly assigned to receive Microsoft 365 Copilot while the remaining participants served as a control group.

The participating organizations were all large firms that relied extensively on Microsoft Office applications. Employees worked in diverse occupations but regularly used Outlook, Teams, Word, Excel, and other Microsoft products. Each participating firm maintained the experimental assignment for approximately six months, allowing workers sufficient time to become familiar with the AI system.

Rather than relying on surveys or self-reported behavior, the study uses telemetry data automatically recorded by Microsoft software. These data measure objective workplace behaviors such as time spent reading and responding to email, participating in meetings, creating documents, and working outside normal business hours. Because the researchers observe software usage rather than document contents, they do not measure work quality or productivity directly. Instead, they focus on changes in work patterns that accompany AI adoption.

The resulting dataset provides highly reliable behavioral measures collected continuously across thousands of workers, substantially reducing concerns about recall bias or inaccurate self-reporting.

Methods

The researchers employ one of the strongest research designs available for estimating causal effects: a randomized controlled trial. Workers within each participating firm were randomly assigned either to receive access to Microsoft 365 Copilot or to continue using existing workplace technologies without Copilot. Random assignment substantially reduces concerns that differences between users and nonusers reflect preexisting characteristics rather than the effects of AI itself.

The primary statistical analysis combines the randomized experiment with a difference-in-differences framework that compares changes in worker behavior before and after treatment while accounting for individual characteristics, calendar effects, and firm-specific trends. This approach improves statistical precision while preserving the causal interpretation created by random assignment.

Because not every worker assigned to Copilot actively used the software, the authors also estimate instrumental variables models that isolate the effects of actual AI usage. Treatment assignment serves as the instrument for AI adoption, allowing the researchers to estimate local average treatment effects among workers who actively used Copilot.

The study further examines heterogeneous treatment effects using modern machine learning methods, investigates possible spillover effects on coworkers, and adjusts for multiple hypothesis testing using sharpened false discovery rate procedures. Together, these methodological choices provide a rigorous framework for estimating causal effects while minimizing the likelihood of false positive findings.

Findings/Size Effects

The strongest evidence concerns email management. Workers assigned access to Copilot spent approximately 1.4 fewer hours per week using Outlook, representing a 12 percent reduction relative to baseline levels. Among workers who actively adopted the technology, estimated time savings reached approximately two hours per week, or a 17 percent reduction.

AI users also changed how they organized their workdays. They completed email activities in fewer sessions, created approximately 2.2 additional hours of uninterrupted work time each week, and reduced email activity outside normal working hours by roughly 9 percent. These findings suggest that AI helped workers process routine communication more efficiently while reducing after-hours work.

The study also found that workers read approximately 6 percent fewer emails without reducing the number of email conversations requiring responses. This pattern indicates that employees processed inboxes more efficiently rather than simply ignoring messages.

In contrast, the researchers found remarkably little evidence that AI changed the overall composition of work. Workers attended essentially the same number of meetings, completed similar numbers of documents, and showed little change in document production or collaborative writing. Although some estimates suggested modest improvements in document completion times, these effects generally lacked statistical significance after adjusting for multiple comparisons.

The researchers likewise found minimal evidence of spillover effects. Coworkers of AI users experienced few measurable changes in their own work behavior, suggesting that productivity gains did not simply shift work onto other employees. Similarly, providing AI access to multiple coworkers did not produce substantially larger organizational effects during the study period.

Overall, the evidence suggests that generative AI primarily improves efficiency within existing work activities rather than fundamentally changing workers' responsibilities during the early stages of organizational adoption. Employees appeared to perform many of the same tasks while spending less time completing them.

Conclusion

This study provides one of the most comprehensive examinations to date of generative AI adoption in real organizational environments. Rather than demonstrating sweeping transformations of work, the evidence indicates that integrated AI tools primarily generate incremental improvements in efficiency, particularly for communication-intensive tasks such as email management.

The findings also illustrate that technological innovations may initially produce modest behavioral changes before broader organizational restructuring occurs. Workers appeared to integrate AI into existing workflows instead of fundamentally redesigning their jobs or assuming new responsibilities. This pattern aligns with earlier research showing that major technological advances often require complementary organizational changes before generating larger productivity gains.

The study's large sample, randomized experimental design, objective behavioral measures, and extended observation period provide unusually credible evidence regarding the early effects of workplace AI adoption. At the same time, the authors acknowledge important limitations. The participating firms were large organizations that voluntarily adopted emerging AI technology, and the researchers could not directly measure productivity, work quality, or long-term organizational performance.

Future research should examine whether broader organizational changes emerge as AI adoption becomes more widespread, firms redesign workflows around these technologies, and workers gain greater experience using increasingly capable AI systems.