AI improved the productivity of a Fortune 500 software company

Generative artificial intelligence (AI) has gained popularity recently. However, little research has been done on the economic impact of generative AI. Generative AI can learn to accomplish tasks in circumstances in which it is not given direct instructions, as well as tasks that previously needed knowledge that could only be gained through direct experience. Generative AI has performed well in laboratory spaces, but some fear that it will fall short in real-world applications. Skeptics fear that generative AI may face unfamiliar problems and provide unproductive responses in important scenarios.

In their paper “Generative AI at work” (National Bureau of Economic Research, Working Paper 31161, April 2023), Erik Brynjolfsson, Danielle Li, and Lindsey R. Raymond evaluated the impact of a generative AI tool that provided conversational guidance for customer support agents. The AI chat assistant was deployed at a Fortune 500 software firm. The chat assistant would read customer chats and then provide customer service agents with suggested responses. However, the agents were free to disregard the AI recommendation and remained responsible for the conversation.

The authors found that the AI tool increased the number of successfully resolved customer issues by 14 percent. The call center agents were able to increase their productivity in three ways: a decrease in the time it took an agent to respond to a chat, an increase in the number of chats that an agent could respond to, and an increase in the share of successfully resolved conversations. This productivity gain was realized mostly by the company’s newer, less experienced agents. Brynjolfsson, Li, and Raymond believe that the productivity increase for less experienced agents occurred because the generative AI captured and emulated the characteristics of the most productive agents.

Brynjolfsson, Li, and Raymond found that newer and less experienced agents benefited the most from the AI chat assistant. Those agents saw a 34 percent increase in the number of issues they were able to resolve per hour. On the other hand, longer tenured and more skilled workers saw little effect on their productivity. The researchers found that the quality of these agents’ conversations may have even decreased. The researchers observed that agents who used the AI recommendations the most saw the largest increases in productivity. Additionally, the researchers found that the use of AI increased workers’ on-the-job learning. When the AI chat assistant was not functioning, the researchers found that the agents were still more productive than their pre-AI baseline.

Call center agents often face a daunting task. Agents routinely speak with angry and agitated customers. The use of an AI chat assistant may increase the chances that agents sound inauthentic and robotic. However, in addition to improving productivity, the authors also found that the agents who used the AI chat assistant had improved customer sentiment and received friendlier treatment.