JPMorgan Chase, backed by an $18 billion annual technology budget, is rewiring itself for a new era, harnessing agentic and gen AI to perform complex, multistep work.
At the heart of the bank’s AI transformation is LLM Suite, a proprietary platform powered by leading third-party large language models (LLMs) that has automated various processes and placed AI tools directly in employees’ hands.
JPMorgan Chase’s chief analytics officer, Derek Waldron, and McKinsey Senior Partner Kevin Buehler, who have known each other for nearly two decades, sat down with Matt Cooke, McKinsey’s communications and marketing director for financial services, for a wide-ranging discussion about AI’s impact on banking and beyond.
In this conversation, which has been edited for clarity and length, Waldron and Buehler discuss LLM Suite and its role in democratizing AI for JPMorgan Chase employees, the two-pillar approach to value (top-down reimagination of core journeys and bottom-up self-service innovation), and the emerging risks and opportunities that will affect the next stage of AI progress.
Matt Cooke: Could you tell us a bit about your roles?
Derek Waldron: I’m the chief analytics officer at JPMorgan Chase, responsible for overseeing and leading our overall AI program.
Kevin Buehler: I’m a senior partner at McKinsey. I helped start and lead our risk practice and later led our corporate investment banking and cybersecurity practices. Most recently I’ve helped lead our gen AI banking efforts and our AI responsibility efforts.
Matt Cooke: The two of you have some history. Derek, you’re a McKinsey alum. How long have you known one another?
We’ve known each other for about 18 years. The first time we intersected on AI was around 2016 to 2018, at the time of an AI boom, with industry developments like AlphaGo.
Derek Waldron: We’ve known each other for about 18 years. The first time we intersected on AI was around 2016 to 2018, at the time of an AI boom, with industry developments like AlphaGo.1 Kevin and I partnered at McKinsey to lead thinking on how to apply machine learning technology in banking and banking risk management. We developed a lot of pioneering thinking on applying machine learning in domains like credit, risk, fraud, and marketing.
Kevin Buehler: It was fun to work together at a time when the industry was changing dramatically, really getting our hands dirty to see from the bottom up how machine learning and AI work.
Derek Waldron: It was that experience with Kevin that led me in 2023 to jump on the gen AI opportunity. That year, I took on my current role as chief analytics officer at JPMorgan Chase, moving from my previous dual roles as chief of staff for the corporate investment bank and global head of strategy. Gen AI solved the shortcomings we experienced in those early years of AI, and it was evident that it would offer an inflection point for transformation.
Matt Cooke: Derek, a core focus for you is enabling the knowledge workers of the future. Can you talk us through the development of LLM Suite?
Derek Waldron: LLM Suite is JPMorgan Chase’s flagship gen AI platform. JPMorgan has been credited as the first major bank to roll out gen AI at scale to employees. The idea was driven by many factors, including a belief that gen AI would be a highly democratizing technology. If we get the technology into employees’ hands—with change management and training—they’ll be best positioned to innovate and put it to good use. That thesis played out as expected.
Today we have nearly a quarter-million people with access to the platform. A little under half of JPMorgan employees use gen AI tools every single day. People use it in tens of thousands of ways specific to their jobs. Lawyers use it to scan, read, compare, and generate contracts; credit professionals use it to read terms, compare covenants, and extract information; sales professionals and frontline bankers use it to distill information and prepare for meetings. I could go on and on.
When we rolled out LLM Suite to all our employees in 2024, uptake was viral. Most employees would say 2024 was the year they developed a personal relationship with AI. I would describe it as nothing short of a cultural transformation for the bank.
A little under half of JPMorgan employees use gen AI tools every single day. People use it in tens of thousands of ways specific to their jobs.
Matt Cooke: You onboarded well over 100,000 employees within months of LLM Suite’s launch. What did you learn about pacing adoption in a regulated environment? Anything you’d change if you were doing it again?
Derek Waldron: Looking back, we learned things we might do a little differently, but we got a lot right. We took the time to consider all the risk and control considerations. Data security was, without a doubt, the most important consideration. We didn’t rush. Once we were comfortable, we invested properly in change management, accompanying the rollout with training and education so people could make the most of the tools. Word of mouth was an important component. We didn’t mandate the use of LLM Suite by anybody. Instead, we used an opt-in approach, making it available to whoever wanted it and onboarding in phases as people approached us. That created a very interesting and fun dynamic and a bit of healthy competition in the early days, as it was scaling. People would look to their neighbors—some people had it first, others didn’t yet. There was a little bit of a fear of missing out. That social dynamic helped create a culture of adoption.
Word of mouth was an important component. We didn’t mandate the use of LLM Suite by anybody. Instead, we used an opt-in approach.
Matt Cooke: How do you measure meaningful use versus novelty use? Any surprises by function or geography?
Derek Waldron: We look at usage in two ways. People use LLM Suite in their day-to-day for varied purposes, and they see a benefit like they do from other productivity tools—Excel, PowerPoint, Word. We don’t try to quantify hours saved precisely; we know it’s a good thing. Strategically, we focus on domains where transformation will have the biggest impact for JPMorgan Chase: credit, fraud, marketing, technology development, operations, and frontline banker enablement. We invest in these areas and expect the greatest aggregate value.
We often talk about a two-pillar strategy. First, we have a top-down approach with a point of view on the few domains with the most transformative value, and we are strategically organized to push transformation initiatives around those. Second, we have an enormous and exciting bottom-up, federated innovation component where we give employees powerful self-service tools and let them use those in day-to-day ways that, on balance, add up to large productivity gains across the firm.
Matt Cooke: How has LLM Suite evolved?
Derek Waldron: When LLM Suite first came out, it was largely like other LLM providers—a chatbot. It has evolved considerably in the past 18 months. It’s best described as a full ecosystem. The vision is an AI-connected enterprise—powerful AI intelligence in the center connected to team knowledge systems, firm-wide data systems, firm-wide applications, and tools for creating presentations, analyzing data, and creating reports. As more applications, data, and workflows operate in this ecosystem, the possibilities grow exponentially. Building out this ecosystem is foundational to an AI-first JPMorgan Chase.
Matt Cooke: Kevin, considering McKinsey’s banking and technology practices—QuantumBlack, McKinsey Technology—and our work with banks globally, what’s our view of how AI will affect the banking sector’s economics and performance?
Kevin Buehler: In 2024, the global banking industry generated about $1.2 trillion of profits, a record, and about a 10.3 percent return on tangible equity—just above its cost of capital, according to our Global Banking Annual Review 2025.2 At the same time, the industry spends about $600 billion a year on technology, with mixed productivity results.
AI will have several impacts. In 2023, the McKinsey Global Institute estimated $200 billion to $340 billion in savings across the banking industry thanks to AI.3 As gen AI evolves and connectivity rises with agentic AI, AI can do more tasks. In Global Banking Annual Review 2025, we created scenarios looking at bank adoption of AI and its effect on operating cost structure, and customer adoption of AI and the resulting flexibility that reduces inertia. In our central scenario, there’s about $700 billion of cost savings available to banks that adopt AI thoughtfully—up from our estimate two years ago. However, as the industry’s cost curve shifts, much of that $700 billion will likely be competed away and will benefit customers. That raises the competitive bar.
Customers could use AI to find best offers. In retail banking, customers sometimes stay with their bank out of inertia. In products like deposits or credit cards, there’s evidence customers would shift if it were easy. It’s hard to move your primary checking account, given direct deposits and autopays.
In our central scenario, profit pools like credit card lending in North America and deposits in Europe are likely to be affected by AI—not fully intermediated by AI, but enough to make it easier to choose the best card, move balances, or sweep deposits. That could affect a significant portion—as much as 30 percent—of the profit pools in those examples. Net effect: the industry’s return on total equity is likely to drop a percentage point or two, potentially below the cost of capital.
The silver lining is a spread between leaders and others: AI pioneers in banking could see their ROTE [return on tangible equity] increase by up to four percentage points, using their lead to reinvent models and capture value, while slow movers are likely to see declines.
Matt Cooke: Derek, the recruitment of a new breed of technology-enabled financial services professionals is changing as needs evolve, and so are in-house training needs. How is JPMorgan dealing with those?
Derek Waldron: Training needs are varied, just like AI applications. The best way to approach this is segment by segment. First, the employee base at large: We need to train them to get comfortable using and understanding AI tools now available and to think about how to put them to good use daily. We branded a training program, AI Made Easy, at scale, and we continue to update that training. It resonated and tens of thousands of employees took it. We launched marketing campaigns through JPMorgan Chase screens, management channels, and town halls to drive people to use these tools. Word of mouth was also important.
Training needs are varied, just like AI applications. The best way to approach this is segment by segment.
Another population is technologists, who will increasingly want to build sophisticated applications using agentic or gen AI. That skill set is something that needs to be trained. There are new frameworks, capabilities, methodologies, and risks that need to be considered in doing so.
A third category is data scientists. They’ve long been skilled at taking data, building quality models, and deploying them. We’re no longer necessarily building models from scratch. Now, we’re taking powerful models from third parties and deploying them inside applications and systems. Data scientists can now apply their skills to designing, evaluating, and optimizing systems. Advanced, cutting-edge data science capabilities are moving in that direction.
Finally, executives—CEOs and business leaders—need to reimagine operating models, processes, and functions. Value from gen AI won’t come just from giving people tools; business leaders must lead cross-functional teams through transformation in the age of AI. That’s another training need.
Matt Cooke: Jamie Dimon has referred to JPMorgan Chase having thousands of new AI experts. How has LLM Suite affected roles and job categories?
Derek Waldron: Gen AI as a technology is creating new opportunities and needs, and we’ll see new job categories emerge. One of the first was the prompt engineer—a whole new category of people who weren’t software engineers or data scientists but who could understand how to convey business logic or objectives in a language that LLMs can understand and execute.
The prompt engineer is evolving into what we would call a context engineer—getting all the context needed into an AI system so it can make the right decisions. Another emerging job family is knowledge management: taking an institution’s knowledge and data and making it readily available and consumable by AI. It takes curation and structure so the system can navigate clearly and not make mistakes. I believe that will become a bona fide job family for enterprises.
Beyond new job families, existing roles are evolving. Software engineers need to be upskilled to build scalable AI systems based on agents and LLM components. Data scientists need to be upskilled to evaluate and optimize end-to-end probabilistic systems.
The prompt engineer is evolving into what we would call a context engineer—getting all the context needed into an AI system so it can make the right decisions.
Matt Cooke: You talked earlier about encouraging staff to use LLM Suite. How do you upskill nontechnical staff to be effective copilots?
Derek Waldron: Our AI training starts simple and grows in sophistication. The first phase: what an LLM can and cannot do. Second: types of instructions or questions.
Once there’s familiarity with capabilities, we move into how to construct good prompts, with frameworks and examples and constraints. From there we move into more sophisticated approaches: how to pivot the persona of an LLM from maker to checker, or how to use two LLMs to debate a concept to get more creative.
Since launch, we’ve built more modules to accompany new features, including how to conduct thorough research from multiple sources and how to take multiple data sets and conduct sophisticated analyses.
Kevin Buehler: We’re all on a steep learning curve as the technology advances. The more people use these models and understand limitations and weaknesses, the more value they create for the institution.
Derek Waldron: Right, and training doesn’t come just from centrally administered courses—learning from peers is important. At JPMorgan, many teams quickly set up prompt libraries, “prompt of the week” emails, and social channels to share power-user innovations. Word of mouth is one of the best training channels.
Word of mouth is one of the best training channels.
Matt Cooke: Kevin, how is AI affecting employment opportunities for college graduates and the pyramid structure in firms?
Kevin Buehler: It’s a timely question. In the past I’d have hypotheses and anecdotes, but now we have hard data. Early on, it became clear we’d see a change in organizational structure with the rise of AI. Incumbent organizations might move from a pyramid to more of a diamond shape. Some AI-native firms want to jump to an obelisk or column structure with a much leaner approach.4
But we didn’t have a lot of hard data until recently. In the past few months, a couple of interesting papers built on hard data from the Bureau of Labor Statistics5 and payroll data from ADP.6 Both concluded roughly the same thing: AI has had a real, modest effect on entry-level employees. Stanford researchers using ADP payroll data found that early-career workers, ages 22 to 25, in the most AI-exposed occupations experienced a 6 percent decline in employment from late 2022 to July 2025.7 That’s noticeable. If you’re a college graduate in that age range, and you’re pursuing a role in software engineering or customer support, you saw a significant difference versus, say, health aides the same age, whose employment has grown faster than it has for older employees. Over that period, employment was stable or even continued to grow for employees of the same age in sectors less exposed to AI, and for more experienced workers in occupations exposed to AI.
Organizations will need to figure out, if we have fewer entry-level folks, what role do they play? Maybe some of the new roles Derek described. And with a smaller pipeline of future leaders, how do we train and apprentice people in new ways so you still have a rich talent set?
Matt Cooke: Kevin, you and colleagues have written about prioritizing AI investments.8 Why does prioritization matter so much?
Kevin Buehler: Prioritizing focus matters because many institutions end up in proof-of-concept [POC] hell, with more pilots than American Airlines. They start many projects but don’t get them to production, so there’s no significant economic difference. What’s most effective is to look at core workflows important to your economics. If you’re an automotive firm, that might be supply chain, manufacturing, sales, and distribution. If you’re a bank, it might be customer onboarding or mortgages and home equity—origination, underwriting, processing and disbursement, and servicing. It’s important to pick a moderate number of workflows that matter and reengineer them using the best tool for the job. That could be robotic process automation, traditional predictive AI, gen AI, or agentic AI. Choose the right mix to reshape the workflow and redesign end to end for the future. That’s how you capture value.
Matt Cooke: Derek, how do you think about these prioritizations?
Derek Waldron: Kevin’s point about prioritization to avoid POC proliferation is spot on. It’s underestimated how difficult it is to take an AI solution from idea to development to production—considering not just the AI but also the surrounding software, business processes, change management, and human impact. Many enterprises get stuck in POCs because they underestimate that and can’t scale.
That said, the need to prioritize can be taken too literally at the expense of tapping a long tail of innovation. Gen AI is a democratizing technology, empowering individuals. On one hand, we must focus on the most important workflows and put muscle behind them. In addition, we empower the employee base with self-service tools and let them innovate. If it’s truly self-service and the cost of development is near zero, it doesn’t matter whether some remain POCs or how many go to production.
Gen AI is a democratizing technology, empowering individuals.
Kevin Buehler: Gen AI has brought down the barrier to entry and the cost. There are good low-code and no-code solutions that a broader part of the organization can adopt.
Derek Waldron: I did an analysis early in gen AI at JPMorgan when we were deciding where to focus. If you look at job families in the enterprise, you get a few very large ones—engineers, call centers, front office—that align with prioritization. But then you have an enormous tail. A great portion of work is in that tail, which will never fall within prioritized initiatives. You tackle the tail through democratized self-service tools.
Matt Cooke: JPMorgan Chase spends a great deal on technology—more than many technology companies. The bank is set to spend $18 billion on technology in 2025.9 How do you prioritize investments to capture ROI?
Derek Waldron: We spend a lot of money on technology based on the conviction that leading in technology yields strategic benefits. When it comes to AI, we do careful financial analysis to understand ROI for the overall AI program. Since the inception of our AI program, the gross benefit that we credit to AI efforts has been growing steadily at about 30 to 40 percent per year, and we believe that trend will continue. We can cite that because we have a mature financial discipline tracking financial benefit at each AI initiative level, both before embarking and when going into production.
Kevin Buehler: There’s debate about how much prioritization is needed. If you line up projects by impact, there are big areas requiring substantial reengineering, and there’s a long tail of self-service activities. When you add up all those self-service activities, how does their impact compare to the sum of the first few areas? What’s your perspective?
Derek Waldron: Both are important value drivers. The democratizing nature of gen AI—accessible to the long tail and enabling federated bottom-up innovation—has more value than most people credit. That said, while productivity gains create capacity, they don’t necessarily translate into cost takeout. An hour saved here and three hours there may increase individual productivity, but in end-to-end processes these snips often just shift bottlenecks. If cost takeout or end-to-end metrics—like an 80 percent reduction in response time—are the goals, you need to prioritize the journey and reimagine it end to end. Both strategies are very important. The long tail’s value may not lead to cost takeout, but it creates organizational capacity that will show up in operating leverage benefits.
Matt Cooke: What’s on the near-term road map for analytics and AI at JPMorgan Chase?
Derek Waldron: About half our employee base uses gen AI tools every day, so the next horizon of value won’t come from more adoption. We’re focused on two things.
The first thing is making the tools more powerful, which largely comes from better connectivity inside JPMorgan. Tools need to be connected to more applications, data, and systems to enable more insights. That connection is a significant undertaking for enterprises of all sizes because of the size and fragmentation of tech stacks. The second thing is maximizing value by leveraging all technologies—taking journeys and reimagining them using these tools.
Matt Cooke: Is there a particular project you’re excited about?
Derek Waldron: I’m most excited about driving connectivity. Every few weeks, we add another data set, application, or connection into the LLM Suite ecosystem. That means every month I see new problems I couldn’t solve last month.
Every few weeks, we add another data set, application, or connection into the LLM Suite ecosystem. That means every month I see new problems I couldn’t solve last month.
Matt Cooke: Turning to the question of risk, Kevin, what should the industry prepare for in the next five years?
Kevin Buehler: In the adoption of AI, benefits exceed risks, but there are real risks to be considered. Let’s take a few. Most organizations have dealt with keeping private information, including customer information, private, ensuring tools keep information separate from LLM training data. There are ways to do that, and most institutions have solved it.
I do worry about shadow IT: Without something like LLM Suite, there’s a real temptation to boost productivity by going to the largest consumer-grade AI tool and typing in information you shouldn’t. Consumer-grade AI tools don’t necessarily have the same guardrails in place. There’s ongoing litigation over how information was used to train some LLMs; I worry about how that reverberates throughout the industry. We need a solution there—maybe like ASCAP and BMI solved royalties for music nearly 100 years ago.10
There are issues about malicious use. I’ve heard deepfakes that are really persuasive. I’ve encountered CEO and CFO fraud, where someone pretended to be a CEO or a CFO, and it was really hard for the organization to understand those instructions are fake. I’ve seen cybersecurity issues, man-in-the-middle attacks, and spear phishing with much greater frequency.
As you put systems in place in a customer-facing way, that increases the risk. Most people start with internal deployments. When facing customers, you need more safeguards. You don’t want the full power of an LLM serving a customer for an address change on a credit card account; you want it narrowly tailored. You probably need guardrails and even other models monitoring inputs and outputs to ensure models are fit for purpose and not offering opinions they shouldn’t. We’re helping folks ensure that occurs.
Derek Waldron: That’s a very good list. I’d add access management and entitlements of agents. As agents become more heavily used and access systems, applications, and other agents, how access credentials get passed and used is a real problem. The industry needs to uplift identity and access management frameworks in a world of agents.
Another area is trust as tools become more powerful and people move from asking a question to giving problems that take minutes or hours to run autonomously. How do these systems become trustworthy? It’s easy today to audit a source and verify. When an agentic system does a cascading series of analyses independently for a long time, it raises questions about how humans can trust that. We need innovation to address it.
When an agentic system does a cascading series of analyses independently for a long time, it raises questions about how humans can trust that. We need innovation to address it.
Kevin Buehler: I agree. A related issue: We often rely on a human in the loop to oversee AI. As models perform correctly 85 percent, 90 percent, 95 percent of the time, human reviewers may let their guard down and start to assume a model is always right. The reviewers might not check the output as carefully as they should.
Derek Waldron: Exactly. As we face these problems, we’ll realize the way we work with and manage AI systems will become more like how we manage people today. I’ve said that AI will make everyone a manager.
Kevin Buehler: Learning how to manage AI is a future skill. One organization brought together the head of technology and head of HR roles—overseeing both as head of work, whether executed by humans or machines. Not many have talent to be head of HR and head of technology, but it’s an interesting vision.
