Goldman Sachs CIO is ‘anxious to see results’ from GenAI, but moving carefully | The AI Beat

With all the generative AI hype swirling among evangelists, one might think that the Fortune 500 is galloping wildly towards putting large language models (LLMs) into production and turning corporate America into one big chatbot. To that, I say: “Whoa, Nelly!” — meaning, think again.

That’s because for all the C-suite executives out there feeling generative AI FOMO and getting pressure from CEOs to move quickly to develop AI-centric strategies, things are actually moving far slower than you might imagine (or AI vendors, who warn companies about falling behind, might want). As I reported back in April, there’s certainly no doubt that executives want to access the power of generative AI, as tools such as ChatGPT continue to spark the public imagination. But a KPMG study of U.S. executives that month found that a solid majority (60%) of respondents said that while they expect generative AI to have enormous long-term impact, they are still a year or two away from implementing their first solution.

Goldman Sachs CIO says company is ‘deeply into experimentation’

Consider Marco Argenti, CIO at Goldman Sachs — who told me in a recent interview that the leading global investment banking, securities and investment management firm has, nearly a year after ChatGPT was released, put exactly zero generative AI use cases into production. Instead, the company is “deeply into experimentation” and has a “high bar” of expectation before deployment. Certainly this is a highly-regulated company, so careful deployment must always be the norm. But Goldman Sachs is also far from new to implementing AI-driven tools — but is still treading slowly and carefully.

While Argenti told me that he thinks “We’re all anxious to see results right away” in areas like developer and operational productivity, as well as revolutionizing the way knowledge workers work and producing content, when I asked him what it would take to put its experimental use cases of generative AI into production, he said it required “feeling comfortable about the accuracy.” He added that this needs to hit a certain threshold “in which we feel comfortable that the information is correct and the risks are actually well managed.”

In addition, he said that Goldman Sachs needs a clear expectation of a return on investment before deploying generative AI into production. One use case that is clearly showing “huge” progress, he explained, is software development, where he said the company is seeing 20-40% productivity gains in its experiments and has a target of getting 1,000 developers fully equipped to use generative AI tools by the end of the year.

Goldman Sachs has no plans to create its own LLM from scratch

Argenti emphasized that Goldman Sachs is not simply randomly running AI models. From the very beginning, he said that the company has implemented a platform that ensures technical, legal, and compliance checks. The front-end server has measures in place to filter out any inappropriate content, while all interactions are logged to ensure the data used is fully authorized. This system, he explained, channels all operations through a single, user-friendly chat interface the company has developed, which allows it to effectively direct interactions and ensure a streamlined and compliant user experience.

That said, the company has no plans to build its own LLM from scratch.

“I might be completely wrong,” he said, but “I don’t believe at this point…that it is necessary to start from scratch.” However, Goldman Sachs is definitely fine-tuning existing models and using retrieval-augmented generation (RAG), an AI framework for retrieving facts from an external knowledge base to ground LLMs on accurate, up-to-date information.

“At the end of the day, a lot of stuff out there is generic, but there is data that we have, that is the most important thing,” he said. “With that data, the combination of RAG and fine-tuning.”

Generative AI requires ‘methodical and thoughtful’ work

ROI is on everyone’s mind when it comes to generative AI, Argenti explained: “Everybody’s trying to seek confirmation that there is usefulness, or really trying to kind of see the ROI for these investments,” he said, adding that Goldman Sachs is ready to expand its generative AI experimentation beyond software development, but that those are “a question mark — we’re not going to throw hundreds of millions to just try things and let them fail. I mean, right now, we have safe experimentation, really good parameters of what we expect. ” Goldman Sachs want to be methodical and thoughtful, he said, because “it’s very easy to get carried away.”

Argenti recalled a recent dinner with over 30 CEOs at large companies, in which he warned against a hyper-focus on productivity enhancement with generative AI. “That will not cause differentiation, sooner or later everyone will have it…it will establish a new baseline on productivity,” he said. “It’s also trying to find cycles and courage of investing in things that might not be profitable today, that are more about how is our business going to change? What’s the new role of an advisor, what’s the new role of an investor, or a trader? We’ve been very careful in trying to strike that balance in a way that we’re still very conscious of the fact that this could actually not just be sustaining technology but also disruptive technology.”

It’s a practical approach, he added — focused on the concrete application of generative AI in specific use cases.

This may not have the exciting, cowboy-like spin of “moving fast and breaking things,” but I think if even a financial services leader like Goldman Sachs — which has long been forward leaning in the AI space — is treading carefully on its journey to generative AI applications, there’s no doubt that other enterprise companies are moving just as slowly and deliberately.

And this isn’t to say that Argenti hasn’t made more hype-producing statements to the media about AI — at a fintech conference in May, he told the audience that AI will make workers “superhuman.” But Argenti also told me that he had early access to ChatGPT, DALL-E and other tools and very quickly saw their potential in the enterprise, and that the company’s CEO and board has been “incredibly supportive” of generative AI efforts. That hasn’t changed the careful trajectory of experimentation and testing.

But while Goldman Sachs may not exactly be galloping at top speed into the Wild West of generative AI, Argenti clearly maintained that the company won’t be falling behind, either.

“We have a lot of horses in the race,” he said. “So we feel pretty good about that.”