Using AI like ChatGPT costs more than other internet searches. It’s a billion dollar problem

Estimated reading time: 5-6 minutes

MOUNTAIN VIEW, Calif. — As Alphabet looks past a chatbot bug that helped erase $100 billion from its market value, another challenge is emerging from its efforts to add generative artificial intelligence to its popular Google search: the cost.

Tech executives are discussing how to harness AI like ChatGPT while considering the high expenses. OpenAI’s wildly popular chatbot, which can write prose and answer search queries, has “maddening” computing costs of pennies or more per conversation, the startup’s chief executive, Sam Altman, said on Twitter.

In an interview, Alphabet President John Hennessy told Reuters that having an AI exchange known as a Great Language Model likely costs 10 times more than a standard keyword search, although fine-tuning will help reduce expenses quickly.

Even with revenue from potential chat-based search ads, the technology could contribute to Mountain View, Calif.-based Alphabet’s bottom line with several billion dollars in additional costs, analysts said. Its net income was nearly $60 billion in 2022.

Morgan Stanley estimated that Google’s 3.3 trillion search queries last year cost about a fifth of a cent each, a number that would increase depending on how much text the AI ​​has to generate. Google, for example, could face a $6 billion increase in spending by 2024 if ChatGPT-like AI were to process half of the queries it receives with 50-word responses, analysts say. . Google is unlikely to need a chatbot to handle navigational searches on sites like Wikipedia.

Others arrived at a similar bill in different ways. For example, SemiAnalysis, a research and advisory firm focused on chip technology, said adding a ChatGPT-like AI to search could cost Alphabet $3 billion, an amount limited by chips. Google internals called Tensor Processing Units, or TPUs, and other optimizations. .

A “neural network”

What makes this form of AI more expensive than conventional research is the computing power involved. Such AI depends on billions of dollars worth of chips, a cost that must be spread over their multi-year useful life, analysts said. Electricity also adds cost and pressure to businesses with carbon footprint targets.

The process of processing AI-powered search queries is known as “inference”, in which a “neural network” loosely modeled on the biology of the human brain infers the answer to a question from a training anterior.

In a traditional search, by contrast, Google’s crawlers scanned the Internet to compile an index of information. When a user types in a query, Google offers the most relevant answers stored in the index.

Alphabet’s Hennessy told Reuters: ‘It’s the inference costs you need to reduce’, calling it ‘a problem of a few years at worst’.

Alphabet is under pressure to complete the challenge despite the expense. Earlier this month, rival Microsoft held a high-profile event at its headquarters in Redmond, Wash., to show plans to integrate AI chat technology into its Bing search engine, leading executives targeting Google’s 91% search market share, per Similarweb. estimate.

Unexpected responses

A day later, Alphabet was talking about plans to improve its search engine, but a promotional video for its AI chatbot Bard showed the system answering a question inaccurately, fomenting a stock drop that slashed $100 billion. its market value.

Microsoft then came under scrutiny when its AI allegedly made threats or said it liked to test users, prompting the company to limit long chat sessions it says ’caused’ problems. involuntary responses.

Microsoft chief financial officer Amy Hood told analysts the benefit of gaining users and ad revenue outweighs the expense as the new Bing rolls out to millions of consumers. “It’s an additional gross margin for us, even at the price of the service we’re discussing,” she said.

And another Google competitor, search engine CEO Richard Socher, said the addition of an AI chat experience as well as apps for graphics, video and other generative technologies is growing. expenses between 30% and 50%. “Technology is getting cheaper at scale and over time,” he said.

A source close to Google warned that it was early to determine exactly how much chatbots might cost, as effectiveness and usage vary widely depending on the technology involved, and AI is already powering products like search.

Still, footing the bill is one of the top two reasons search and social media giants with billions of users didn’t deploy an AI chatbot overnight, says Paul Daugherty, Chief Technology Officer of Accenture.

“The first is accuracy, and the second is that you have to scale it the right way,” he said.

Make the math work

For years, researchers at Alphabet and elsewhere have studied how to train and run large language models cheaply.

Larger models require more chips for inference and therefore cost more. The AI ​​that dazzles consumers with its human authority has swelled in size, reaching 175 billion so-called parameters, or different values ​​that the algorithm takes into account, for the updated OpenAI model in ChatGPT. The cost also varies depending on the length of a user’s request, measured in “tokens” or chunks of words.

A senior tech executive told Reuters that such AI remains prohibitively expensive to put in the hands of millions of consumers.

“These models are very expensive, and so the next level of invention is going to reduce the cost of training these models and inferring so that we can use it in every application,” the exec said in a covert diary. ‘anonymity.

So far, OpenAI computer scientists have figured out how to optimize inference costs through complex code that makes chips more efficient, a person familiar with the effort said. An OpenAI spokesperson did not immediately comment.

“An Open Question”

A longer-term problem is how to reduce the number of parameters in an AI model 10 or even 100 times, without losing precision.

“How to eliminate (parameters) most efficiently is still an open question,” said Naveen Rao, who previously led Intel’s efforts on AI chips and is now working to reduce AI computational costs. thanks to his startup MosaicML.

In the meantime, some have considered charging for access, like $20 per month OpenAI subscription for better ChatGPT service. Tech experts also said a workaround is to apply smaller AI models to simpler tasks, which Alphabet is exploring.

The company said this month that a “smaller” version of its massive LaMDA AI technology will power its Bard chatbot, requiring “significantly less computing power, allowing us to scale to more users.”

Asked about chatbots like ChatGPT and Bard, Hennessy said at a conference called TechSurge last week that more focused models, rather than a do-it-all system, would help “get the cost under control.”

Contributor: Greg Bensinger

Related stories

Most recent business stories

Jeffrey Dastin and Stephen Nellis

More stories that might interest you

Leave a Comment