As firms selling artificial intelligence models release a stream of new versions promising ever better performance and lower costs, companies adopting their technology are sampling what the field has to offer. 

San Francisco-based OpenAI grabbed mind share in November 2022 when it launched query-response system ChatGPT; it staked out market share a few months later when it began letting other developers build atop the large language models (LLMs) powering its chatbot. But neither it nor any of its competitors have cornered the field.  Companies are choosing between AI inputs from dozens of startups and technology giants, some designed specifically for business applications and others shared as open source. 

Some firms are taking an all-of-the-above approach, testing and swapping models based on what they do best. In a recent small-sample survey of enterprise AI buyers, each with budgets in the millions, research firm CB Insights found that 97 per cent were using LLMs from two or more developers. Firms of all sizes are taking a similar tack. “We want to remain flexible,” said Sam Talasila, head of LLMs at Toronto-based Wealthsimple. 

The fintech scale-up recently built an internal tool that classifies the data from calls between clients and customer-service agents or portfolio managers, determining what topics they discussed and whether the clients are positive or negative toward particular products. The system uses a version of Llama-3, an open-source LLM from Meta, which Wealthsimple has fine-tuned with its own data. To transcribe and summarize conversations, the company uses Whisper, an OpenAI speech-recognition model.

Toronto-based Ada employs GPT-4 Turbo for the part of its system that plans how its AI “agents” should go about fulfilling users’ customer service requests. Then it uses other, smaller models to execute those steps. Both Wealthsimple and Ada have also experimented with AI systems from Toronto-based Cohere.

Firms are looking for LLMs that get details right when it matters. The systems produce responses based on the data on which they were trained, or to which they have access. Most are fine for “a great one-time demo,” said Alon Talmor, CEO of Ask-AI. But when companies move to deploy a proper product, “they find that the actual accuracy of models is not great.” The startup, which has offices in Tel Aviv and Toronto, sells a workplace assistant to employers built atop publicly available models. But it uses its own technology to sharpen their outputs. 

In addition to accuracy, Wealthsimple’s developers consider whether the application they’re working on requires real-time results—some tasks can be batched and processed overnight, which is cheaper—as well as the provider’s security standards and approach to handling personally identifiable information. The firm uses models that can run in-house for sensitive data, while an automated “prompt inspector” ensures employees aren’t inputting any into externally hosted systems.

Price is also a factor. The technology is “medium-expensive” right now, according to Talasila. Projects to prove out applications cost a few thousand dollars a month in LLM access fees, but scaling them for company-wide deployment would run the firm upwards of $100,000. Wealthsimple is finding savings by swapping in open-source models. 

The firm also isn’t committing to one model-maker. It’s set up a toy chest of LLMs for staff to play with from Cohere, Google, Meta, OpenAI and the French open-source provider Mistral. “It’s just too soon to pick a winner,” said CTO Diederik van Liere. “Actually, I felt very good about [Wealthsimple’s approach], given what happened over the weekend at OpenAI.” Van Liere was  speaking to The Logic in November, three days after OpenAI fired CEO Sam Altman and two days before he was reinstated; the tumult caused some clients to try other LLM providers.

All the major model-makers offer application programming interfaces (APIs), which allow clients’ developers to add generative capabilities to their products or internal systems. But there are plenty of buyers for AI tools that incorporate someone else’s LLMs. 

Last October, New Zealand’s Xero launched a feature that answers users’ how-to questions about its small-business accounting platform, based on its library of help content. While Xero has engineers who could work with an LLM API, it bought the new system from Quebec City’s Coveo, a publicly traded seller of AI search and recommendation tools. “We’re comfortable with technology [but] that doesn’t make us the experts in AI,” said Nigel Piper, Xero’s executive general manager of the customer team.

Coveo’s expertise, by contrast, is how to apply AI—and do so in ways that meet the needs of enterprise clients. “LLMs are a commodity,” according to CEO Louis Têtu, who said the value lies in “the engineering you do around it—how you use it.” Coveo’s technology takes model outputs and adds the indexing, relevance and security needed to produce useful results.  

Businesses may also find it easier to buy a ready-made AI product and let the vendor keep up with the rapid change in the field, said Ada CEO Mike Murchison. “The cost of being on the wrong model is really high when the rate of improvement is so quick.”

While many companies are still shifting from trying out generative AI to fully deploying it, model-swapping is here to stay. Developers can now survey all their options in marketplaces hosted by their cloud providers and compare them on the many industry benchmarks. “We don’t believe there’s going to be one large language model to rule them all,” said Eric Gales, managing director for Canada at Amazon Web Services. 

Indeed, some businesses may find that small language models are a better fit for their applications. Startups like Montreal-based Reliant AI are building domain-specific AI systems trained on relevant data for clients in fields like life sciences.  

Wealthsimple’s Talasila expects the competition among model-builders to continue. “Right now, it is hard to see one company coming out as a winner,” he said. “Every announcement is [one] company leapfrogging another one.”