Toronto-headquartered chip startup Untether AI is touting the top speed and efficiency of its artificial intelligence hardware in tests against established players like Nvidia, Dell and Lenovo. Here’s what you need to know.

The tests: The AI boom is driving increased demand for processing power to train and run machine-learning models. MLCommons, an industry group of major tech companies and startups, is trying to set industry benchmarks for those systems. Its evaluation, called MLPerf and released on Wednesday, measures how fast each participating firm’s hardware produces results for selected, already trained models. Companies cross-check each others’ submissions.

The test-taker: Untether makes chips designed for inference, the process of running an AI model after it’s been trained—for example, identifying an object on the road spotted by a sensor on an autonomous vehicle. 

Founded in June 2017, it’s raised over US$153 million to date, according to PitchBook data, from backers including the Canada Pension Plan Investment Board, Intel Capital and Radical Ventures. 

The results: “We see this as a watershed moment,” said Bob Beachler, Untether’s vice-president of product. The firm’s entries—so-called accelerator cards containing a chip and other components—did particularly well in two categories. 

The first is a speed test, using a model that classifies objects in images. The Canadian startup’s hardware was quicker than hardware systems filled with market-leader Nvidia’s chips. 

That’s good news for Untether, which is marketing its current technology for applications that require fast visual processing out in the real world like agricultural technology, autonomous vehicles, robotics and low-Earth orbit satellites. 

The second is an energy efficiency test, using the same model but measuring the amount of power systems required in conditions like those of a data centre. Untether’s accelerator used fewer watts to produce the same output as Nvidia’s flagship H200 offering. 

“Energy costs money,” said Beachler. “If I can do the same work at a quarter of the power, that means a quarter of the cost over the lifetime of the card.” The AI industry also needs more efficient hardware to mitigate data centres’ environmental impacts. 

The caveats: Untether only tested its systems on that one image recognition system, and topped categories with relatively few entrants. Other firms like Nvidia are touting their performance for large language models (LLM), used in popular generative AI tools like chatbots and assistants.

Some chip companies don’t participate at all. “We typically haven’t submitted to MLPerf, because it doesn’t tend to represent real-world use cases,” Jonathan Ross, CEO of Groq, told reporters in June. 

(“He’s wrong,” Beachler said, citing MLPerf’s use of Llama-2.7B, a widely used LLM from Meta.)

What’s next: No data centre operator or carmaker buys chips purely based on MLPerf results, but they give the startup a calling card as it seeks clients. “Our customers have their own neural networks that they want to run,” Beachler said. That means putting Untether’s hardware through more tests.