Aidan Gomez, CEO of Cohere, a Toronto-based artificial intelligence startup, has sent a letter to staff criticizing effective altruism, a controversial philosophy that’s highly influential in the AI world and reportedly played a role in ChatGPT-maker OpenAI’s recent power struggle. Here’s what you need to know:
What is effective altruism?: Effective altruists aim to use reason and data to orient their charitable giving, careers and other aspects of their lives towards doing the greatest good for the greatest number of people. Recently, effective altruists have shifted their focus away from improving health in impoverished nations to AI safety.
This shift is based on the rising influence of a related philosophy called longtermism, which argues we should focus on actions with the greatest positive impact on distant generations, including preventing human extinction at the hands of AI. Prominent effective altruist William MacAskill popularized the concept in his 2022 book What We Owe the Future.
Gomez’s take: As first reported by Bloomberg, Gomez criticized the movement’s “self righteousness” in a Wednesday letter to staff. “When people begin to believe so deeply that they are uniquely qualified to benefit, or even save, humanity, they tend to be willing to take extreme actions towards that end,” he wrote. Gomez has previously said he believes it’s a mistake to focus on hypothetical existential risks to humanity posed by AI rather than immediate ones.
In an email to The Logic, Cohere spokesperson Josh Gartner confirmed the contents of the letter, but declined to comment further.
Why weigh in now?: Tensions between OpenAI CEO Sam Altman and the firm’s safety-focused board members with links to effective altruism appear to have contributed to Altman’s dramatic firing and re-hiring, which has captivated the tech world since Friday. Tasha McCauley and Helen Toner, two of the board members involved in firing Altman who have since been replaced, have links to organizations that are backed by or explicitly affiliated with effective altruism, although OpenAI has denied any directors are adherents.
Another board member involved in Altman’s firing, Canadian Ilya Sutskever, has expressed views about AI risk that are consistent with those held by effective altruists and longtermists. The chief scientist of OpenAI had spearheaded a technical breakthrough (a model called Q*, or “Q-Star”) prior to the boardroom coup, The Information reported, raising concerns among safety-focused staff about whether the company had sufficient safeguards in place.
Who else is weighing in?: In an opinion piece for The Information, OpenAI investor Vinod Khosla said the firm’s “board members’ religion of ‘effective altruism’ and its misapplication could have set back the world’s path to the tremendous benefits of artificial intelligence.” Shopify CEO Tobi Lütke, Coinbase CEO Brian Armstrong and tech reporter Kara Swisher have all criticized the philosophy in the wake of the OpenAI saga.