OTTAWA — Frances Haugen left Facebook to let the world in. The former product manager turned whistleblower a year ago, after the social platform shut down its civic-integrity team. “I have to figure out how to get more people involved, because Facebook can’t heal itself,” she recalls deciding.

Last September, The Wall Street Journal began publishing a series of stories based on a cache of internal Facebook documents Haugen sent to them and to U.S. regulators and lawmakers suggesting the firm was aware of and had failed to fix problems affecting children, minority communities and other groups. 

The articles revealed a program called XCheck that exempted celebrities and politicians from the kind of enforcement that regular users might face, and detailed research showing nearly a third of teen girls said Instagram worsened their negative feelings about their bodies. (Meta CEO Mark Zuckerberg has said the documents were taken out of context; Kevin Chan, then-head of public policy for Canada, told The Logic the coverage ignored the platform’s positive effects.)

Haugen has since testified before lawmakers in the U.S. and Europe and founded Beyond the Screen, a non-profit focused on accountability for social platforms. 

A year after she stepped forward, Haugen sat down with The Logic for an interview in Ottawa on Tuesday to discuss the problems with algorithms, how regulation can address those issues, and the role she hopes Canada will play in a global attempt to rein in social platforms.  

This interview has been edited and condensed for length and clarity.

You’ve identified algorithms or engagement-based ranking systems as a key issue. What is the problem with them?

Facebook has run the same study in at least four different contexts: They take a blank account, and all they did was click, each day, on the first few items and join groups or follow hashtags that are suggested. The algorithms slowly, over time, pushed them toward more extreme topics. They have done this for centre-left, centre-right, Indian politics, and healthy eating in the case of kids on Instagram. Within a couple of weeks, for the centre-left or centre-right, they were getting to things like, “We should kill all the Republicans,” or white genocide. In the case of food content, you go from something like healthy recipes to pro-anorexia content within two or three weeks. 

The reason that happens is these feeds are not random sets of content. When they’re trying to decide what to show you, they might start out with 40,000 or 100,000 possible pieces of content. They have all this context on you and [there’s an] optimization function. In the case of Facebook, it’s [whether] you click on it, comment on it, reshare it. 

There’s a [Facebook] white paper from 2018 [in which] Mark Zuckerberg says that the danger with engagement-based ranking is that people are drawn to more extreme content. When we ask them after they’ve clicked it or put a like on it, “Did you like that content?” They’ll say, “No, I didn’t.” That’s because humans are drawn to it. 

Facebook consistently says it has no incentive to show harmful content, because no one wants to run ads against that. How can both of those things be true? 

No one at Facebook said, “We want to make you angry,” or, “We want to show you violence.” At the same time, they had a business incentive. They used to optimize for “How long could they keep you on site?” And they switched over to saying, “Can we get interaction from you?” They didn’t do that because they were like, “We don’t have enough extreme content today.” They did it because they saw they were getting a decline in content production, and found the only thing that was sustainable to get people to produce more content was giving people more social incentives. So they started optimizing feeds, not necessarily for your happiness, but for the gift that you are giving your friends in the form of likes or comments. 

Facebook has this really hard trade-off. They know that advertisers don’t like to advertise against extreme content. They also know that if they accidentally take down totally fine content, it angers you, and will make you not produce content in the future. So they’re always in this tension: “What’s the least we can take down that will let advertisers feel like we’re doing a good job?” versus “What’s the least we can do that will keep users from actively leaving the system?” 

Many of the worst instances of content on Facebook that make news are things that are already illegal in the places that it operates—hate speech, incitement to violence and so on. Is it that Facebook just isn’t following those laws, or that the laws don’t really work for a digital-platform context? 

There’s two ways you can write these laws, to be grossly reductive. One strategy is to say, “It’s illegal to post hate speech. It’s illegal to post incitement of violence. You can’t have any of it—it’s all illegal.” The other is to say: “You need to understand what the risks are of your platform. Why are you distributing hate speech? How much hate speech are you delivering? Is it going into just a few communities where it’s oversaturated?” With any other harm type on Facebook, that’s what’s happening. 

The difference between those two laws is one is implementable. You can say, “You need to be public about your risks, [and] have plans for reducing those risks. We need to be able to monitor them, make sure you’re actually executing on them, that you’re making progress.” Process—having public transparency, being collaborative, being iterative—works. 

We have no current way with technology—and we will not have a way for at least 20 years—of being able to take down all the hate speech. The way our computers work today, there’s so much context and nuance in terms of what would make something inciting to violence that you either have to accept that you’re going to accidentally take down 10 or 20 times as much innocent content as actual incitement, or that you’re going to miss most of it. That’s what we saw in Facebook’s documents: Facebook was taking down three to five per cent of hate speech, and that was [what] they could monitor.

So therefore you need that more transparency and risk-assessment-based approach. 

The reality is we don’t know how to build these systems yet. 

One of my core issues is about the talent pipeline. We’re going to Mars because SpaceX can hire aeronautical engineers with PhDs. They don’t train that person for 10 years and then hire them. When it comes to places like Twitter, we see the consequences of the fact that everyone who has my skill set was trained inside these companies. I got paid way too much money to do my job because there were only a couple hundred of me in the world. 

We have architecture schools in almost every university. We do that because if you design this room just the way it is, versus slightly different, it could have a totally different experience. Online spaces are just like physical spaces. You can make very subtle tweaks in how these spaces are designed, and you’ll get wildly different outcomes. Right now, we don’t have even the basic tools to teach people that intuition. We focus on content, because we have personal experience with it. We don’t focus on the things that would actually scale and be effective in making these platforms safer.

A lot of the proposals that we’ve seen to deal with the outcomes of platforms—Canada’s online-harms proposal—

Like the U.K.’s bill.

—or Senator Amy Klobuchar’s COVID disinformation bill would likely require algorithms to handle the policing because of the sheer scale of content. Can we effectively use algorithms to police the negative outcomes of algorithms?

The downside of having that talent pipeline, where we’re relying on the companies to develop expertise in the technical side [and] the policy side, is that we don’t have legislative aides that really have a good grasp on it, either. [So] we fall back on solutions that seem obvious at first, but in reality are very difficult to execute. I don’t think saying you have to take down every single piece of misinformation is feasible. 

You could imagine a world where we put a large public investment in decentralized content moderation. Instead of having factories in Africa or the Philippines where content moderation takes place, imagine if we [came] up with a democratic process. What’s the equivalent of jury duty for content moderation? 

Ethan Zuckerman at [the University of Massachusetts] Amherst has this wonderful metaphor: Our platforms today are designed for us to be subjects, ruled by a king. We either accept the rules, or we don’t and leave. But we can’t really act as citizens. Citizens participate in the governance of their systems. They do work to have democracy. 

Until we have that world, we need things like, Facebook should have to publish, every week, a random 10,000 posts that come out of Canada, weighted by impressions. We would see that because you’re in the information ecosystem [of] the United States, you get a much more dangerous experience than the United States does, because Facebook actually invests in safety [there]. In Canada, because there’s less legislative threat, they unquestionably don’t tack on as much of that safety. 

You just described a type of transparency. Are we talking about opening up the black box of algorithms so people can see how the decisions are made, or about seeing what the outcomes of those decisions are?

I think you have to do both. There are certain transparency bills that say, “You need to publish your recommender system.” But as long as we’re using [AI] neural nets to do some of these [content] ranking processes, it doesn’t actually matter that you see the inputs—the model is combining them in ways that we don’t understand.

I think a lot of what’s missing right now from how we govern social platforms is we don’t just govern these [kinds of] systems with laws. Cars [in the U.S.] are not just governed by the Department of Transportation. They’re also governed with litigators that understand what it means to cut a corner, and that hold people responsible. Or we have Mothers Against Drunk Driving—informed citizens that understand enough context on the problem to feel confident standing up and saying, “We need to have safer highways.” 

Have government officials in Canada reached out to you to consult on online harms or privacy or other digital policy files here?

I have just started having conversations with people around online-safety issues. I’m very happy to help. We’re founding a non-profit that’s focused on capacity-building across the ecosystem of accountability. 

Canada is a small market for Facebook and Instagram. Where, realistically, can Canada play in regulating platforms?

I think there’s a huge opportunity for Canada, working in coalition with other countries. 

These platforms are not designed for people who speak smaller languages. Canada has things in common with the Nordics. There’s four million French Canadian speakers. French Canadian is different than French—it has different contexts, different slang. The issues that are relevant in politics in even English-speaking Canada are different than what’s relevant in the United States. A misinformation classifier that’s tuned off of the memes and issues that are popular in the U.S., bolting [that] on [in Canada] after the fact leaves Canada behind. 

Remember when the U.K. [published] their children’s-safety bill last year—there’s 65 million people in the U.K., and yet, the standard for safety for Instagram rose that week. The platforms don’t want to do one-offs; they want to find whoever has the highest floor and go to that floor.

There’s many smaller, less powerful countries in Africa, Southeast Asia, South America that are the most impacted, and yet they’re never going to get to write a comprehensive bill. It’s very expensive to run a regulator—it’s very hard to get enough critical mass of experts that you can have meaningful opinions. Working in concert is a super interesting opportunity. You get a non-EU perspective, a non-U.S. perspective, but you can also get critical mass enough to be able to do meaningful things.

You spoke to Canadian Innovation Minister François-Philippe Champagne in Paris in November. He told The Logic that conversation influenced the protection-of-children aspects of the new privacy law. 

That was a really good conversation. He’s the one who suggested the coalition thing. I met with him and he was like, “How can Canada help you out? I’ve built coalitions before. We can do this.” I had been so heads-down in the EU for the [Digital Services Act, a platform-safety law], I hadn’t really been thinking about what we do next. 

What should be the scope of regulation?

A first wave is things like transparency, because right now, we don’t have a common vocabulary. You could do very small changes to these platforms and make them much safer. It’s not like we need to give up social media tomorrow in order to be safe. 

A common thing across all of these platforms—be it TikTok, be it Facebook—is we don’t have anything that helps people go to bed. Instead of popping up a thing saying, “Hey, it’s 10 p.m. Want to go to bed?” You may be a better person than me, but I just hit “Dismiss” on that. 

No, I’m not a better person than you. 

On all social platforms, the 99th percentile consumes a hundred times as much content [as the median user]. They influence the model more—they leave 100 times as much signal. The Atlantic [showed that] you can go and look at the comments from those people, and you can tell they’re not happy people. They’ve been stuck in the doom loop for too long. If you meaningfully help those people go to bed—you let them pick their own bedtime, but you slow the app down progressively so they can actually soft-transition off—you end up giving less misinformation to everyone else.

Phase 1 is mandatory disclosures. The public gets to ask [social firms] questions and get real answers. Phase 2 is: We now know enough that we can start talking about a duty-of-care standard. Should you be mandated to have ways to help people go to bed? What is enough to keep an under-13-year-old off of a platform? Laws where we come in and assume we already know the answers to those questions will give us much less than we deserve. 

We have so much opportunity here. I want to start making T-shirts that say, “Fatalism is a sign someone is trying to steal your power.” Facebook has spent hundreds of millions of dollars telling us the only thing that can save us is content moderation. So many people feel so overwhelmed. But the reality is we have lots and lots of options.