How a handful of tech companies control billions of minds every day

I want you to imagine walking into a room, a control room with a bunch of people, a hundred people, hunched over a desk with little dials, and that that control room will shape the thoughts and feelings of a billion people. This might sound like science fiction, but this actually exists right now, today.

I know because I used to be in one of those control rooms. I was a design ethicist at Google, where I studied how do you ethically steer people's thoughts? Because what we don't talk about is how the handful of people working at a handful of technology companies through their choices will steer what a billion people are thinking today. Because when you pull out your phone and they design how this works or what's on the feed, it's scheduling little blocks of time in our minds. If you see a notification, it schedules you to have thoughts that maybe you didn't intend to have. If you swipe over that notification, it schedules you into spending a little bit of time getting sucked into something that maybe you didn't intend to get sucked into. When we talk about technology, we tend to talk about it as this blue sky opportunity. It could go any direction. And I want to get serious for a moment and tell you why it's going in a very specific direction. Because it's not evolving randomly. There's a hidden goal driving the direction of all of the technology we make, and that goal is the race for our attention. Because every new site -- TED, elections, politicians, games, even meditation apps -- have to compete for one thing, which is our attention, and there's only so much of it. And the best way to get people's attention is to know how someone's mind works. And there's a whole bunch of persuasive techniques that I learned in college at a lab called the Persuasive Technology Lab to get people's attention.

A simple example is YouTube. YouTube wants to maximize how much time you spend. And so what do they do? They autoplay the next video. And let's say that works really well. They're getting a little bit more of people's time. Well, if you're Netflix, you look at that and say, well, that's shrinking my market share, so I'm going to autoplay the next episode. But then if you're Facebook, you say, that's shrinking all of my market share, so now I have to autoplay all the videos in the newsfeed before waiting for you to click play. So the internet is not evolving at random. The reason it feels like it's sucking us in the way it is is because of this race for attention. We know where this is going. Technology is not neutral, and it becomes this race to the bottom of the brain stem of who can go lower to get it.

Let me give you an example of Snapchat. If you didn't know, Snapchat is the number one way that teenagers in the United States communicate. So if you're like me, and you use text messages to communicate, Snapchat is that for teenagers, and there's, like, a hundred million of them that use it. And they invented a feature called Snapstreaks, which shows the number of days in a row that two people have communicated with each other. In other words, what they just did is they gave two people something they don't want to lose. Because if you're a teenager, and you have 150 days in a row, you don't want that to go away. And so think of the little blocks of time that that schedules in kids' minds. This isn't theoretical: when kids go on vacation, it's been shown they give their passwords to up to five other friends to keep their Snapstreaks going, even when they can't do it. And they have, like, 30 of these things, and so they have to get through taking photos of just pictures or walls or ceilings just to get through their day. So it's not even like they're having real conversations. We have a temptation to think about this as, oh, they're just using Snapchat the way we used to gossip on the telephone. It's probably OK. Well, what this misses is that in the 1970s, when you were just gossiping on the telephone, there wasn't a hundred engineers on the other side of the screen who knew exactly how your psychology worked and orchestrated you into a double bind with each other.

Now, if this is making you feel a little bit of outrage, notice that that thought just comes over you. Outrage is a really good way also of getting your attention, because we don't choose outrage. It happens to us. And if you're the Facebook newsfeed, whether you'd want to or not, you actually benefit when there's outrage. Because outrage doesn't just schedule a reaction in emotional time, space, for you. We want to share that outrage with other people. So we want to hit share and say, "Can you believe the thing that they said?" And so outrage works really well at getting attention, such that if Facebook had a choice between showing you the outrage feed and a calm newsfeed, they would want to show you the outrage feed, not because someone consciously chose that, but because that worked better at getting your attention. And the newsfeed control room is not accountable to us. It's only accountable to maximizing attention. It's also accountable, because of the business model of advertising, for anybody who can pay the most to actually walk into the control room and say, "That group over there, I want to schedule these thoughts into their minds." So you can target, you can precisely target a lie directly to the people who are most susceptible. And because this is profitable, it's only going to get worse.

So I'm here today because the costs are so obvious. I don't know a more urgent problem than this, because this problem is underneath all other problems. It's not just taking away our agency to spend our attention and live the lives that we want, it's changing the way that we have our conversations, it's changing our democracy, and it's changing our ability to have the conversations and relationships we want with each other. And it affects everyone, because a billion people have one of these in their pocket.

So how do we fix this? We need to make three radical changes to technology and to our society. The first is we need to acknowledge that we are persuadable. Once you start understanding that your mind can be scheduled into having little thoughts or little blocks of time that you didn't choose, wouldn't we want to use that understanding and protect against the way that that happens? I think we need to see ourselves fundamentally in a new way. It's almost like a new period of human history, like the Enlightenment, but almost a kind of self-aware Enlightenment, that we can be persuaded, and there might be something we want to protect. The second is we need new models and accountability systems so that as the world gets better and more and more persuasive over time -- because it's only going to get more persuasive -- that the people in those control rooms are accountable and transparent to what we want. The only form of ethical persuasion that exists is when the goals of the persuader are aligned with the goals of the persuadee. And that involves questioning big things, like the business model of advertising. Lastly, we need a design renaissance, because once you have this view of human nature, that you can steer the timelines of a billion people -- just imagine, there's people who have some desire about what they want to do and what they want to be thinking and what they want to be feeling and how they want to be informed, and we're all just tugged into these other directions. And you have a billion people just tugged into all these different directions. Well, imagine an entire design renaissance that tried to orchestrate the exact and most empowering time-well-spent way for those timelines to happen. And that would involve two things: one would be protecting against the timelines that we don't want to be experiencing, the thoughts that we wouldn't want to be happening, so that when that ding happens, not having the ding that sends us away; and the second would be empowering us to live out the timeline that we want.

So let me give you a concrete example. Today, let's say your friend cancels dinner on you, and you are feeling a little bit lonely. And so what do you do in that moment? You open up Facebook. And in that moment, the designers in the control room want to schedule exactly one thing, which is to maximize how much time you spend on the screen. Now, instead, imagine if those designers created a different timeline that was the easiest way, using all of their data, to actually help you get out with the people that you care about? Just think, alleviating all loneliness in society, if that was the timeline that Facebook wanted to make possible for people. Or imagine a different conversation. Let's say you wanted to post something supercontroversial on Facebook, which is a really important thing to be able to do, to talk about controversial topics. And right now, when there's that big comment box, it's almost asking you, what key do you want to type? In other words, it's scheduling a little timeline of things you're going to continue to do on the screen. And imagine instead that there was another button there saying, what would be most time well spent for you? And you click "host a dinner." And right there underneath the item it said, "Who wants to RSVP for the dinner?" And so you'd still have a conversation about something controversial, but you'd be having it in the most empowering place on your timeline, which would be at home that night with a bunch of a friends over to talk about it. So imagine we're running, like, a find and replace on all of the timelines that are currently steering us towards more and more screen time persuasively and replacing all of those timelines with what do we want in our lives.

It doesn't have to be this way. Instead of handicapping our attention, imagine if we used all of this data and all of this power and this new view of human nature to give us a superhuman ability to focus and a superhuman ability to put our attention to what we cared about and a superhuman ability to have the conversations that we need to have for democracy. The most complex challenges in the world require not just us to use our attention individually. They require us to use our attention and coordinate it together. Climate change is going to require that a lot of people are being able to coordinate their attention in the most empowering way together. And imagine creating a superhuman ability to do that.

Sometimes the world's most pressing and important problems are not these hypothetical future things that we could create in the future. Sometimes the most pressing problems are the ones that are right underneath our noses, the things that are already directing a billion people's thoughts. And maybe instead of getting excited about the new augmented reality and virtual reality and these cool things that could happen, which are going to be susceptible to the same race for attention, if we could fix the race for attention on the thing that's already in a billion people's pockets. Maybe instead of getting excited about the most exciting new cool fancy education apps, we could fix the way kids' minds are getting manipulated into sending empty messages back and forth.

(Applause)

Maybe instead of worrying about hypothetical future runaway artificial intelligences that are maximizing for one goal, we could solve the runaway artificial intelligence that already exists right now, which are these newsfeeds maximizing for one thing. It's almost like instead of running away to colonize new planets, we could fix the one that we're already on.

(Applause)

Solving this problem is critical infrastructure for solving every other problem. There's nothing in your life or in our collective problems that does not require our ability to put our attention where we care about. At the end of our lives, all we have is our attention and our time. What will be time well spent for ours?

Thank you.

(Applause)

Chris Anderson: Tristan, thank you. Hey, stay up here a sec. First of all, thank you. I know we asked you to do this talk on pretty short notice, and you've had quite a stressful week getting this thing together, so thank you. Some people listening might say, what you complain about is addiction, and all these people doing this stuff, for them it's actually interesting. All these design decisions have built user content that is fantastically interesting. The world's more interesting than it ever has been. What's wrong with that?

Tristan Harris: I think it's really interesting. One way to see this is if you're just YouTube, for example, you want to always show the more interesting next video. You want to get better and better at suggesting that next video, but even if you could propose the perfect next video that everyone would want to watch, it would just be better and better at keeping you hooked on the screen. So what's missing in that equation is figuring out what our boundaries would be. You would want YouTube to know something about, say, falling asleep. The CEO of Netflix recently said, "our biggest competitors are Facebook, YouTube and sleep." And so what we need to recognize is that the human architecture is limited and that we have certain boundaries or dimensions of our lives that we want to be honored and respected, and technology could help do that.

(Applause)

CA: I mean, could you make the case that part of the problem here is that we've got a naïve model of human nature? So much of this is justified in terms of human preference, where we've got these algorithms that do an amazing job of optimizing for human preference, but which preference? There's the preferences of things that we really care about when we think about them versus the preferences of what we just instinctively click on. If we could implant that more nuanced view of human nature in every design, would that be a step forward?

TH: Absolutely. I mean, I think right now it's as if all of our technology is basically only asking our lizard brain what's the best way to just impulsively get you to do the next tiniest thing with your time, instead of asking you in your life what we would be most time well spent for you? What would be the perfect timeline that might include something later, would be time well spent for you here at TED in your last day here?

CA: So if Facebook and Google and everyone said to us first up, "Hey, would you like us to optimize for your reflective brain or your lizard brain? You choose."

TH: Right. That would be one way. Yes.

CA: You said persuadability, that's an interesting word to me because to me there's two different types of persuadability. There's the persuadability that we're trying right now of reason and thinking and making an argument, but I think you're almost talking about a different kind, a more visceral type of persuadability, of being persuaded without even knowing that you're thinking.

TH: Exactly. The reason I care about this problem so much is I studied at a lab called the Persuasive Technology Lab at Stanford that taught [students how to recognize] exactly these techniques. There's conferences and workshops that teach people all these covert ways of getting people's attention and orchestrating people's lives. And it's because most people don't know that that exists that this conversation is so important.

CA: Tristan, you and I, we both know so many people from all these companies. There are actually many here in the room, and I don't know about you, but my experience of them is that there is no shortage of good intent. People want a better world. They are actually -- they really want it. And I don't think anything you're saying is that these are evil people. It's a system where there's these unintended consequences that have really got out of control --

TH: Of this race for attention. It's the classic race to the bottom when you have to get attention, and it's so tense. The only way to get more is to go lower on the brain stem, to go lower into outrage, to go lower into emotion, to go lower into the lizard brain.

CA: Well, thank you so much for helping us all get a little bit wiser about this.

Tristan Harris, thank you. TH: Thank you very much.

(Applause)

Translated by Lipeng Chen

Reviewed by Jing Peng

我想让你们想象 走进一个房间， 这个房间是个控制室， 里面有一群人，大概上百人， 都在有控制器的桌子旁伏案工作， 而那间控制室 会决定上亿人的 思想和情感。 这听上去像是科幻小说， 但是今天，此刻， 它真真切切地发生着。

因为我就在这种控制室工作过。 我曾是谷歌的设计伦理学家， 在那里我曾研究过如何 名正言顺地左右人们的想法。 因为我们不常讨论屈指可数的 几家科技公司的那么几个人， 是如何通过他们的选择 来影响现今上亿人的思想。 因为当你拿出手机， 他们将设计下一步会如何 以及应该推送哪些新闻， 就像在我们的脑子里设定出了小小的模块。 如果你看到一个消息， 这会促使你形成 或许原本没有的想法。 如果你忽略了这个消息， 他们还会设计让你在原本 不关注的东西上 多花些时间。 当我们谈到科技时， 我们倾向于把它描绘成 充满机遇的蓝图。 天马行空，任你想象。 而我想严肃地告诉你们， 为什么它选择了某个特定的方向。 因为科技的发展不是随机的。 在我们创造的所有科技背后 都隐藏着明确的目标， 那个目标就是追求我们的注意力。 因为所有新的网站—— TED演讲，选举，政客， 游戏，甚至是冥想软件—— 都需要为同一种东西彼此竞争， 那就是我们的注意力， 而注意力是有限的。 获得人们注意力的最佳办法 就是了解我们的大脑是如何运作的。 大学时我曾在一个叫做 说服性技术实验室的地方 学到了许多说服别人的技巧， 来获取人们的注意力。

一个简单的例子就是YouTube。 YouTube希望你在 它上面花的时间越多越好。 那么他们做了什么呢？ 他们增加了自动播放下一个视频的功能。 效果应该说很不错。 人们在上面花费了更多的时间。 如果你在Netflix工作， 看到他们这么做之后会觉得， 这会减少我的市场份额， 我也要自动播放下一集。 但如果你是Facebook的员工， 你会说，这减少了我的市场份额， 所以我会在你点击播放按键前 自动播放新闻推送中的所有视频。 所以网络不是随机发展的。 我们感觉上瘾的原因在于 我们的注意力成了被竞争的对象。 我们知道这样发展下去会如何。 科技并非中性的， 它变成了那些可以深入了解 人脑运作的人之间的竞争。

给你们举个Snapchat的例子。 有人可能不太了解， Snapchat是在美国青少年中 排名第一的交流方式。 如果你们和我一样用短信交流， 那么Snapchat就是青少年的短信， 有近一亿人在使用。 他们发明了一种叫Snapstreaks的功能， 它会展示两个人 持续互动的天数。 换句话说，他们所做的 就是给了两个人不想失去的记录。 因为如果你是一个青少年， 连续交流了150天， 你不想失去这些东西。 试想一下孩子脑子里 已被设定好的时间模块。 这不是理论空谈，而是事实； 当孩子们去度假时， 即使他们不能使用 Snapstreaks，也会把密码 告诉至多五个朋友， 来帮助他把Snapstreaks接力下去。 他们要同时关注大约30件这种事情， 所以每天他们就忙于拍拍画，拍拍墙， 拍拍天花板来消磨时间。 这甚至不能被称为真正的交流。 我们可能会这么想， 他们用Snapchat的方式 就像我们曾经用电话聊八卦一样， 应该是没问题的。 但人们忽略了在上世纪七十年代， 当你们用电话聊八卦时， 在另一头并没有数百名工程师 准确知道你们的心理活动， 并精心策划着如何让你们加强联络。

如果这让你有些愤怒了， 请注意，你刚刚才意识到这点， 愤怒也是获取你注意力的一种好方式。 因为我们不会选择愤怒。 它会自动发生在我们身上。 如果你负责Facebook的新闻推送， 信不信由你， 人们愤怒的时候你会受益。 因为愤怒不仅仅是人们 在情感和空间上的一个反应。 想要与人分享这份愤怒， 所以我们会按下分享键，然后说， “你能相信他们说的吗？” 而这种愤怒真的特别容易获得注意力， 甚至于如果Facebook 可以在展示惹人愤怒的消息 和一般的消息之间进行选择的话， 他们会选择向你发布让你愤怒的消息， 不是因为刻意的原因， 只是那会更好的获得你的注意。 消息控制室不用对我们负责， 它只对最大化获得人们的注意力负责， 鉴于广告的商业模型， 它也对可以花钱进入 控制室的人负责，他们可以要求说 “那一组人，我想把 这些东西灌输给他们。” 所以你可以定位， 你可以准确定位到 那些最易受到影响的人。 因为这有利可图， 所以只会变得越来越糟。

今天我在这里（跟大家讲） 是因为代价已经太高了。 我不知道还有什么问题比这更紧急的， 因为这个问题隐藏在所有问题之中。 它并不仅仅是剥夺了我们的注意力 和选择生活方式的自主权， 更是改变了我们进行交流的方式， 改变了我们的民主意识， 改变了我们与他人交流， 维系关系的能力。 这影响到了每个人， 因为亿万人的口袋里 都装着这个东西（手机）。

那么我们要如何解决这个问题呢？ 我们需要对科技和社会 做三个大胆的突破。 首先，我们需要承认我们可以被说服。 一旦了解 我们是可以安排大脑去想一点其他事情， 或无计划地占用一些时间， 我们难道不能利用这点认识 来改变现状吗？ 我认为我们需要 以全新的方式审视自己。 就像是人类历史上的新篇章， 就像启蒙运动， 但这是自省式的启蒙运动， 意识到我们可以被说服影响， 意识到有些东西需要我们去保护。 第二点是，我们需要新的 模型和责任系统， 以便让世界变得越来越好， 也越来越有影响力时—— 也就是更能说服我们时—— 那些控制室里的人们 才会对我们负责，且行为对我们透明化。 道德说服只有当 说服和被说服者的 目标一致时才存在。 这就涉及到对热门举措的怀疑， 比如广告的商业模型。 最后， 我们需要一次科技重塑， 因为一旦你开始了解人的这种本性， 你就可以控制上亿人的时间—— 想象一下，有些人有这样的欲望， 他们想做什么， 他们想思考什么， 他们想感受什么， 他们想了解什么， 而我们被吸引到这些不同的方向。 数亿人会跟随着这些不同的方向。 试想一下一个完整的科技复兴 将会指导我们准确有效的 分配时间。 这包含两个方面： 一是防止我们把时间花在 在不想花的地方， 阻止我们产生不想形成的想法， 即便提示音响了， 我们也不会被牵着鼻子走； 二是让我们按照所期待的时间轨迹生活。

给你们举个实际的例子。 今天，比如你的朋友 取消了和你共进晚餐， 你感到有些寂寞。 这一刻你会做什么呢？ 你打开了Facebook。 在那一刻， 控制室里的设计者要做一件事， 那就是要你盯着屏幕，时间越长越好。 现在想象一下， 如果设计者规划了另一条时间轴， 利用他们所有的数据， 用最简单的方法 帮你约出你关心的人呢？ 试想如果消除社会中所有的孤单， 才是Facebook想要为人们实现的理想。 或试想另一个对话。 比方说你想要在Facebook上 发表备受争议的言论， 能这么做是很重要的， 谈论争议性话题。 现在有一个巨大的评论栏， 它就像是在问你， 你想要输入什么东西？ 换句话说，它设定了你将要 继续在屏幕上做的事情。 试想如果有另外一个提问框跳出来问你， 怎么花费时间最好？ 你点击 “办个晚餐聚会” 紧接着下面有个选项问道， “谁想要去这个晚餐？” 虽然你仍想对 有争议性的话题进行讨论， 但你的时间花在了最好的地方， 晚上在家里和一群朋友 讨论那个话题。 想象我们正在使用“查找并替代” 功能， 把所有那些正在促使我们 花越来越多的时间在屏幕上的事情， 用我们在生活中的真实意愿 来逐一替换掉。

事情本不必如此复杂。 与阻碍我们的注意力相反， 试想如果我们利用所有这些数据和功能， 加上对人类本性的全新认识， 给我们以超人的能力来集中注意力， 来关注我们应该关心的事， 来进行民主所需要的 互动交流。 世界上最复杂的挑战 要求我们不仅独立的运用我们的注意力， 还要求我们同心协力。 气候变化需要许多人 使用最有力的方式 彼此协作。 试想创造出这种超人能力会是什么情景。

有时世界上最紧要，最关键的问题 不是假象出来的未来事物。 有时最紧要的东西 恰恰是我们眼前的东西， 是已经影响了上亿人思想的东西。 也许，与其对新兴的增强现实， 虚拟现实等炫酷玩物感到着迷， 使得它们也会成为 注意力竞争中的一员， 我们不如改进亿万人口袋里 影响注意力的那个东西。 也许，我们与其对 新潮炫酷的教育软件感到兴奋， 还不如想想如何拯救那些已被操纵， 来回发送空洞消息的孩子们。

（掌声）

也许，我们与其 对假象的未来中的最大主角， 人工智能感到担心， 不如解决当下就存在的 人工智能问题， 就是那些争夺注意力的新闻推送。 这就好比与其逃跑和殖民新的星球， 我们可以解决现在所在星球的问题。

（掌声）

解决这个问题 是解决其他问题的关键所在。 在你生命之中或是我们共同的问题中， 没有一件事不需要我们 把注意力放到我们关心的事情上去。 归根到底，在一生中， 我们共同拥有的就是注意力和时间。 时间会被我们自己充分利用吗？

谢谢大家。

（掌声）

克里斯·安德森： Tristan，谢谢你，请留步。 首先，谢谢你。 我知道我们很晚才通知你要做这次演讲， 这周准备这个演讲， 压力确实很大，再次谢谢你。 有些听众可能会说， 你抱怨的不就是上瘾吗， 而对于真正这么做的人来说， 他们确确实实是觉得有趣的。 所有这些设计策划 使得用户获取了十分有趣的内容。 这个世界从未如此有意思过。 这有什么问题吗？

特里斯坦·哈里斯： 我认为这确实有趣。 换个角度思考一下 举个例子，还用YouTube， 你总是想让下一个视频更有趣。 你想要在建议下一个视频 这件事上越做越好， 但即使你可以建议一个 所有人都想要观看的完美视频， 其实只是在吸引人们 盯住屏幕这件事上越来越好。 所以这个等式缺失的是 知道我们的极限在哪里。 比如，你想要让YouTube对 入睡这样的事儿有所了解。 Netflix的CEO最近说过， “我们最大的对手是Facebook， YouTube和睡眠。” 所以我们需要认识到 人体本身是有极限的， 在我们的生活中有某些界限或方面 是要得到尊重的， 而科技是可以帮助我们实现这些的。

（掌声）

CA：你是否可以说明一下 我们是不是对人性的认识太天真了？ 从人类偏好来看， 这些东西是可以合理化的， 我们有这些极其棒的算法 为人类的偏好优化上做着贡献， 但是，是什么偏好呢？ 有我们确实关心的 偏好， 也有我们只是下意识点击的偏好。 如果我们在每个设计中 多加一点对人类本性的关注， 这会不会是一种进步呢？

TH：那是肯定的。我认为现在 科技基本上只是利用我们的下意识， 找到最好的方式让我们自然而然地 对微不足道的事上瘾， 而不是问我们在生活中 时间要如何花费才最有意义。 什么才是完美的时间安排， 这可能包括后面的事情， 例如你利用在这儿的最后一天 参加TED，是很好的利用了时间吗？

CA：如果Facebook和Google 或任何一个人上来对我们说， “嘿，你想要我们为你的思考进行优化， 还是优化你的下意识？你来选择。”

TH：是的。这可能是个方法。

CA：你说到可说服性， 对我来说这个词很有趣， 因为在我看来有两种不同的可说服性。 有一种可说服性是我们现在正尝试的， 关于推理，思考以及做出论证， 但是我觉得你在谈论另一种， 更加内在的本能的一种可说服性， 即在不经思考下就被说服了。

TH：是的。我十分关注 这个问题的原因是， 我曾在斯坦福的 说服性技术实验室学习， 那里就是教学生这些技巧。 那有专门的论坛和讨论会， 教授人们如何用隐蔽的方式 来获得人们的注意力， 从而策划人们的生活。 正因为大部分人不知道它的存在， 才使得这个演讲如此重要。

CA：Tristan，咱们都认识 许多来自这些公司的人。 他们当中许多人也在这里， 我不知道你的想法， 但是就我的经验而言， 他们是心存善意的。 大家都向往更美好的世界。 他们确实是这样想的。 我不认为你的意思是 这些人都是坏人。 只不过是这个系统产生了不经意的结果， 超出了我们的控制范围——

TH：在争夺注意力 这件事儿上，没错儿。 当你要获得众人的注意力时， 便成了经典的厮杀， 而且特别激烈残忍。 取得更多注意的唯一办法， 只有深入大脑， 深入愤怒，深入情感， 深入本性。

CA：感谢你帮助我们 对这个问题有了更深的认识。

特里斯坦·哈里斯，谢谢你。 TH：非常感谢。

（掌声）