[MUSIC PLAYING]

I’m Ezra Klein. This is “The Ezra Klein Show.”

[MUSIC PLAYING]

A core focus of the show this year is going to be attention. But not your attention, not my attention, not attention as a capacity of the individual, where we give you hacks to grayscale your iPhone or meditate in the morning or eat better food. Our attention — attention is seen as a collective resource, as a public good.

Attention is, in total, the depth of thought and consideration a society can bring to bear on itself, its problems, its opportunities — everything from how to find economic prosperity, to solving climate change, to strengthening our democracy, or for that matter, doing the reverse of any of those things. All of it depends on our capacity to pay attention, on the quality of the attention we pay, and on the condition we’re in when we pay attention.

But like any collective resource, attention, it can be polluted, it can be exhausted. And I think to a large extent, it has been. And to see how and why, we have to get really deep into the business of attention. So today’s episode is part of that inquiry.

Tim Hwang was director of Harvard M.I.T. Ethics and Governance of AI Initiative. He was a global public policy lead for A.I. at Google.

And for our purposes, most importantly, he’s the author of this weird fascinating book called, “Subprime Attention Crisis,” which is a really good explanation of the business model responsible for our collective attention today.

I always worry when approaching this topic. It is a problem of something we all think we know about. We see the banner ads. We know we’re tracked across the internet.

We’re familiar with this, but we’re really not — the scale of it, the technology that really underpins it, the pervasiveness and centrality of this business model to almost all of the information and entertainment we now consume, the way it is something completely different than it used to be. When you get into the technical underpinnings of our whole attention economy, it’s almost wondrous. It will really make you — makes me step back and marvel. But our attention here isn’t just being bought and sold. It’s being used and changed. And really tracking the plumbing of this economy is where we have to start to see how. And so I asked Hwang on the show to talk me through it more. As always, my email, [email protected].

[MUSIC PLAYING]

Tim Hwang, welcome to the show.

Ezra, thanks for having me on.

So the argument at the heart of the book is that the internet as we know it flows out of its business model. So what is that business model?

Yeah, absolutely. So the book “Subprime Attention Crisis” is really about advertising on the internet, which over the last few decades has become the sort of primary funding model for everybody — everything from the biggest platform companies to the smallest publishers and websites on the internet. And it’s specifically about a very particular kind of advertising, which is known as programmatic advertising, which is the use of algorithms to buy and sell attention online.

And something that you talk through in the book that I’ve heard you talk about — and that I think people know, but when you hear it, it’s really remarkable — is just how central this is. Go through a bit of how much advertising supports the things that we use on the internet every day.

Yeah, so the fascinating thing about advertising is that it is really kind of embedded in our experience online. The very fact that you get to use a lot of services for free on the internet is really powered by the fact that they are funded through advertising.

So you log on to Google. You use the Google search engine. The reason it’s free, of course, is advertising. And it’s not just that it’s free online, but that these companies are extremely dependent on this ecosystem of advertising as well. So we’re talking hundreds of billions of dollars worldwide flow in and out of the online programmatic advertising marketplace.

And for both Google and Facebook, to take two examples, we’re talking about companies that have over 80 percent of their revenue coming from these advertising sources. And so what’s interesting about it is that it is not only something that influences our present day experience of the web, but continues to shape why things are designed on the internet the way they are.

So one of the examples that I always give is thinking about the Like button on social media. Why do you even have a Like button at all? Well, one of the reasons is that it makes “engagement,” quote unquote, with content a lot more measurable and, therefore, a lot better for buying and selling and serving ads. And so almost from the smallest aspect of the web to the biggest economics of the web, it’s all shaped by this core ecosystem.

This gets to an interesting question I had reading the book, though, which is which comes first, the design of the internet or what advertising needs for the internet? So I think classically, the way we understand these platforms is that we have a weird model that emerged online, where you get attention by any means can.

And you can run at a loss as long as you’re getting attention for a very long time because venture capital firms and other kinds of financial investors will support you so long as they can tell — they can see measurably that you are commanding people’s attention. And then eventually, you have to figure out how to make what you’re doing legible for advertisers. You figure out search auctions at Google, and you figure out different kinds of personalization at Facebook, and whatever it might be.

But that it’s not so much that we built the internet for advertising as that we’ve been very clever about figuring out how to make advertising fit onto the internet we’ve built. But you see it more of the first way, that we build the internet to make it legible to the advertisers. So make that case for me.

Sure. Definitely. And yeah, I think that these are a bunch of debates, and it’s, of course, really hard to replay the tape of history — to say like, oh, if we just ran the internet again, do we end up with an internet that is largely based on advertising? And I think there are some people who would make the argument that that’s the case — that we’re just, like, history is unfolding and we’re discovering that it’s powered through advertising essentially.

I’m a little bit skeptical of that position because I think that there’s a deep path dependence in the sort of funding of online platform companies. That I think that the fact that you had Google, which was this company that was this, frankly, lab experiment that came out of Stanford, and was this huge, huge loss leader for such a long time. V.C.s kept pouring more money into it, and there was just lots and lots of users coming in, but no real way of making money.

And then at some point, they figure out the ad marketplace. And obviously, Google becomes one of the most valuable companies in the world. It’s almost, I think, that dream that almost kind of gives momentum to this business model in two respects.

I think one of them is that investors are willing to give these companies the benefit of the doubt. If you’re able to aggregate a lot of attention online, we just have this almost religious faith that there’s just some way that you’ve got to be able to turn this into money. You will become a Google. You will become a Facebook.

And what that allows is, I think, these companies to continue going on for much longer than they would otherwise. So I think one of them is that the anchor of Google and Facebook have given room for a lot more advertising companies to emerge.

I think the second one, too, is that there’s the flip side of that, which is basically that if you come to a V.C. and you say, I want to do a subscription business model, they’ll say, well, I don’t know — we don’t have a whole lot of examples of that really blowing up, so why don’t you just do advertising?

And I think that also has another effect, which is that you basically don’t have funding going to true experimentation with lots and lots and lots of different alternatives that might build the internet in a really different way. I think the only final thing I’ll note here is that what I use always as a marker for how unplanned it was that advertising ended up becoming the business model of the web was actually to look at Google.

And what’s fascinating there is that they were basically like, we’ll have advertising, but it’ll constitute 10 percent, 20 percent — maybe 15 percent of our revenue. Most of it’s going to come from licensing our search algorithm, which is obviously not the case. But I think it’s clear to me, at least from the historical record, that there are lots of different business models at play. I think the debate ends up being just, do you think the incentives inevitably push us towards that? But I feel it’s kind of a failure of imagination in some respects.

The collapse into advertising as so often the chosen business model is — it’s really widespread. So I think about basically what are the different industries that make up our informational commons — give us the information we use to talk to and debate with each other and know about the world. And I’ve worked in a bunch of them — newspapers, which are heavily advertising supported — not only, but heavily — magazines — heavily advertising supported.

If you think about television news, including cable news, that is heavily advertising supported. Radio is heavily advertising supported. Then you go online and you have search — advertising supported. You have social media — advertising supported. Really, the only major one that always comes to mind that is not built on advertising is books. But almost everything else we use to talk to each other about the world in which we live is built on people trying to sell you sugar water.

That’s right. I’m arguing a little bit against myself, but then I’ll give the counterpoint. I think one of my favorite historical examples of this is the creation of the visual telegraph, which is basically this communication network that was built in France.

And the way it worked was basically a bunch of these, effectively, windmills with shutters on top, and you’d use them to basically flag another tower that was a number of miles away to communicate across long distances. And they set up this network in France, and pretty soon afterwards, people were like, you know what this would actually be great for? It’d be great for advertising, [CHUCKLES] basically. And so it is interesting. And I agree with you — I do think that there is — the availability of advertising is always there because it’s always attractive to offer a product for free, and then have it subsidized by a third party. I guess I’m not trying to argue that option isn’t always there.

But one thing I would point out, though — and this is kind of my counterpoint — is maybe to think a little bit about what makes businesses viable or not viable. And for me, I think it’s wrong to be like, well, the market will always kind of collapse in on a particular business model because really, what we’re talking about is people and what are people willing to pay for or not willing to pay for. And I do think that with books, what we have is a great example. Like the example that you brought up is that it’s an example in which I think a market has been trained and that it is actually — there is a norm to say, yes, a book is a thing that I will buy. And even if it’s basically just a text file, I’d be willing to pay X amount of money for it. And I think that just ends up being a question of what people want, and what the market will sustain, and our norms around what’s worthwhile or valuable, online or not.

And so I think if you buy that thesis, I do think that there’s this kind of interesting, maybe self-reinforcing, cycle, which is that we’ve lived with all of these advertising models that make content available for free. It makes it really difficult for you to convince someone to pay for content because the psychological cost of content is just zero.

And so you almost have to shift cultural norms to get these business models to work. I just happen to believe that if there was, say, V.C. funding there — venture funding — to really push that ahead, we might very well be able to shift those norms. But I think it’s reasonable minds can differ on how stuck we are with that particular way of thinking.

Well, let’s put a pin in the question of V.C. funding, because I do think there’s at least some evidence that this is changing. But now let me join you in arguing against myself and speaking to this question, which is I just gave this quick argument for continuity. But something you argue in the book, and that I believe is true, is that digital advertising is discontinuous with the advertising before it.

That the fact that they have the same name actually obscures how different what is happening when a magazine supports itself in print on ads, and that same magazine moves online and supports itself on digital ads. So talk about the change — about online advertising as a different kind of phenomena.

Sure. Definitely. I guess I can give you the dream and the nightmare — maybe is the best way of thinking about this. So it’s true. It looks quite different from what you’d usually imagine advertising to be.

So you think about advertising, you think about “Mad Men.” We have people smoking in an office, and they’re coming up with creative to try to sell Pepsi. And that was the way advertising worked for the history of print magazines and print newspapers and older forms of media.

What we see with programmatic advertising is something which is a lot more automated, a lot more data-driven, a lot more scaled than that. And the way to think about it is Google, Facebook — these were pioneers in the space. And what they needed to do is not just sell enough advertising to fill a magazine. They needed to sell enough advertising to sell ads against every possible search query that someone might have online.

And so they invented the system — programmatic advertising — which, in my book — and the book kind of refers to it — looks a lot more like the capital markets, like high frequency trading. And to give just a quick sketch of it the way it looks is, when you go online, you go to a website that’s going to serve you an ad.

Essentially, what happens is that there’s a signal that basically is put out to a marketplace saying, hey, we’ve got Tim — male, 25 to 35 — on the East Coast who’s looking at this website. Who wants to advertise to him? And essentially, there’s a number of algorithms that operate on behalf of advertisers that compete to basically bid to deliver the ad to me. And depending on which bidder wins, they upload that to my website and into my eyeballs. And this happens billions and billions of times every single day.

And so the dream of this model was that you could do advertising at much larger scale, but it also was a dream that this kind of online advertising could transcend the limitations of older advertising. So it’s very data-driven. You know exactly who’s looking, who can click on it. It’s scalable. You can get a message out to lots and lots and lots of different people.

So there’s a dream that you’d finally be able to create advertising as a science — that we could measure, quantify, and deliver ads in a way that would truly finally get rid of the touchy-feely aspects of the industry, and that’s the dream. And I ultimately argue that that dream didn’t end up being true, but it was something that motivated this industry. And I think, frankly, it’s been responsible for a lot of ad dollars moving out of old media to the web, because of this promise of measurability and scalability and data.

And one of the points you’re making is that it isn’t — what is different about online advertising is not just the way in which it is technologically differently structured than, say, a newspaper ad, but that the framing is financialized in a way that advertising wasn’t before that. So can you talk a bit about that — the financialization — and what is being financialized, what is being commodified and chopped up into little pieces to be sold on marketplaces exactly?

Yeah, absolutely. So I think the good frame to think about this is in the context of what it means to set up an advertising marketplace of this scale. So you imagine you’re running your local paper in the 1950s. You really know the audience that’s going to be likely reading your paper, and the advertiser is putting ads to reach that handful of people.

And in some ways, attention is very heterogeneous because the media markets are really small and, in some ways, the publisher knows the reader maybe on even a person-to-person level. Like, I know Jim, he comes into the office and he complains about this all the time. And programmatic advertising, in contrast, is very, very big, very, very arms length, very, very anonymous.

And effectively, what has been done — what I think of as a form of financialization — is to commodify, and spread out, and render all sorts of attention very much the same. So you go and you say, well, I have a couple of demographics I’m trying to reach, but I don’t really care where they come from or where they are — just serve these ads out to the internet.

And so there’s an attempt to turn it into a tradable commodity — that oh, well, this blog versus that blog, they may differ on a couple characteristics, but so long as I can reach a person of a certain characteristic, I don’t really care what channel the advertising is flowing through or how it gets done.

And this actually accounts for what has led to actually some interesting litigation against Google — these systems that act on behalf, algorithmically, of the ad buyer. And so I think that’s kind of a little bit of what I think about as like financialization, is that there’s been an effort to turn this into something that’s a lot more tradable and that you can really, really do at scale. It’s like the formation of a market at scale is how I think about the financialization of this ecosystem.

So attention for me is really the key word. It’s why we’re having this whole conversation. I think of attention as the most important, often worst theorized, least explored thing in life. It’s everything, and then it’s very hard to get your hands around what exactly it is.

But you have a lovely line where you write, “The amorphous, shapeless concept of attention has been transformed into discrete comparable pieces that can be captured, priced and sold. Buyers and sellers can quickly evaluate opportunities and transact in attention at massive scale without individually evaluating each opportunity.”

So there’s this point here that our attention, like much in the way that other kinds of bonds and assets are, is somehow being regularized, packaged together and sold. How? When you say that buyers and sellers can transact in attention, where do they do that? What do these marketplaces look like?

So there’s two elements to this. So I think one of them is, what are you buying and selling exactly? And it’s actually really fascinating because there’s a bunch of standards that have been written in the ad tech industry to say, so what is a page view? What is time on site anyways? What is attention is, I think, the question they’re really, really ultimately trying to answer. It’s written in quite terse speech, frequently. But I think it’s, in some ways, an absurd exercise, which is, how do we turn this thing — attention — which is just me looking at something into something that can be bought or sold.

And typically, what that ends up looking like is measuring it based on your activity or your kind of engagement that you have with, say, a website or an app. One common definition is, you have held the ad in the window of your browser for at least a second before moving on. And that’s how we define the proxy for attention. That’s really what a buyer is buying when they buy your attention.

And then the second bit of your question was, so then how is this bought and sold? And this goes to a little bit of the process I was describing a little bit earlier, which is that you have a marketplace which consists of these kind of two types of infrastructure. There’s essentially a set of bits of software that act on behalf of the sort of publisher.

So this would be your New York Times, your Facebook or your Google or places that are aggregating attention. And then you have a set of pieces of software that basically represent the ad buyers here — so your Coke, and Pepsi, and your agencies. And these kind of two bits of software plug into an ad exchange, where essentially these kind of bids and offers are made.

So in the description I gave a little bit earlier, it would be, hey, we’ve got male, 25 to 35, clicking on this website. Who wants to buy the ability to deliver an ad to him? And the exchange basically facilitates the auction by which these bits of software acting on behalf of agencies and Pepsi and Coke go and compete to basically deliver those ads.

And so this is, by and large, an instantaneous process. It happens between the time at which you click to go somewhere and the time in which the page actually loads on the other side. And that’s kind of what it looks like. And so this is all done really in an automated way. We’re talking about global servers talking to one another rather than anyone having to pick up the phone or call anyone to facilitate the delivery of ads.

I mean, that’s truly amazing. Put aside however you might feel about advertising or attention or anything else —

Sure. Yeah, of course.

— that this is all happening. There’s an auction for your attention, which serves you up something in a personalized way in the time it takes you to load a website on a broadband connection — is nuts.

It’s wild. Yeah, I agree. I mean, I think it is, maybe at the risk of overstating it, one of the premier engineering achievements of the 20th century. In some ways, the reason why Google succeeded. There were other people playing around with these types of marketplaces when they were coming up. They succeeded because they were able to do very high performance infrastructure running at really, really high speeds. That was the secret to their success, and getting that to work is, again, an enormous challenge. I think it’s just wild that it works like that.

Wait, can I hold you on that for a minute, because this is, I think, a pretty unknown story for most people about Google. I think you say, what did Google do? People say, oh, my god, they figured out search.

[CHUCKLING]

And that’s half of the story of what Google did. But they also figured out the business model of online advertising to a very large degree. So what did Google do to make their business work? What is the great Google business innovation?

That’s right. So the complexity — put yourself in the shoes of these two youngsters from Stanford with this company with a search engine. You have lots and lots of people using this thing called search to find things on this wild new thing called the internet.

And this is all well and good. You’re very excited that you have a bunch of users, but you’re just burning cash on a month-to-month basis just paying for the servers to keep this thing running. And the engineering challenge of delivering advertising here is to basically say, someone could come to our website and search for literally anything — how do we make sure that we can deliver an ad that’s relevant to that search query at the time at which they do that search query?

And so the engineering that needs to take place in order for this to happen is that you basically set up a marketplace on a keyword-by-keyword basis — a product called AdWords. And this was the system that they set up was essentially an auction infrastructure. So what they would say is, well, if you’re a plumber, you can advertise against the plumber keyword, and other plumbers who also want to advertise against that keyword can bid against you, and an auction will be sort of run to deliver that ad.

And part of their innovation there was not just setting up that marketplace, but famously, one of their kind of key things that they sort of figured out was that they also wanted to kind of rate the person who won the ad based on the quality of the ad. And that was built on a number of different features about the kind of ad you’d submit to the system.

And so what they were trying to do in this early phase was balance the scalability of advertising, but also to ensure that advertising didn’t become sort of terrible — that it wasn’t low quality ads that you didn’t want to see. And I think in nailing that and getting a marketplace to really get up and running, that, in some ways, is their true sort of business innovation and the thing that created this waterfall of money that’s kind of supported the company to this day.

So now, I want to go back to this question of what kind of internet the ad business has created. Because whether or not everything begins with ads, it does, at least in the case of almost every major online platform with the exception of Wikipedia and then a couple of the subscription-based streamers like Netflix — it all becomes ads. So once you have an ad business — once you’re there, what are you incentivized to do?

This has been kind of a very long legacy. I think it’s a metanarrative of the last few decades of the web. Famously, Larry and Sergey have this paper where they first describe the architecture of their search algorithm — so PageRank.

And one of my favorite bits is they have an appendix where they go out of their way to say, let’s consider whether or not a search engine should ever adopt advertising. And they conclude, much to, I think, the chagrin of their future lawyers that would be representing them before various antitrust proceedings and otherwise, that essentially, no search engine in their right mind should adopt advertising because once you have a third party that is paying to access eyeballs in your search engine, you have lots and lots of incentives to shape your experience to cater to the people who are really paying your bills.

I think this is part of the concern about online advertising is to say, well, in that case, you really want to be able to provide a lot of data about your users to these advertisers. That’s one thing that you might want to do. The other thing is that you might want to make sure that the ads are kind of indistinguishable from legitimate search results that the search engine is showing up. And so one thing that people have pointed out is it’s harder and harder to tell what’s an ad and what’s not an ad. And those types of incentives have always been at the heart of the web — that no, no, we can really balance these incentives. And I think some people — and I would count myself in this category — would say, well, no, even the strongest company becomes victim to these incentives over time, and that they have these profit incentives to structure the web in a way that are ultimately adverse to their users.

I think there is a view out there that I have conflicted feelings about, which is that the obsession with engagement is a byproduct of the advertising-based business model. That if you’re not based on advertising — if you’re subscription or something else — you don’t really care how long people are on your site, just that they’re paying for it.

On the other hand, you’ll hear the head of Netflix say, our big competitor is sleep, and Netflix has autoplay, and they are obsessed with how long you use Netflix for a month.

So do you buy the narrative that the attention hungry outrage-oriented — or often outrage-oriented — engagement-based nature of most major online platforms now is a factor of the ad buying process?

I’d say no, and I think this is one of the reasons why the book pissed off ad optimists as well as ad critics. I don’t really believe that view. And I think the view, at least as I’d frame it, is basically that the internet is kind of like to blame for a lot of the content ills that we see on the internet.

But I don’t think it’s so certain because I think the incentives for a creator — I’m going to use that very broadly — everything from your user to your professional YouTube influencer — I think is interested in generating an audience online, whether or not the person pays for you by subscription or you release that content for free or if you get ad dollars.

I think that in all those cases, there’s a desire to grab attention, and I think that introduces bad incentives no matter what the business model is. Now, I do think that there’s two subtleties that I’d point out there. I think one of them is that I think there is content that might not otherwise exist unless there was the additional kind of impetus of advertising.

And I’m thinking here of the really interesting ecosystem around fake news websites. It’s like, well, maybe we can really gin up very profitable websites very quickly by generating a certain type of content. And it’s like, well, would they be willing to go to those extremes in natural terms if their only incentive was to aggregate an audience? That’s one view, and I think there’s kind of some cross-bleed there. I think there’s a lot of gray area.

I think the second thing which I would maybe pivot is to think a little bit more about the influence of things like content recommendation on shaping the types of content that people produce.

I think in the default, in all cases, people want an audience. But I think when there’s the additional incentive to say, oh, well, if you publish this type of content, you’ll get distributed to everybody on our platform, I think in some ways, that’s almost even a stronger motivator than simply like ads being the driver. And so I think, obviously, it’s a complicated question. But I think it’s very wrong to believe that, at the core of this, ads are to blame for the outrage culture, if you will, of online discourse.

I want to pick up on one of those subtleties. Something that ads permit is financing mechanisms that are built very much on scale. So you can have — the example of fake news websites is really good.

If a lot of people are coming to your website, people don’t really know it or like it — it just, for whatever reason, is getting a lot of drive-by traffic — you can make some money off of that under ads. Subscription, you actually need to connect people to the site.

Now, you can do that in good or bad ways. A lot of things that do very well in subscription are very bad things because people are subscribing because they believe or want to support something very bad. So I’m not an idealist about subscription either.

But if you take, say, what’s happened on Substack and what they’ve proven out, one thing that subscription really allows is you can have 5,000, 10,000, 15,000 people, and if they’re willing to pay $20, $30, $50, $80 a year for what you’re doing, that’s actually quite a bit of money.

Whereas 10,000 or 20,000 people for ads is nothing at all unless you have intense industry-specific advertising for a decision-maker class. And so there is something about what is permitted in the two. I do think that the internet moved way more towards scale-based business models because of advertising when a lot of what is best is often a little bit more niche, but that tends to thrive better in a subscription model.

I never really thought about it like that, but I think it’s a great way of taking it, which is I think we can think a little bit about the fact that the ad market was designed to be as frictionless as possible. That the old generation of advertising was bad because, yeah, you had to call up the Mad Men, and get your ad produced, and it took a really long time, and to sign an engagement letter with them, and it was just complicated.

And the effort to commodify, to financialize attention, has been to make it as frictionless as possible. You have a website on the internet you can just plug into this ecosystem and start making money. You have ad dollars you want to spend — just put your coin in the slot and you go.

And yeah, I think that’s right. I think we’ve landed on metrics and an infrastructure which, I think, might reward kind of content on a page by page basis — maybe is a way to think about it. That there’s actually something about the business model that suggests the ordering or the structuring of content and information online.

Now, like you say, I hesitate to be like, oh, and that means that the content is more shallow on the internet, or oh, isn’t it the case that it’s just intellectually less interesting? I think that brings a lot of baggage to it.

But I think that if you’re just purely talking about how we experience content online, do we think that the experience of it as just a single page that you hit and then you go away — the viability of that I think is a lot more possible because of the ad ecosystem.

And so, yeah, it is interesting to think about almost away from a values-laden judgment way of thinking about it. Almost like, what’s the collection of texts that you encounter online and how is it connected to one another is very much shaped by ads. I think it’s — yeah, a super interesting way of thinking about it.

[MUSIC PLAYING]

Tell me about the central metaphor of the book, the “Subprime Attention Crisis.” What is that?

So I mentioned a little bit earlier that I can tell you about the dream and the nightmare. The dream has been this sort of like laissez-faire vision of online ads. That these ad exchanges would connect ad buyers and ad sellers, and we would have a kind of transparent marketplace where finally, finally, finally, we’d be able to overcome what’s known as Wanamaker’s law.

And so Wanamaker’s law is 50 percent of the money that I spend on advertising is wasted — I just don’t know which. And the idea is, well, with data, with targeting, with all this machinery we’ve built, finally, now, we can get this thing to work. And this is something that you still hear from ad boosters, which are people in the industry that say, no, we’ve cracked this — we can put in a dollar here, get $1.25 out and, by and large, the system works — like, we’ve really cracked it.

And I think one of the things I point out in the book is that this marketplace may be sort of a lot less functional than it might initially appear because it relies on a number of assumptions that don’t turn out to be ultimately true. And so what I argue is that we’ve got — implicit in the title of the book subprime attention crisis is a bubble in the making.

That we have a market that people believe is extremely, extremely valuable and only getting more valuable, whereas the reality is actually like things are getting more and more dysfunctional with time, and that the central promise of this market, which is I can reach a consumer and get them to buy my product, is not really ultimately playing out. And I can sketch out just a few arguments, and I’m sure we want to go into them.

Yeah, make the case for me that this is not what it appears to be, this is not worth what people think it’s worth.

So maybe a couple of quick arguments I’ll make on this front. The way I think about it is that when people think about online advertising, they think about hyper-targeted mind control laser that you fire from space to this consumer. Like, I can get to Tim to change his behavior.

And as I said before, that relies on a couple of different assumptions about things working really well in the ad ecosystem. So the first one is that you can reach me or you can reach a real human at all. That is to say, when I deliver an ad on this marketplace that it actually goes to someone who is a human.

And there’s these really interesting studies that have been done — so Forrester, the research group, did this paper that suggested that 56 percent of display advertising — so the kind of ads that you see on places like Facebook — may actually never see a real person, that it’s lost to fraud.

And the way the gimmick works is, basically, that these are ads that are being delivered to a fraudster that actually has a bot that’s loading a website or has what’s known as a click farm where you pay a bunch of people to click on ads. And the scale of that’s huge. So a really kind of ambitious estimate is $1 out of every $3 is lost to fraud online.

Well, maybe you take a look at that and you say, OK, Tim, lots of markets have fraud. Let’s assume that you actually reach a real person. And then I’d say, actually, that’s also another assumption — the assumption that you actually reach a person and you have relevant data about that person.

So I’ve been using the male, 25 to 35, through the episode. There’s some studies that suggest that essentially 41 percent of ad campaigns don’t actually reach the person because the ad data is all inaccurate. So you think you’re reaching male, 25 to 35. Turns out, it’s female 75 to 95 that’s living somewhere completely different from where you thought. And then I think there’s another kind of layer of arguments which is to say, well, Tim we’re going to reach a real person, we’re going to have accurate data about them — does this actually work? And again, a number of really funny shocking stats. So Google itself did a study a number of years ago that suggested that even if the ad is delivered, 56 percent of ads are just never seen. They’re below the fold, they’re in some weird corner of the site. You’re just not looking at it because you’re reading the article. And so I think the way this goes is, ultimately, you’re grinding, grinding, grinding, grinding down. And you’re like, are these ads actually even reaching people — is this as effective as we think it is?

And I think in the very least, even though we can come up with examples of ads working, the question is whether or not that’s the median case. Is that, by and large, what’s happening in the ad markets, or is it largely money that’s being like wasted in this weird system of fraud and lack of attention and inaccurate data and all these sorts of things. And so the vision of this mind control laser from space is a kind of vision of perfection that doesn’t actually exist once you actually get into the guts.

And so what if it doesn’t work? Why should I care if this ad bubble — if it is a bubble — pops?

Yeah, I agree. I think the reason we should care is not so much that we’re worried about Mark Zuckerberg having a couple less billion dollars. [CHUCKLES] I think the reason we should care is that this ecosystem funds almost everything that we know about — the web.

It subsidizes the fact that we can access a lot of services for free online. It funds journalism and media. If you think that’s important, that’s a huge reason to care.

One of the things that I think is another weird angle on it, just to throw it out there, is a lot of these companies that have become so rich off of the ad ecosystem are simultaneously subsidizing lots and lots of other things outside their industry. So if you think about the entire wave of advancements in A.I., those are being done by companies that can only subsidize those at a huge, huge loss because they make money through advertising. So there’s also this relationship between the health of this ecosystem and the advancement in science and technology.

So I think for all those reasons, we should care more about this because it touches so many parts of our lives. And I think if it is as shaky and as messed up as a lot of evidence suggests that it is, we might want to think about the stability of that market and the bigger question, which is what do we think we want to replace all this?

One thing that was interesting for me and challenging for me about your book is that it takes this reverse view of the way I normally think about this market, which is that what I care about is the quality of the attention, not what is happening to the advertising. And you’re making this point that, well, if it turns out the advertising isn’t working, then a lot of the economic underpinning of the web all of a sudden blows up.

But let me push on this a little bit, because I’ve covered financial crises and, in particular, covered the subprime mortgage crisis, and so I spent a lot of time thinking about this metaphor. And one thing that occurred to me about it — there’s an old Keynesian insight about financial crises, which is that what is weird about them and the damage they inflict is that nothing has happened to the factories, nothing has gone wrong with the people, nothing has been lost in our knowledge of how to do work.

That the economy has not become any less productive in its potential. There is just this weird financial dynamic that makes it less productive, in fact, because people lose their money and stop investing and so on.

And one thing that struck me is that that is a disanalogy here, because I think the thing that I worry about most — not with whether the ads work or not, because you could imagine this maybe even happening worse in a situation where all the ads work perfectly — but is that if you think of attention as a collective good, a public good, a public resource — which I do think of it as — then if the ad industry is getting either too good or too pervasive or too attention-grabby, or whatever it might be, or what it’s supporting is not good information, then it’s degrading the collective quality of the attention.

Unlike in the subprime mortgage crisis where all the houses were basically fine — you could inhabit them — I worry that our collective attention is being degraded by the way the web has evolved. And every individual player has an individual incentive to grab more and more attention. It’s highly competitive. They’re all competing with each other for our attention.

And so you get this tragedy of the commons dynamic, where everybody is basically exhausting our attention, making us irritable and angry at each other and unable to focus. And it’s not really in anybody’s interest to figure out how to collectively manage the attentional commons better. I’m curious how you respond to that.

Yeah, I think that’s right. And luckily, I don’t think we’re too far apart, Ezra, in some ways. The stat that really sticks out to me when you say that is this comparison I do in the book between the first banner advertisement that ended up on the web.

It was this AT&T campaign that was running on, I think, WIRED.com’s website. And the clickthrough rates there in 1994 were 44 percent, which is wild. It’s almost like a science fiction universe, because I think if you’re familiar with the stats that we have today from banner advertising, it’s more like 0.1 percent. You’re thrilled if you get anywhere near 1 percent clickthrough.

Anyways, I come to mention that just because I think it’s almost a proxy for what you’re talking about, which is attention as a common good and whether or not the incentives of this market are essentially driving disattention. That basically, your content feeds, your information feeds are just filling with so much garbage that even the good stuff can’t get through.

And the end effect is that people just pay attention less, or they pay attention in a much more shallow way than they used to. And that that ultimately is a kind of problem — I guess, implicitly what you’re saying is it’s a problem from a societal standpoint, almost from a democracy standpoint.

I think it’s also an interesting way of using the subprime metaphor again. Because what is the core asset that we’re buying? We’re buying these clickthroughs. We’re buying these engagements. We’re buying these moments that you look at online. But the core of it, which is how much are people actually paying attention to the ad, is declining, declining, declining. And I think it’s a little bit like the subprime mortgage crisis in the sense that you have these collateralized debt obligations, where people are like, we’re going to just keep buying the same mortgages as we’ve always done. But underlying that, the asset is just getting worse and worse and worse and worse and worse until the whole thing kind of busts.

And I think that’s similar here, which is the market’s still behaving as if it’s buying attention circa 1994, but the collective incentives of the market — to take a Keynesian approach — has resulted in a situation where the underlying societal dynamics of this, if you will, are not really supporting the kind of attention that was bought and sold back in the day. And I think that really is quite parallel to the crisis.

The weird effect of that, if you buy what I’m saying, though, is that when we looked at ads in the early ‘90s, they really did work better, which is kind of the funny thing. There may have been a period where, actually, ads did work better. But essentially, the core underlying asset, which is attention, has become so degraded with time that this channel is becoming useless.

And maybe this is a way of thinking about advertising, which is that the effect of advertising is more a function of novelty than it is a function of anything else. And so that the internet is just going through the same thing that all other media channels have had, which is that it fills with terrible advertising, and then people are attracted to the high value attention channel, and they move on. And maybe that’s happening with the internet as a whole.

I think this is a place where, in a weird way, advertising and basically everybody else on the internet, the incentives are aligned, but nobody seems to know what to do with it. Which is I’m not very convinced by the idea that advertisers are going to wake up one day and decide their ads aren’t working anymore, because I’m more of the belief, as you mentioned that law earlier, that they know a lot of them don’t work, but they also don’t know which ones do work, and everybody is trying to be the place where it really works.

And so that can just roll on for a very long time. But the danger to ads is ad blockers. We see a pretty significant growing proportion of the online user base using ad blocking technology. And that can’t get away from everything, but it actually can block quite a bit.

You see Apple dealt a pretty significant blow to Facebook — now Meta’s business — by just creating this little ask app not to track — just knocking app tracking, which is part of the advertising machinery from iOS — or at least making it easy to block app tracking within iOS.

And the more this happens, the more you’re seeing these symptoms of, on the one hand, really significant user discontent. People don’t like the way the modern internet feels, they don’t like the ads on it. To some degree, they just don’t even like what they’re doing on it.

I also think it’s a much bigger vulnerability for Twitter than people give it credit for, that so many of its power users have this hellscape attitude towards it. It’s like, oh, another day on the hell site. Like, that’s not a great way for your loyal user base to feel about you.

I think it’s fine to say that’s a bad user experience. [CHUCKLING]

Yeah, so something is going wrong there that — I mean, even in Elon Musk’s letter to advertisers, he talked about Twitter’s potential as a hellscape. There’s something wrong there. That if there becomes a technological or competitive answer to that — something else that does something similar for you but does it make you feel like that — you can see mass migration.

And so one of the things that feels to me much more rickety on the modern internet than people realize is people don’t like the experience of it. And they may not feel like they have good alternatives, but the more they do, be that browsers that block ads, or other kinds of services, or whatever, the more they might end up getting into some tipping point dynamic, or once the network effects of things begin to fade, everybody begins to leave en masse.

So a lot of things to respond to there. I think that’s right — that you’re almost kind of articulating an alternative path to how this whole ad thing might come apart. And it isn’t the kind of explosion that I’m putting out in the book, but more the shift in what people want when they go online.

And I think that’s right. I think ad blocking, in some ways, is just the most kind of potent or most striking proxy for the fact that people just don’t want to pay attention to ads. And so the two ways of avoiding ads are you just don’t pay attention to them or you go out of your way to install software that actually blocks those ads.

And the numbers that we’ve seen on ad blocking is that it is showing up. And so there’s maybe one small stat here that I’ll maybe drop, which is pretty interesting. There’s a study from Comscore a number of years back that concluded that 8 percent of the population is responsible for 85 percent of the clicks online.

The clicks on ads or all clicks?

I believe it’s based on ads specifically. And so one of the things that’s interesting about that is an ad ecosystem, where you could sort of imagine this kind of weird ecosystem that is maybe parallel to what you’re talking about, which is basically that people get tired, they drop off, they go off these platforms.

We have more and more the ad ecosystem resting on a small group of people who are just ad maniacs. I think that’s a potentially interesting outcome from what you’re talking about. And I do think that when people say, oh — I was catching up with a friend recently who was like, it feels like the era of Twitter is over.

We’re not going to see a Twitter competitor just because people just don’t want that experience anymore. It is interesting to consider. It’s just like, is the future of the internet really social? Maybe we want to escape that just because that channel has become so polluted and has so much baggage now that it’s almost unsustainable because it has — it’s toxic to launch a social network given the norms and expectations that people bring to that kind of product.

Twitter brings up an interesting shift here though, which is that another thing that could happen is a shift to more mixed business models. And in particular, I think we’re at least seeing both in terms of actual platform and publisher behavior and in investment at least a lot more experimentation with subscription. So there’s, of course, the big streaming platforms which are mainly subscription-based.

You think of Netflix, or Disney+, or H.B.O. Max — those are not advertising-based businesses. Maybe they’ll have some now or later, but that’s not really what they’re doing. You saw YouTube roll out a subscription product. I pay for that. I find it very much worth it. I don’t have ads on YouTube anymore and it’s better.

Twitter blew under Elon Musk. They fired a bunch of people, they’ve made themselves much more toxic to advertisers, and now they’re trying to get people to pay a subscription fee. I don’t know if that will work out, but it is certainly, what they are attempting. I’ve mentioned Substack, which isn’t a huge business exactly, but I think it’s had a very outsized effect on how people think of how you might fund media online. And I think actually much more so the Vanguard of that is The New York Times itself, which has built a huge subscription business online — much bigger than, I think, at this point anybody else’s and proved at least it is possible to do. But of course, the F.T. and The Wall Street Journal and The Washington Post and others have them as well.

And so it seems to me there’s a shift right now, both in fact and in zeitgeist, towards subscription — maybe not always to the exclusion of ads. A lot of these are hybrid models. But trying to hedge your bet a little bit. How do you think about that?

Yeah, I’m really excited by these types of models. And this maybe takes us all the way back to the early part of the episode — is for me, a lot of this comes down to when someone encounters content — whatever that word means on the internet — what do they expect to pay for it? What is the expected value I guess and expected price of that?

And a lot of that, I think, is it’s a common good in some ways, which is OK, we could live in a society where everybody says, oh, yeah, just by default, I just think that this should cost money; I will pay for it. And all of these attempts to scale subscription, normalize subscription — if you will, shift the Overton window around subscription — are, to me, really, really important. Because they, in some ways, create the normative scaffolding that other businesses can build on.

Like, well, great — we’ve got actually like an audience of people that have proven that they want to pay for things in this way online. Now that expectation is set, maybe we can build on that. It’s not so weird for you to pull out your wallet and do that online.

So I’m quite interested by it. And I think it’s exciting because if we can win that war for hearts and minds, I guess — not to be too dramatic about it — possibilities start to open up on the web, because I think, ultimately, this ends up being people and how they want to spend their attention and how they price their attention.

Now, I think the only note that I’ll strike maybe on the other side of that is that there are some people that say, dot, dot, dot, and then we’ll live in a media utopia, and that’s a little bit of a straw man. But I think I point it out to say that there are trades — there are very real trades in imagining an internet that’s all subscription.

One of them is just accessibility. One thing that a lot of people have interestingly pointed out is a world in which high quality media sources are moving behind a paywall is one in which low quality media sources remain out and widely available. And I think that kind of dynamic is a really interesting one.

And I think part of it is just, do we want to live in a walled garden web, where it’s like, well, you can access this better world, but only if you can pay? I think overall, I tend to be still in favor of subscription and normalizing subscription because I think it forces that hard conversation, which is what kinds of these services are so valuable as to maybe being something we want to subsidize or think about as a form of infrastructure that you have access to as a right?

We’ve been able to avoid those discussions because the price has been free, but the creation of subscription also, I think, forces that thorny conversation in a way that I think is ultimately really helpful.

[MUSIC PLAYING]

So you worked at Harvard M.I.T.‘s Ethics and Governance of A.I. Initiative. You worked as Google’s global public policy lead on A.I. That was a couple of years ago. A.I. is exploding much more into wide use now.

[CHUCKLES] I hear that things are happening.

Yeah, there’s been a lot. I don’t know if you’ve noticed. [CHUCKLES] And one thing that I would say concerns me about it is that a lot of the major A.I. shops either are part of ad-based businesses. So Google — or Alphabet, I guess — has a lot of AI work going on. Meta has a lot of A.I. work going on. Or are attached to them in one way or another. So OpenAI has a lot of investment from Microsoft, which obviously sells a lot of software. But if you think about what they want to achieve one day with Bing — I mean, there’s a big ad market there too.

And what the kind of A.I. we’re creating is good at is just creating endless quantities of personalized manipulated content and figuring out what people want from that content. One thing that worries me is actually a world in which A.I. is used to make online advertising much, much, much more effective, and personalized, and omnipresent, and creepy, but maybe I’m looking at this the wrong way. So I’m curious — as somebody who’s thought a lot about ads and thought a lot about A.I. — how you see A.I. changing what is the fundamental business model of the internet now that the major A.I. systems are being created by the people who control those business models?

Yeah, big question, and I think there’s a couple of ways at it. I think we can think a little bit about the influence of the current wave of A.I. innovation, which is a specific technology called Large Language Models, or L.L.M.s, as having an impact on the product landscape on one hand and then on the business landscape — the ad landscape on the other. And so let me first begin, I think, by thinking about the product landscape.

I think what’s so interesting — and one of the reasons that has been reported — there’s a code red at Google. That OpenAI’s ChatGPT has gotten so many users, we’re worried because it might ultimately replace search. And we can almost think about the epistemology of search, if you will.

The kind of experience we’ve had with a Google-centered web over the last few decades is one in which you punch a search query into a box and what comes out of it is a set of options. It says, we’ve ranked these, and you choose. And hopefully, we’ve delivered something that’s relevant to you and you click on it and you go about your day.

What’s intriguing — if you think about L.L.M.s as a thing that might eventually replace search, because it’s easier, people want to just deal with one conversation and not have to translate through or trawl through a bunch of different search results — is that you almost change the epistemology of what it means to explore the web. That rather than putting your search query into a box and getting 10 links, what you now have is a single voice that synthesizes the information and then offers an opinion. And there’s ways of crafting that. You could make your agent more or less uncertain, more or less ecumenical, more or less willing to give you different types of points of view or different links to different types of information. But it is a lot more kind of constrained because it happens in the context of a voice versus what you could think of as search being much more of like a computer in the old vein, like an information terminal that you typed things into to get information.

And I do think that that is potentially worrisome, which is we’re moving to an internet that’s based on monolithic narrative synthesized by a machine rather than an internet that’s based on searching through lots of different options on your own. And one has less friction. One actually might give you better results. It’s possible that the agent gives you the result that you’re looking for even better than Google.

But I think part of the question is, do people start to think about the internet quite differently? Do they not necessarily really want an internet of websites, what they want is an internet that’s translated through an assistant? And I think that leads to a very, very different experience of the web if we go down that route.

It’s funny because I remember a previous iteration of this fear for Google, which I think was less true, obviously. But there’s a period when Quora, the question and answer site, was a very big darling of Silicon Valley. And I don’t remember where I read the major article on this. I vaguely think it was Wired, but that might be wrong.

But there was a belief among some people that Quora was going to replace Google. Like, this would be the future of search. You would ask a question, or you would search questions that had been asked, and you get this great answer from the community, et cetera, et cetera, and that never came to pass.

And Quora kind of became a search engine mill with decreasing quality over time. But it does seem to me that the vision here is something like what that vision was. That instead of asking Google and Google gives you an array of websites you might want to look at — I mean, what you can do right now with ChatGPT is ask it a question and get some answer that, to be honest, in most cases, when I ask it a direct question, I get a better answer than I do directly from Google now, which I think has become a much degraded product.

But nevertheless, I wouldn’t exactly trust it because it constantly hallucinates information at me and it doesn’t know anything that happened after 2021. So it’s a little bit of a weird — I mean, it’s not really search. It’s a — I don’t know what to describe it as, but it strikes me as odd.

It’s reality fan fiction at the moment. [CHUCKLES]

But that’s why, it seems to me, to make more sense for advertising than for search in a way. I’m sure people will try to integrate it in search, and I suspect Google is going to make Google somewhat worse by integrating it into search very quickly out of panic. But I don’t know — advertising doesn’t have to have that much relationship with the truth, and A.I. doesn’t have to have that.

And a lot of the programmatic advertising you’re talking about is terrible because they don’t have budgets to get good ads built for them. So you could really imagine them working much more with A.I., and A.I. is much better taking in feedback, and then altering what it’s doing based on metrics coming in from the customer.

It just feels to me like A.I. — these large language models are almost perfect to be hooked into the online advertising schema in a way that they’re not perfect for quite anything else where you really need to trust what’s happening because you’re not just trying to get manipulation metrics done.

Yeah, I think that’s right. I think what I’d say is whether or not you believe that we finally have crossed the Wanamaker threshold — when we think about it as like an event horizon — where the ongoing dream of the internet is, I can deliver a message that is truly persuasive, and I can do that automatically.

And I think you have to believe that there’s something that’s been so dramatic that’s been unlocked by L.L.M.s that the ads are going to be that much more effective, and that, I’m not so sure on. Maybe — I think one thing that I will — I can see — we’re going to create the ability to create very high quality, high fidelity ads and distribute that to everybody.

There’s a project I’m working on with a friend of mine, Noah, where the plan is to basically do a Turing test for ads. We basically have a bunch of people generate human based ads, have a machine generate a bunch of synthetic ads, and the idea will be to get a bunch of ad execs to say, can you actually tell the difference between the two?

And I think the purpose of the exercise is demonstrate that — I think the quality is getting to the point where, do you need these creative agencies, do you need the whole kind of, again, infrastructure of marketing to produce ads that have that vibe that we spent a lot of money on it and it’s brand advertising? And I think that is something that will definitely change in the next few years.

But as always, as I’ve done multiple times in this conversation, I think, again, we still have to decouple that from the question of persuasion. We may be flooded with these kinds of ads, but I think, again, I’m not so certain that OpenAI has cracked here is finally the mystical persuasion engine that the ad industry has been looking for all these years.

Well, to take the other side of that and to try to beef up OpenAI’s valuation just one more time —

[CHUCKLING] They need it.

— I think the theory here would be that it’s not that what they’ve cracked is that their system can write a better ad, though maybe it can in many cases. It’s that what A.I. can do is enter into a tighter constant feedback loop because it’s basically costless to create a new ad based on information you’re pulling in from the person. So I’m going around the internet, I’m getting tracked.

And as you say, a lot of what I get now is unintentionally hilarious. Like, I go, I buy a bicycle, and then for two months, everybody is like, you want to buy a bicycle? How do you feel about bicycles?

But in this case, if you hooked the A.I.s into that tracking data, and you can see what I was doing and not doing, and they can generate possible ads — new ones — for no cost at all. And they’re quite good at it. I mean, what they are able to do now, I think, is quite amazing, and it’ll be better in a year, and that much better in two years, and that much better in five years.

It’s the connection of the A.I.‘s ability to learn, and track you, and costlessly experiment with what will work for you, then within the ability to deliver that to you that makes something different. I mean, Jaron Lanier has often talked about the fear of the internet turning into a Skinner box — Jaron Lanier, the technologist, and V.R. pioneer, and I think techno-philosopher — where it just gets so good because of A.I. or other things at taking in signals from you, and then using it to manipulate your behavior. And that we don’t want that level of persuasion to be cracked. That, in many ways — maybe another way of putting this is maybe the nightmare is not the nightmare of your book, the nightmare where the ad agencies recognize persuasion is collapsing. It’s the reverse.

It’s that the systems actually get really, really, really good at persuasion and have the information from you, which they maybe have now, but then also the systems to iterate and experiment based on that information, so they’re not giving you returns on that that are ridiculous. That makes a truly scarier internet not just for selling you things, but political propaganda and disinformation and fake news and all the other things one might worry about.

Yeah. I mean, it’s possible. It’s a really delicious long bet, because the root thing of what you’ve just said is people having an intimate relationship with these agents. And I don’t know. I should just ask you, are you friends with ChatGPT? Do you feel that you’ve really built up a relationship with any of the L.L.M. technologies or could build relationship with any of those technologies? I’d be curious to hear what you say about that.

Oh, I don’t think I’m really friends with ChatGPT. But what I do think is that I’m — Google knows enough about me that if you imagine Google has its own version of ChatGPT, and that is not having any direct relationship to me that I know about, but it’s pulling in this data that is used in very crude way now by the ad marketplaces.

But is now consistently experimenting based on these signals and serving me up new, and new, and new content — really micro-targeted at me, which would be prohibitively expensive to do right now when you’d need a person creating this targeted experimental content for every other individual person. That that’s where you pass some kind of threshold that the corporations can’t currently do now because the scaling cost of that would be prohibitive.

Yeah, it’s — again, I think it’s possible. I would do the meta-dodge here, which is I think that the very fact that we’re talking about this, and it is dystopian and creepy, makes it a hard sell to the average consumer, I think. Because this is basically the creepiness that people experience with retargeted ads for bikes, for instance, times 1,000.

And I guess the question is whether or not that level of intimacy is also going to come at the expense of trust, which I think is also maybe an interesting outcome. Which is I can deliver you the most micro-targeted excellent ad ever in some respects, but off the bat, you find it really weird that a system has provided a recommendation of that level of granularity. Again, as always, I think the exercise will be, can it become normalized? And I think that is almost a picture — something that sits outside of technology.

I think that’s a very smart way of thinking about it. I think it’s a good place to end, too. So always our final question. What are three books you’d recommend to the audience?

There’s three books I’ve been reading — or have read as of late. First one is, I’ve been revisiting a bunch of books from the early part of the 21st century, like the 2000s, that were all excited about the prospect of the social web.

So the book I’ve been reading there is Clay Shirky’s “Here Comes Everybody,” which is this early book and filled with excitement about the promise of social technology and social media. And I think it’s so interesting to revisit that rhetoric just because I think it teaches us a bunch of lessons about what went right, what went wrong.

The second one is — my friend and I — Craig — are working on this podcast, “Almost Impossible,” so I’ve been reading a lot about Vegas. So there’s a great book I’d recommend by Sally Denton called “The Profiteers,” which is about this company Bechtel, which is one of the big companies that helped build the Hoover Dam, and basically became the mega infrastructure builders of the 20th century. It’s completely fascinating — explains much of the west, explains nuclear power. The company is in everything, and so it’s a totally fascinating story.

And then the last one I’ll mention is maybe a little harder to find, but I’d totally recommend it for anyone who’s interested in it — the book called “Jim Ravel’s Theatrical Pickpocketing.” It’s a book that came out in the ‘80s but is still available.

There’s a magician by the name of Mike Cavaney who sells it on his website. And it’s basically a very practical how-to guide to do pickpocketing as entertainment performance. And I’ve been looking into it because there’s this really fascinating history between the history of clothing and how magical illusions are performed — where your pockets are, whether or not your average audience member has a vest on, that kind of stuff.

And that’s a final one that I’ll mention that is just an incredibly fascinating read. And if you want to steal someone’s watch, it’s good for that as well.

[MUSIC PLAYING]

Tim Hwang, thank you very much.

Ezra, thanks for having me on the show.

“The Ezra Klein Show” is produced by Emefa Agawu, Annie Galvin, Jeff Geld, Roge Karma and Kristin Lin. Fact-checking by Kate Sinclair, mixing by Sonia Herrero, original music by Isaac Jones, audience strategy by Shannon Busta.

The executive producer of New York Times Opinion Audio is Annie-Rose Strasser. And special thanks to Pat McCusker and Kristina Samulewski.

[MUSIC PLAYING]