Podcast Transcript: John List on Insights From Uber, Paying Bonuses and Scaling Ideas

This is an edited transcript of our podcast episode with John List published on 25 February. He is the Kenneth C. Griffin Distinguished Service Professor in Economics at the University of Chicago. His new book is The Voltage Effect: How to Make Good Ideas Great and Great Ideas Scale. He has worked with firms such as Lyft, Uber, Citadel and several non-profits. His academic research includes more than 200 peer-reviewed journal articles and several published books. In the podcast we discuss, why field experiments are important in economics, how to avoid false positives and knowing your audience/market, what makes a good business idea, and much more. While we have tried to make the transcript as accurate as possible, if you do notice any errors, let me know by email.

Introduction

Welcome to Macro Hive Conversations with Bilal Hafeez. Macro Hive helps educate investors and provide investment insights on all markets from crypto, to equities, to bonds. For our latest views, visit macrohive.com.  

It’s been a wild week with one story that has dominated all. The Russia-Ukraine conflict. In many ways, events like this show how Macro Hive is different from others. While many others pretend to suddenly become experts on Russia, Ukraine, and post-Soviet politics we get the real experts. For example, the day before Russia invaded, we hosted a special webinar with Russia expert, Dr. Samuel Ramani. He provided the historical context for this conflict and laid out a roadmap of what could happen next. You can watch the webinar and read our notes on it at the Macro Hive website. Another difference with Macro Hive is that we’re not prone to sensationalise the news. Many pundits are talking about World War III and are veering into amateur political analysis. That might be appropriate when you’re talking in the pub with friends, but not when you’re talking to investors about their life savings. Then there are those on Twitter and the blogosphere that are trying to shock investors into making dramatic investment decisions. We prefer to keep a cool head, recognise the level of uncertainty we have and focus on capital preservation. If you’re a Macro Hive member, you’ll know this already, and if you’re not, you can become a member at macrohive.com. With membership you can get access to all of our updates, webinars, transcripts, and our member slack room, where you can interact with the Macro Hive research team and other members all hours of the day. There’s been a lively discussion on Russia in recent days, which I personally have found to be invaluable. Membership to Macro Hive costs the same as a few weekly cappuccinos so go to macrohive.com to sign up now. 

And if you’re a professional or institutional investor, we have a more high-octane team product that features all of my and the Macro Hive research team’s views, our model portfolio, trade ideas, machine learning models, and much, much more. You can look me up on Bloomberg or email me at [email protected] to find out more. 

Now a word from the sponsors of this episode, Masterworks. Are you worried about inflation? Perhaps you should be. Every month inflation is reaching new highs, 6.2%, 6.5% and most recently 7.2% in the US. That’s how much more food, electricity and housing is costing these days. And if you are anything like us, now may be time to consider alternatives. Why alternatives? Well, for starters, a survey last year conducted by Motley Fool reveal that ultra-high net worth investors had 50% of their portfolio in alternatives, on average. We’re talking about portfolios valued at over $1 billion. But one alternative that nobody seems to be talking about is fine art. That’s because up until now, you’ve had to have millions to own fine art, but that’s not the case anymore. Thanks to Masterworks, they let you purchase shares that represented investment famous artworks. Pieces from masters like Warhol and Banksy at a more approachable price point. You can go to masterworks.io/macro to skip the line and join over 300,000 members already on Masterworks. That’s masterworks.io/macro. See important regulationary disclosures, and the offering circular at masterworks.io/about/disclosure. 

Now, onto this episode’s guest, John List. John is the Kenneth C. Griffin Distinguished Service Professor in economics at the University of Chicago. His new book is The Voltage Effect: How to Make Good Ideas Great and Great Idea Scale. He has worked with firms such as Lyft, Uber, Citadel, and several non-profits. His academic research includes more than 200 peer review journal articles and several published books. John was elected a member of the American Academy of Arts and Sciences in 2011 and a fellow of the Econometric Society in 2015. He received a 2010 Kenneth Galbraith award. The 2008 Arrow Prize for Senior Economists for his research in behavioural economics, and was the 2012 Yrjö Jahnsson Lecture Prize recipient. He is current editor of the Journal of Political Economy. You can tell he’s a super smart person. 

Now, onto my interview and look out for my coffee chat with John on sports at the end of the interview. 

Greetings John, I’ve been looking forward to this conversation. I recently finished reading your book and I have to say it’s one of the best books I’ve read so far this year. And I think it could be really used by a lot of people in lots of different domains. Welcome to the podcast. 

John List (00:04:18): 

Wow. Thank you so much. And thanks for your kind words about my book and thanks for having me. 

Bilal Hafeez (00:04:24): 

Great. Now, before we go into some of the concepts of the book and some of the practical advice, I always like to get the origin story of my guest. And so it’d be great to hear something about your background. We can perhaps start with what you studied at university. Was it inevitable you’d become an academic, you’ll be in economics and just a history of how you’ve ended up where you are today? 

John List (00:04:44): 

Oh, sure. Wow. Okay. Let’s begin with a family that is a blue collar family. My grandfather was a truck driver, my dad was a truck driver, my brother currently is a truck driver, and my family is one that had never sent anyone to college, so when I had the idea to go to college, it was met with some scepticism. And the typical question of a first gen kid is, “Why do you have to go to college? Look at us. We have a great life and we never went to college.” My mom’s a secretary, my dad was a trucker. And then they argue by example, look at Bill Gates, he didn’t have to go through college and he’s rich. I ended up going to college for the most part to play golf. And my dream was to be a professional golfer so I was good enough to get a partial golf scholarship so I went to a school called University of Wisconsin at Stevens Point. 

We were it’s called division two/division three, and that meant I could get a little bit of money for an academic scholarship. I did reasonably well on the standardised test too, so I could qualify for a little bit of money that way. We started golf that fall. This would’ve been the fall of 1987, my first year as an undergrad. And I learned very quickly that I would never be good enough to be a professional golfer, at least not the kind that you’d see on TV. It’s important to fail early and to quit early so I learned early on that I would never be on TV making money. That was clear. I could have been a golf pro at the country club level where I would give lessons and stuff, but that really wasn’t that interesting to me. 

I quickly pivoted to my other dream, which was I loved economics. I took one econ course in high school and what I found was economics was just how I naturally thought about the world. It wasn’t contrived for me, it was a natural way to think through choices and to think through problems that I would confront, or I would see others confront. And I was oftentimes puzzled when others didn’t do the obvious thing, and the obvious thing what I learned later was through the lens of an economics model. As I took econ courses, I fell deeper and deeper in love with econ. And I knew it was my comparative advantage so I knew that this was something that I could be good at. And when my professors at the university started to say, they think I had talent and I should go to graduate school that’s when I became serious about wanting to be a professor because I really enjoy mysteries and solving puzzles and I think I have a knack for thinking about problems in different ways than others. 

And if you can think differently than others, you’re a step ahead in the academy because at least you have a shot at doing something that’s novel, so that led me to the University of Wyoming. And at the University of Wyoming, that’s where I did my PhD. And there I was put in machine field experiments. My field experiments really started in the late eighties when I was a baseball card collector and baseball cards are these little cardboard things that people take to a convention and they buy, sell and trade. I had been doing that on the weekends with my girlfriend to try to make some side cash when I was an undergrad. And I was doing experiments there to try to learn about, did these people do what I read in the books that they should do? 

And I was exploring economic theories in that marketplace so when I went to grad school, my idea was, look, I can use these conventions as my laboratory, and I can do it because I can fund my own research because I have this huge baseball card collection that I collected when I was a kid. When I was a kid, I would shovel snow in the winter, I was raised in Wisconsin in a small village, I would shovel snow for money. In the summer I would cut grass for money. And after I was paid, I would go to the local store and buy baseball cards in the late seventies and then I would keep them in good condition. Later on what I found was they became very valuable and I could use them to fund my science. 

As a graduate student, I wanted to use the world as my lab and I would start, because nobody was doing it back then and nobody would fund that work back then I had to be self-funded. I would go to these baseball card shows as a grad student in the early nineties and I would start to do the experiments proper that I’ve been doing ever since. 

Why field experiments are important in economics 

Bilal Hafeez (00:09:45): 

At the time, obviously field experiments were not as popular, or as common as they now started to become. But why do you think economics didn’t adopt this sooner? Because economics tries to say to science, and obviously if you look over at the physics department or the biology department, they have experiments. And so why did economics not embrace experiments sooner? 

John List (00:10:05): 

It’s really a great question. When I look through the history of economics and I look for how the great people were thinking about empiricism, so you can look at Joan Robinson. Joan Robinson has a quote, basically that says, “It would be great if economists could be like the hard scientists and do experiments, but they can’t, they have to use historical data.” You look at Milton Friedman, you look at Paul Samuelson, in their writings what you see is this thread that says, wouldn’t it be great if we could be real scientists and run real experiments, but we can’t because the world is really complicated and there are simultaneously many moving parts. And because there are so many moving parts, it’s impossible to have control like the chemist does. The chemist has these really clean test tubes, and as you take your first chemistry class and run your first experiment, what you learn is if there is a speck, just a speck in your test tube, you’re done because you’ve lost control of the experimental setting. 

The mentality of the great minds through the 20th century in economics was because there is a speck and a lot of specks of dirt in our economy and in the real world, there’s no chance. They were wrong. They were wrong because they didn’t imagine that if you used randomisation and you randomly put people into treatment and control groups that that would allow you to balance the dirt across treatment and control, and then when you differenced out the outcomes, the dirt would just get differenced out. They were in this mentality of a chemist, a physicist, a biologist, control, control, control, and they just missed the beauty behind what a lot of people, Pasteur and the great medical workers, and Fisher, what they were doing is they were leveraging randomisation and the beauty behind randomisation to identify a causal treatment effect. 

And that’s really what they missed, so I was lucky because I entered the world where there were some lab people, but the lab experimentalists have always gotten chastised for “not being in the real world,” and it’s an artificial environment. Now, I think it’s artificial for a beautiful reason, but it never became huge. Even though Vernon Smith won the Nobel Prize for lab experiments, but they never became huge and took over because people always had a bit of scepticism about them. Now, field experiments, of course, it’s using randomisation in the real world in an environment where people like to be, and they like to learn from. That’s a big reason why, in the early nineties, I was lucky, I was fortunate because I came at the right time and no one else had really done them in mass so I got to be one of the first people, and that’s what I’ve become known for now. 

And I would call that more or less the first part of my career is using the world as my lab and asking questions like, is there discrimination? And then why do people discriminate? The beauty behind the experimental method is you not only can measure something. Mountains and mountains of data can measure things, you can take the CPS data set, you can go to the Bureau of Census and you can measure if people are treated differently. 

Field experiments can do that too, but field experiments can do something else that’s beautiful that no other approach can do, and that’s determining why that is happening. First of all, measuring discrimination and then determining, why does the discrimination occur? And that’s why the method is so beautiful because it allows you to unlock the underpinnings or the mechanisms for why something’s happening, and then that of course unlocks the solutions to why this is happening. Really through the nineties and two thousands, I was working in discrimination, early childhood, all kinds of social issues. And then around 2014 is when I was hit with the contents of this book. But I’ll stop there and say that we want to go back to the nineties or 2000s we can talk more deeper there, but I’ll let you take it from here, Bilal. 

Bilal Hafeez (00:14:40): 

Yeah. That’s a really good background there. One thing that struck me is, this is a more philosophical issue around theory versus empiricism, in science there’s this big debate, should you let the data lead you, or should you have the theory and then the data validates? It’s like a Karl Popper argument, or David Deutscher, explanations are the important thing and the data, the problem conduction. How do you think about that? You do need ex-ante a theory of something. You need to have some model in your mind, or you need to have some thinking about why something should be the way it should, or can you not have a model? It’s like model agnostic and you just let the data talk, so to speak. 

John List (00:15:14): 

Let me cop out and say both, and let me explain why. You need both theory and data, full stop. These are compliments to one another. Now a beautiful image of the scientist writing down a model, and then looking at that model and saying, okay, I need these five treatments to figure out which parts of my model are right. That’s wonderful. If you could simply do that, write down a model, test aspects of the model using the experimental approach, great. I have a lot of papers that do that. Now on the other hand, sometimes you don’t imagine something happening in data and the data just jump out at you. And you say, wow, that is a wonderful, wonderful data pattern. Now, what could that be? 

And then you do a little ex post theorising, letting everyone know this theory is ex post, and then the next generation comes and tests your theory. I don’t see anything wrong with that approach either. I always am a believer that it’s theory that’s portable. It’s very difficult to have only data because data are of course, a function of the times. They’re a function of the situation, which I talk a lot about in the book. When you talk about generalising and you’re talking about people, now you have to think about generalising across types of people, across types of situations, across types of people by situations. And there’s this dimensionality issue that becomes very rich and important, which is all together different from the laws of gravity. Or certain laws of physics that we know, if it works in Seattle, it’s going to work in London and we have a quantitative law and that’s beautiful. 

But when you’re dealing with humans, it’s very difficult to conceive of a quantitative law. We have laws like the law of demand, the law of supply, in a way they’re obvious. When price goes up, quantity and demand, it goes down. That’s the best we can do when dealing with humans. I think that there’s a voltage effect law, that’s why I titled my book, The Voltage Effect. When dealing with humans it’s an altogether different type of story and approach than when we do our research without humans. Without humans, you can envision a lot of basic science being extremely important. Basic science is important in the area of economics, but in a way that’s just the beginning to something that you have to do in the end to figure out in which types of situations and for which types of people will this thing work for. And the ones that it won’t, I need to develop something new, a new programme. And that’s just the nature of working with humans. 

Bilal Hafeez (00:18:04): 

Yeah. Points well taken. I would also add, I suppose, that when you are working with data, you’re also applying some theory as well on how you want to organise the data. It’s not as if the data is there and you just, in an unfiltered way, just use the data. You’re bringing certain assumptions to mind, you make adjustments, you select certain data, you drop other data, so there is some thinking around how you even approach data. 

John List (00:18:26): 

That’s exactly right. A lot of times we think about tying yourself to the mast. Here’s an experiment I’m going to run. These are the four treatments, this is the number of people I’m going to put in each treatment, and this is going to be exactly the outcome set that I look at, and this regression will be exactly the one I estimate. We are starting to move in that direction and that’s a good thing because we claim to control the false positive rate and we’re really not doing it that well as a science. We claim that alpha’s 0.05, but by the time it gets to the journal and somebody reads it, that Alpha’s probably 0.4 or 0.5, given everything that this whole thing has gone through to get you into the academic journal pages. 

What John learned at Uber 

Bilal Hafeez (00:19:12): 

No, no, no. Point well taken. Now, you then had this second part of your career, that’s been running in parallel where you’ve done work in the private sector. You worked for Uber, Lyft, amongst others. What made you go into that side as well? 

John List (00:19:26): 

I think the second part of my career is one part working with firms. Now, that really started in the late nineties when I started working with nonprofits and I started to explore why people give to charitable causes and what keeps them committed. It evolved into working with, as you mentioned, I was a chief economist at Uber for two years. I’ve been the chief economist at Lyft, which is Uber’s main rival in America for four years. And I was attracted to private firms, really a hundred percent because of the sandbox. 

And what I mean by the sandbox is it gave me an opportunity to enlarge my footprint on using field experiments to learn about the real world. And it unlocks this incredible black box within a firm to not only figure out how’s the firm operating, but also in ride share, I have the demand side and the supply side and in a way it’s beautiful because I can explore each and I can look at them simultaneously, and I can ask a set of rich questions and answer a set of rich questions in a way that nobody else has before. And that’s really attractive to me. 

Now, back in 2010, I was offered the chief economist position at amazon.com and I nearly took it. I didn’t take it for one reason. Jeff Bezos said, John, we can’t wait until you’re here and by the way, none of your your work will ever make it outside Amazon into academic journals or books. And I said, I’m out. And I said, I’m a scientist, and I’m doing this for science. I want to help your firm. But I also want to do this for science. Look that decision, the opportunity cost of that decision is tens of millions of dollars. But I would make it again in today if I had to. 

Bilal Hafeez (00:21:28): 

Yeah. You were offered the gold to give up your philosophy and you decided not to, and stick to your philosophy of openness. 

John List (00:21:34): 

No, I think that’s right. I think that’s right. Now, in tandem with working with organisations comes the problem scaling. And then so the intellectual part of my research agenda in the last 10 years has decidedly moved into thinking more about scaling. 

The optimal way to get tips 

Bilal Hafeez (00:21:51): 

Okay. Yeah. Now, before we go into scaling in a bit more detail, you talked about some of the work you did at Uber and Lyft. Was there anything that really surprised you about the data you were working with or the behaviour of either passengers or drivers? 

John List (00:22:03): 

Where do I start? This is like choosing… I have eight kids, Bilal, this is like choosing which two who I want to talk about. Let me talk about something fundamental first, and that’s the usage of data and the usage of the field experimental approach in these firms. As I talk about in the book, when I first walked into Uber, I saw sign that said, data is our DNA. And you think in the back of your mind, all these firms have slogans and none of them are followed. In this case that’s not true. These firms, they believe that data is a world’s most valuable resource. It’s no longer oil, which I believe too. And they are data driven. And if they don’t have the data, they will generate the data through a field experiment. That’s point number one, there is more science going on within these firms than is going on in most college campuses in America. That’s point number one. 

Point number two is some fun facts. We rolled out tipping in the Uber app in 2017. And I was partly responsible for that because I pushed for tipping in our app because our drivers really wanted tipping. And what was interesting is because I pushed for it and won, my team, which is called team Ubernomics at Uber, we got to be in charge of the rollout. We did this as a nationwide field experiment. And what’s neat is that when you look at the data, one sad thing is only 1% of people tip every trip. Sometimes people ask me, “John, how can I get in the 1%?” And I say, “All you need to do is tip every trip.” They’re talking about a different 1% of course. The other side of that is three out of five people never, ever tip. 

It’s interesting, a juxtaposition is that in America, at restaurants, when you look at tipping, it’s almost like a tax. It’s not a tip anymore, it’s a tax. When you sit down, you have to pay 20%. You have to pay the piper. On purpose we separated the decision to tip in both space and time. What I mean by that is the trip ends, you leave the car, the driver gives you a rating and not until after they give you a rating, can you give them a rating and decide on the tip. That separation in space and time makes the active tipping, not attacks anymore, but something that truly should be for good service, for excellent service. And we know this is because when you look at the tipping data with cabs, now there might be a different selection of people, et cetera. But I think the real difference is when it’s face-to-face tipping with cabbies, people tip 95% of the time. 95% of trips are tipped. On Uber, only about 15% of trips are tipped. And I think the big reason is because you don’t have the social pressure. 

Bilal Hafeez (00:24:55): 

Why didn’t you want to have it face-to-face tipping? 

John List (00:24:58): 

Travis Kalanick was a founder of Uber and he was a CEO back then. One of the conditions was he wanted it not to be a tax, he wanted it not to be a price increase because he claimed that there were a lot easier ways to increase prices. And he said, “My goal is 10 to 15%.” He just drew that out of thin air, but that was then our goal. And as it turned out, it was 15% so we were in that band, which is more luck than anything else. But we did a bunch of beta tests and stuff too that ended up showing that that might be the sweet spot. 

Why understanding scaling is important 

Bilal Hafeez (00:25:30): 

No, no, that’s great. Yeah. Okay, let’s talk about scaling. I’m in the startup world now. I worked for big banks for a long time, then two and a half years ago set up Macro Hive startup, and when you speak to other startup founders, everyone talk about scaling, you want to do something that will scale rapidly. And one level that sounds very inspirational, but then at another level it sounds a bit trivial to say, okay, scaling just means become big and successful. When you think about scaling, you’re an academic as well, so I’m sure you’ve done this, but how would you define scaling? 

John List (00:26:03): 

We have words like creativity, critical thinking and scaling, that when you ask the random person on the street or ask a VC person or ask a college kid, whatever, what is the meaning of this word? Ask 30 people you’ll get roughly 25 different definitions of each of those. And it’s sort of, you know it when you see it. Now for me, the way I think about scaling, what was really driven by the roots of why I became scientifically interested in scaling, which was I started a pre-K programme with Roland Fryer and Steve Levitt in a little city here in Chicago called Chicago Heights. And we built that thing from soup to nuts. It started in 2008. 

Bilal Hafeez (00:26:47): 

So this is schooling for young children. 

John List (00:26:49): 

This is three, four and five year olds. So I started a pre-school for three, four, and five year olds. And every day I had my boots on the ground, getting licences, hiring teachers, getting janitors, getting buses, et cetera. So we create this programme and four, five years later, we start getting great results. We find that we are moving our kids tremendously over a six to 12 month period, both cognitively using cognitive standardised test scores and non-cognitively or executive function skills. So I then start to talk to policy makers because I not only wanted to help Chicago Heights kids, I wanted to test economic theory. And this is called the education production function for three, four, and five year olds. I wanted to figure out what works and why. And then I wanted every child in the world to receive my curriculum that I developed. 

And I was met with a slap in the face. And that slap in the face came in the form of, “Professor List, as policy makers we see this all the time. You have these great results and then when we scale it, we scale it from Chicago Heights to all of Illinois, or all of the Midwest, or all of the United States. You know what happens? We only get a fraction of the impact that you promised we would get.” So to me now, scaling has two dimensions. One dimension is I try it in Chicago. Does it also work in Denver? Does it in Washington, DC? Does it work in New York? Does it work in London? That’s what I call horizontal scaling. The other aspect of scaling is what I call vertical scaling. And what that means is I have a pre-K or a school for three, four, and five year olds in Chicago. I had to hire 30 teachers. Let’s say now, I want to populate all of Chicago with hundreds of pre-schools. So now I’m going to have to hire 30,000 good teachers. This is vertical scaling because I’m hiring teachers from the same input market, from the same labour pool. And that is an all together different type of question, vertical scaling versus horizontal. They’re both important. And they both have a seat at the table when it comes to scaling, but that’s the way I think about scaling. 

Bilal Hafeez (00:29:19): 

Okay. No, I understand. Yeah, that’s clear. And then in terms of the use of the term voltage effect. Why did you pick that rather than the scaling effect or something like that? 

John List (00:29:30): 

No, absolutely. So I think of scaling as an act, you have something in the small and you expand it or you scale it up. So early on, I thought about, “Well, maybe the title of the book should be scale up.” But then I started to think of something more active and that more active thing is what I continuously see happening in data is that in the small, you get one kind of result, but then in the large you get a totally different result. And that’s what I call the voltage effect. It’s an effect that is a law really, because in every data set, you see it. It looks great in the small, but in many cases it doesn’t look so great in the large, that’s what I call a voltage drop. But there are other ideas that look great in the small and they become even greater in the large, that’s what I call either a high voltage or a voltage gain. So the voltage effect really is a description of what happens to your impact when you move from the small to the large. 

Bilal Hafeez (00:30:37): 

Okay. Understood. And so in your book, you lay out the ideas behind this very clearly, and you start off by talking about whether an idea can scale full stop. Does it have that potential to scale? And you list five different factors that you think are very important here. Let’s run through each one of them. And perhaps we can start with false positives as the first ones you start off with. First, why was that first? Then you can talk a bit more about what it means. 

John List (00:31:04): 

No, absolutely. So you’re right. So the first half of the book details the five vital signs that any idea must have to have the potential to scale. And I started with false positives because if you don’t have voltage in the small, you never have a chance. So false positives essentially mean it looked great, but the data were lying. In statistics, that’s called statistical error. We have a population of people and we go and draw a sample, but that sample was not representative of the broader population. So if you would go back and draw another sample again, you won’t find a result. And if you draw another sample again, you won’t find a result. The first one was just incredibly unlucky because it told you there was voltage and there really wasn’t. So I thought that was a useful one to lead off with because that’s a necessary condition. If you have an idea that just doesn’t have voltage, switch ideas. 

How to avoid false positives and knowing your audience/market 

Bilal Hafeez (00:32:07): 

And what’s an example of a false positive, say in some of your studies that you’ve looked at or in the case of some of the companies you’ve worked with? 

John List (00:32:13): 

Absolutely. So the example that I lead off in the book with is Nancy Reagan. And a lot of us might not remember Nancy Reagan. But Nancy Reagan was the first lady throughout the ’80s, she was president Ronald Reagan’s partner. And she was a wonderful lady. So when I worked in the white house in 2002, 2003, she would visit and she would bring in cookies and just a wonderful person. In the 1980s, she took it upon herself to take on what she viewed was a major problem amongst teenagers in America, which was their drug use. So, and it kind of helped to popularise a programme called, Just Say No. Which was really a social inoculation programme from Dare, which the Dare programme was essentially a programme that was based on information. You should give teens information about how bad drugs are for their brains and for their human selves. And then that will inoculate them from using drugs. That was the theory. 

So I can still remember when they came into our high school. And the local officials came in and said, Just Say No. And I looked at my teacher and said, “No way, this is working. I don’t use drugs, but I have a lot of friends who do, and this isn’t going to affect their use.” And he looked at me and said, “You what John, maybe you’re right, but they do have data behind it.” So in this research, I went back and looked and they did have data actually. They had a really good data set from Honolulu. So Honolulu, Hawaii, they ran a field experiment with like 1777 teens, where some got the control group, the placebo and others got the social inoculation and they found it work. 

The problem was they didn’t try to replicate it and draw a new set of kids from Honolulu to see if it would work again and again. In the end, they just got really unlucky and they tried to scale it. They did LA, Denver. I lived in Sun Prairie, Wisconsin, did it all over America. We ended up wasting tens and tens of millions of dollars because we were using a programme that never had voltage to start with. And that’s a travesty in a way because we really wasted a lot of public resources. 

Bilal Hafeez (00:34:40): 

And I guess the question then is like, why didn’t they do more sampling? And I suppose there’s always this imperative to do something quickly. As soon as you get good results, you got to show results and you need to demonstrate to the public that you’re doing something. So there’s something that makes you skip some of the homework that you have to do before you scale it. 

John List (00:34:58): 

Yeah, 100%. I think that’s part of it. We want to move fast and break things and I totally get that. That’s art, that’s not using science, but it’s art. I think the other issue that was going on is confirmation bias. That it sounds kind of right and it sounds good like it should work. So when you have this kind of belief in something, when you see data that confirms it, all of a sudden in your mind, you think that’s the truth. You don’t say, “Well, maybe 10% that might be right.” You immediately, when you see a study, if you believe it at your core, when you see an academic paper, you automatically race to, “This must be the truth. And now I have an academic paper to prove it.” But just because one academic paper is published, peer reviewed journal, whatever, always remember one swallow does not make a summer. And we have to be sure that it is the truth and it’s not just a false positive. And we do have to fight with ourselves because confirmation bias is really powerful. 

Bilal Hafeez (00:36:02): 

And the best way to deal with false positive is simply to do more experiments across space, across time, just in all the different dimensions to make sure that it’s robust. 

John List (00:36:11): 

Yeah. I think that’s right. I think you not only want to think about across space or across time, but even that same population that you just drew from. Does it replicate in that same population? And in some organisations, it’s really easy to do that. I can do that in a night at Lyft and I can have hundreds of thousands of observations. For other ideas, it’s harder, like my early childhood programme. Now there, what you want to do is do pieces of it and see if pieces can replicate, if you can’t do the full blown thing. And that’s fine too. Now, government organisations, what I want them to do is at the very beginning do multi-site trials. So you can figure out very quickly. So I don’t want to slow down progress, I want to speed it up. But I want to speed it up in a way that we’re actually using policies that work. Because most of the policies we put forward just don’t work and they don’t work because we haven’t been serious and scientific about figuring it out from the beginning, whether it had voltage. 

Bilal Hafeez (00:37:07): 

Yeah. No, that’s a really good point. So, the first vital sign of an idea that could scale is avoiding false positive or dealing with false positive. The second one you have vital signs is know your audience. So what do you mean by that? 

John List (00:37:18): 

Yeah, absolutely. So when you’re working with a firm, most of the time they will say, “We want to introduce a new product,” and they go out and get a focus group. So that’s what McDonald’s did in the ’90s. And they had a CEO that was very bullish, a new sandwich called the arch deluxe. With that name, how can it fail? Well, I’ll have the arch deluxe. It sounds great. So they brought in some focus groups and the focus groups told them, “Wow, this is a great sandwich.” And then they gave them the question. “Well, if it was on the menu, would you purchase it?” Nearly everyone said yes. Now at this point you have to ask yourself. Number one, what are the incentives in that focus group to say, no? So if I’m in a group and someone says, “Look, I might introduce a new product.” Even if I’m not sure I’m going to ever like, and it’s an option value, I should say yes. Because then when you introduce it, maybe I’ll like it, maybe I won’t, but at least I have the opportunity to purchase it later. 

So really the incentives there are always to say, “Check, check, check, introduce.” The other problem here is the sample. Is that focus group a bunch of lovers of McDonald’s and lovers of hamburgers, or is this representative of the large that you’re selling into? Unfortunately in this particular case, is the case for nearly every focus group I’ve ever seen. The people who sign up and show up are people who have a greater affinity for burgers or for McDonald’s in this particular example. So they ended up introducing the arch deluxe and it was a huge flop. It ended up costing the CEO his job. 

So think about the academic. I announce, “I’m going to do an experiment on a heart drug.” The first people in line are the people who are most likely to believe that they need this heart drug. So that’s a unique sample. Now, if it works for the people who are first in line, you should ask yourself, does it work for the people who are last in line, or the people who haven’t even entered the line? We need to understand from the beginning, how big of a slice of the pie can our idea take on before we invest a tonne of money in it? 

Because look, it’s okay if it’s a really tiny slice, as long as you know it. And as long as you understand, “Look, we’re going to go after this part of the market because our idea gets a very tiny slice, but now I’m going to invest accordingly.” The big failures come when you take that tiny slice and think that it’s going to be a much bigger slice and you invest in, let’s say deep sunk costs or deep fixed costs that you can never retrieve. And now you’re in trouble because you can never recoup those initial investments that you made because you overemphasised the strength or voltage of your idea. 

Bilal Hafeez (00:40:13): 

And the way to deal with this issue is through randomisation presumably? 

John List (00:40:16): 

Not only randomization, but it’s also to get a representative sample. So this isn’t about just have people sign up and randomise over them. That’s the participation group, but you are also selling into the non participation group. So you not only need to randomise over the people who agree to taking on your idea. But if that’s only 5% of the population, what about the other 95% who haven’t signed up? I need to know something about them too. And that goes beyond just randomisation. This is now into sampling. 

Bilal Hafeez (00:40:50): 

Yep. Understood. I’m kind of thinking now when you speak about polling, political polling, there’s been some big misses in some elections in the US in particular, but even the UK as well. And there’s been a big debate around whether the polls capture the right people? Is it capturing white working class members? Is it too skewed towards education levels and so on? Do you have a view on that? 

John List (00:41:11): 

No, absolutely. I think a big part of it is sampling and representation and predicting who’s actually going to show up. A lot of times you don’t know two things. One, your probability of showing up and two, conditional on showing up, who are you going to vote for? The inherent weakness of polling is there are some voters… and this is just a fact of dealing with humans, that both of these probabilities are uncertain enough to where you’re going to have a pretty big air band. Now, the things that we can account for is the representation within our polls. And I think we don’t do that well enough typically because we tend to use more convenient samples. We say they’re representative, but what they’re representative of is people who have signed up to be part of it. 

The people who are living under a rock and never even answer you, how do you get information from them? You don’t. We do know the probability of them voting because we can look at the voter records. But there tends to be a disconnect on that cord. And then when you add with that disconnect, the inherent uncertainties of probability of voting and conditional on voting, who do you vote for? Then it becomes I think a standard air bar that is oftentimes bigger than what the pundits typically give us, because it’s hard to understand the size of that air bar. 

Bilal Hafeez (00:42:29): 

Yeah. That’s a good point. And also we know in lots of these studies has the issue of weird samples as it’s called. And maybe you could talk a bit more about what we mean by weird, that sample group and how to address it. And maybe some studies that have worked on weird samples and not worked on non-weird samples. 

John List (00:42:46): 

No, absolutely. So the weird sample is a tribute to Joe Henrich who was an anthropologist, or is an anthropologist. But as a student, he decided to tackle the question of, are there results that we find in the laboratory in the Western world? So what that really means is on college campuses, so we’re Western educated, that gives you the first part, rich democracies. So it gives you a flavour of, Look, these are highly educated college sophomores. And we’re going to them and we’re asking them questions or having those people be our experimental subjects to give us a glimpse into the world. So what Joe did is a little bit like what I was doing in the early ’90s and throughout as I was going to baseball card conventions, I was going to other markets. I wasn’t using the lab at the university, I was using the world. 

Joe expands that even more and goes to civilisations that very few people in the Western world have ever even heard about. So you can think about the anthropologist visiting a civilisation that it hasn’t even introduced a market. And their market might be purely barter, they don’t even have a currency. And what Joe finds is he finds results that are in many cases at odds with what these Western educated college sophomores were giving us, which only makes sense. The subject pool is different, the situation is all together different. When someone comes into an experiment in America or in the UK, in the Western world, they bring with them a set of norms about what are the appropriate behaviours. If it’s tipping and it’s face-to-face, I’m going to have to pay a social penalty if I don’t give you a tip, or at least I’m going to feel bad about myself. That’s a social image concern and a self image concern. 

These are altogether different than other civilisations that have evolved in different ways. And Joe’s data pretty much show that, that these other elements are very important. And if we want to have a glimpse into those parts of the world, it should behove us to go to those parts of the world and introduce institutions and mechanism to see, do they work in the exact same way that they worked in the Western world? Now this comes back to what we’re talking about before, because Newton would say, “I don’t need to go to Tanzania to figure out how fast the apple drops. I can just sit here.” When you’re dealing with people in situations, it’s not good enough. And that’s what at once makes our decision and choices and research problems a little bit harder when we’re dealing with humans. But it also makes it a little bit more interesting in that there are a lot of other dimensions that we need to concern ourselves with. 

Bilal Hafeez (00:45:49): 

Yeah, that’s true. Yeah. So many studies are biassed towards students from Harvard or Chicago or Oxford or Cambridge. And so it’s unclear whether they’ll replicate across different types of populations. It reminds me of a time where I used to manage global research teams at the various banks I worked for. And I always found managing teams in Japan, quite different from managing teams in London or New York, where HR would tell us the way you give feedback to somebody is that you are very straightforward. You say these are the three areas you’re doing really well and these three areas, you’re weak in. What I found was that went down terribly with the people I managed in Japan because they took it as an affront that I was criticising them. And so instead I found out, the best way to manage them was to use intermediary to give the negative feedback. So then I’d go through back channels to report the feedback. And then when I had the face-to-face with the person I was managing, it was all very pleasant. So yeah, everything happens in the background so that the face-to-face is very pleasant. So everything’s in the background. Whereas in the Anglosaxon model, you don’t go through back channels, it’s direct. 

John List (00:46:50): 

Oh, that’s super interesting. Now, that’s a great point. And you’re exactly right. It’s a wonderful parallel to what’s happening here. 

Whether it is the leader or the team that matters. Chef or the ingredients? 

Bilal Hafeez (00:46:57): 

Yeah. And then the other thing I realised was often I used do these big trips where I’d see clients in Tokyo, then I’d fly to New York, see clients. And I found that in New York clients would push back against your view. And if you didn’t push back to them, they would view that as you don’t have conviction in your view. So we’d have these heated arguments, and you can imagine New Yorkers, hedge fund guys, very aggressive, we’d have these tussles. Then once I was flying back from New York to Tokyo to do the same meetings and I was still in my New York mode and I was super aggressive with a client. And then the person from my company that accompanied said, “What did just do? You’re offending the customer.” And so I had to dial everything down and speak in a different way. So I can certainly see that there’s this bias, I think, especially if you are raised in Western countries at these so-called elite universities and elite Western institutions, that these are universal laws, that they aren’t really and you have to really understand who the audience is. But let’s go into the next vital sign, which is one of my favourite ones, the chef or the ingredients. We were talking at the top of this podcast about sports. And I had a question to you whether the coach of the team is more important than the players. And we ended up having some discussion around that, but I assume this is what this vital sign relates to. Is it the chef or the ingredients that’s important within your idea? 

John List (00:48:05): 

No, that’s right. So this is probably the richest of the vital signs in terms of the number of entries or dimensions that you should be thinking about or can think about. And it really begins with trying to understand from the very beginning, what are your non-negotiable? So when we started the pre-K school, what we learned is that having a good teacher is a non-negotiable. Our programme can work with good teachers, and it likely won’t work with a really bad teacher. So that gives us an indication now of the limits to the vertical of scalability, because within the same labour pool, I might be able to hire 30 or 60 good teachers. But 30,000 or 40,000 good teachers, good luck, won’t happen if you keep the same budget. So that means I have to go back to the lab and say, “If I want to vertically scale this idea, I can’t use this curriculum because one of the non-negotiables will not be available at scale.” 

Bilal Hafeez (00:49:17): 

And in this case, in the pre-K, I imagine it would’ve been tempting to say, it’s the curriculum itself that’s the non-negotiable. How did you know good teachers were part of the non-negotiable? 

John List (00:49:28): 

Yeah. Because the point is that, in your original design, you have choices about what data to generate. We decided to generate data where we had classrooms with really good teachers and classrooms with not so good teachers. So that feature itself allows us to determine is that feature a non-negotiable? So this is what I want listeners to do is you have to think about what are the potential constraints that you have at scale? They might be laws or regulations. When you scale across countries, for example, you might have a different set of regulations. It might be a constraint on an input. If it’s a human, what I find in my work is that unique humans do not scale, like a teacher. If your idea revolves around having a great or unique human, it won’t scale because you cannot hire those as you expand, you will end up giving on quality. And then you’re going to have lower voltage because you have lower quality humans. 

And that’s really why I start out this chapter by saying, “Is it the chef, or is it the ingredients now?” Because as you read it in the book, I talk about Jamie Oliver and all of your listeners have probably been to one of his restaurants. So Jamie Oliver is a story that is sort of an analogy or parallel to many, many restaurants. So you have a great, great restaurant that has wonderful success when it’s one restaurant. Now, from there, you have to determine, are we great because of the chef or are we great because of ingredients? And if it’s the latter, can we get those ingredients at scale? 

So you have to ask yourself, if it’s the chef, never going to work. Because you scale that up to 50 restaurants and you might think, “Well, wait a second, John, the chef can be at one, but you can hire a bunch of other great chefs.” No. And then you might think, “Well, wait, John, that great chef can teach the other chefs.” Good luck. That’s a fool’s errand. If you have something unique, it’s unique because it’s inherently unteachable and it’s unique because there’s no cookbook. If there’s a cookbook, it wouldn’t be unique anymore. So trying to scale humans is folly, it just won’t work. Now, if the restaurant is based on great ingredients, and if those great ingredients can be replicated at scale, we’re in business. Now, there are some go betweens. So think about the great pizzeria that has a brick oven. So is that brick oven great because it’s been in the same building for 200 years and the mixture of ashes, soot and all the compounds in there, make this great pizza? Or can I take that brick oven and replicate it? If I can replicate it, I’m in business. If I can’t, just stick with one restaurant. So the idea here now for the policy maker, I’m proposing something pretty radical in this chapter, because what I’m saying is typically we do public policy by following an approach that’s called evidence based policy. So people say, well, have evidence and then do the policy around it. 

What I’m telling you to do now is policy based evidence. Look at the constraints at scale and bring them back to the lab and figure out what are the ideas that work with those constraints at scale. And then those are the ones that we scale. Because from the very beginning, we will know that they can scale. That’s policy based evidence. 

Bilal Hafeez (00:53:22): 

Okay, understood. So rather than having a blank piece of paper and coming up with the first best solution, well, apparently first best solution. You go backwards to say, okay, what will be the policy constraints at scale? Bring that into the lab. And then only look at the subset of solutions that work. And so you end up with maybe the second best, but it can scale. 

John List (00:53:41): 

Well, it’s the first best, second scale. It’s the second best efficacy test. So if you’re doing an efficacy test, it won’t win. The best it will do is tie. But if you’re creating a policy that can scale, that’s first best now. 

How to think about spillovers and network effects 

Bilal Hafeez (00:53:54): 

Okay. No, that’s great. So we’ve gone through false positives, know your audience, chef or ingredients, and then we have two more vital signs. One is spillovers. And then there’s the cost trap. Let’s talk about spillovers. What do you mean by spillovers? 

John List (00:54:06): 

So spillovers is another vital sign that is pretty rich. And I talk about four types of spillovers in this chapter. Let me just talk quickly about two. The first one was really discovered by Sam Peltzman in the 1960s. So I don’t know if you remember, some of your listeners might remember Ralph Nader. So Ralph Nader was the hard charger in the sixties that really led to a lot of social change and a lot of regulatory change. So on the heels of Nader’s famous book, in 1968, the federal government enacted a law in the United States that said every automobile had to have a seatbelt. And they did that because they said, because of Nader’s work, we are going to save thousands and thousands of traffic fatalities. Fast forward to 1975, Sam Peltzman publishes his groundbreaking work that shows those regulations led to zero traffic fatality deaths going down, literally led to no impact. 

And the reason why is because people who were wearing seat belts undid the good stuff by driving more aggressively. So that’s called the Peltzman Effect. So that’s one kind of spillover. On the other end of the extreme are spillovers that can happen through markets. And here’s where I talk about Uber in this part of the book. And what example is, as I mentioned before, my team at Uber, team Ubernomics, was responsible for the tipping introduction within the Uber app. 

So what that meant was we did some beta tests of it in the summer of 2017. What we found was when you only allow 5% of the drivers to have ability to receive tips, they earn more and they work more. So it’s a win, win, win. But in that same market, when we allow all of the drivers in the marketplace to have tipping, what happens? They work more, but there’s such an increase in supply that the drivers drive around with empty cars more often. And they end up making the exact same amount per hour as what they made before tipping. So now the market has come to a new equilibrium. And that new equilibrium undid all of the good stuff that we found in the small. And that’s a voltage drop because the drivers earned the same amount of money as they did pre tipping when you allowed all drivers to receive tips. So that’s another kind of spillover that I talk about in the book. 

Bilal Hafeez (00:56:45): 

So there’s basically these unintended second round effects that create a new equilibrium. And we see that in markets all the time. Somebody discovers a great new model to trade markets or some new insights, but as soon as you act on it or it gets released into the wild, everybody copies it, suddenly it disappears and the outfits disappears. And so you are in this constant mouse trap almost. They’re trying to find all these best results, because you’re not factoring in how fast the system will adapt to this new intervention. 

John List (00:57:11): 

A hundred percent. Now, I’ve been very negative in this part. But there are many ideas that have network externalities. And what that means is in the small, it looks like, ah, it’s okay. But when you expand it, think about the number of Facebook users, or any social media type of platform. As it expands, it becomes more and more valuable for each user. In economics, that’s called network externalities. So the idea is to try to create products that also have that feature. That as you expand, these network-wide effects work for you. 

Bilal Hafeez (00:57:52): 

And in terms of dealing with the more negative spillovers, what can you do to try to overcome them? 

John List (00:57:57): 

Yeah. In some cases, what I want you to do is explore. So in our case, do 5% of people, maybe 20, maybe 40, and then figure out, is this a product we want to introduce? If in the end, it’s not going to increase driver pay, for example. Another way to think about it is, are there ideas that you can only give to part of the market and you can do it in a way that’s rotating? So then we do this a lot when it comes to pricing on the Lyft app. So if you get, let’s say, a case of excess demand. Now prices are changing to try to make the market come to a new equilibrium. There are some ideas that if you know about these market wide effects, you can tastefully introduce it in a rotating way that allows you to get the good stuff, and then everyone’s better off as well. 

Avoiding the cost trap 

Bilal Hafeez (00:58:51): 

Okay. Yeah. Understood. That makes sense. And then on the cost trap, which is the final of the five vital signs, what do you mean by the cost trap? 

John List (00:58:58): 

So what’s really interesting is when I worked in government, we were pretty much always talking about the demand side or the benefits of our programmes. So we were talking about who was helped by our programme? Are there distributions that we should care about? What I mean is, one group’s health and one group’s hurt. So what are the distributional aspects of our programme? If we scale it, how do the benefits change? That’s vital sign one, two, three, and four. And that’s where the government typically focuses. 

What’s interesting is that firms, so for profit firms tend to focus more on the supply side of scaling. So many times when you look at Elon Musk, Elon Musk is a very, very successful business person in large part because he gets scale economies. He gets what’s going to happen on the supply side as I expand. In economies, it’s called economies of scale. 

As I expand, does my cost per unit output, does it actually go down or does it increase? Every good business has some scale economies attached to it. And it ends up working because as you get bigger, it’s now very hard for new competitors to come in and take you on because you have these great scale economies. Maybe you’re producing at a rate that’s a third of where they’re going to start. And it’s very hard for them to compete with you unless they get some kind of government subsidy. And that’s why you have a lot of governments trying to subsidise new firms because they’re trying to allow them to get economies of scale. 

So there are a lot of ideas like the teacher example. So if at check, if I want to conform to vital sign number three and only hire great teachers, what am I going to have to do? I’m going to have to raise wages. And I’m going to have to go up the supply curve. Now I’m going to get a voltage drop because of the supply side. The benefit side stays great because I’m hiring all good teachers. But the supply side’s killing me because for each student I’m helping, it is costing more and more and more dollars because I’m going up the supply curve and I’m stealing people from Wall Street or a hedge fund to come and teach and I need to pay them more. 

How to incentivise scaling and why clawback bonuses work better than year-end bonuses 

Bilal Hafeez (01:01:28): 

Yeah. Yeah, no, that’s great. Okay. So we’ve kind of got a sense of what makes an idea have the potential to a scale. But then on the second half of your book, you then go on to talk about how do you get the voltage, high voltage scaling? The secrets to make it grow. So we believe we have something that can scale, but you need an additional number of steps to really enable that voltage to expand so to speak. And you talk about four different components or secrets to this. The first one you talk about is incentives that scale, all types of incentives. And this was a really fascinating chapter with some of the ideas you had, which were, I hadn’t thought about in the way you described them. But perhaps you can talk about that first as a way entry point to this section of the book. 

John List (01:02:09): 

Absolutely. So I think of the second part of the book exactly as you say. On the one hand, after you launch an idea, how can you maintain high voltage? And in part, these are mistakes that government, regardless if its local, state or federal in the United States or worldwide government, or firms, they always make the same mistakes. So on the one hand, I wrote this as these are the mistakes that I always see made. On the other hand, I wrote it as this part of the book can be really useful for my father and for my brother who are ordinary blokes. And they want to learn about how economics and how simple economic thinking can help their lives in the real world. So I like to think of this last part as me being a tutor and explaining these are the mistakes that even the best in the world make. 

And even if you don’t have a firm, a very simple economic lens into the world that can help make your world better, read this second part of the book. And one way it has nothing to do with, well, I don’t want to be an entrepreneur. Well, I don’t care because this part of the book is for you then. So you’re right. It starts out by saying the first part of the book is, what are the incentives that scale? And the way I think about this is what are the incentives that I use for my own kids to get them to go down a path that I want them to go down? So what this means is incentives are much more than just money. The types of incentives that we respond to in the world are in many cases, social norms. They are in many cases complying to what others think is appropriate. And what should I be doing and what should be my course of action. 

And then secondly, it’s the way that we frame our incentives. So one example is the clawback. So the typical way that people do incentives in the world is you work for a period of time. Maybe you work all year, and at the end of the year, you get your Christmas bonus. Great. It’s a thank you from the boss to you. And you’re happy. What I talk about in this chapter is we should flip that and we should flip it in that January 1st, you’re given your bonus. And then throughout the year, we talk about, can you maintain your bonus? Or at the end of the year, should we take away part of the bonus? Now this is important because it flips. The typical problem we have in the world is climate change, people dropping out of school, people not going to the doctor enough. This is because the effort comes today and the benefits come tomorrow. 

Humans have a really hard time figuring that problem out. So what this idea does is it moves the benefits to the front. And if you have the benefits to the front end, what will happen is something called loss aversion will kick in. And there are old psychologists, Danny Kahneman, Amos Tversky, that talked about loss aversion in the 1960s. A lot of you might know this. What it effectively means is losses are felt more than comparable gains. So this idea tries to leverage loss aversion to get people to work harder during the year. And it’s not just a theory. I did this in Chicago Heights with the teachers. I did it all around the world pretty much. With farmers. We do it with bean sorters in developing countries. And what we find is a lot of people like it because it serves as a commitment device for them. It’s their commitment to work hard during the year, rather than slack off. 

Bilal Hafeez (01:05:54): 

That’s a great concept. And it’s quite elegant that you can take advantage of loss aversion. I’m wondering why hasn’t it been adopted more? I guess one reason could be that companies may not know or be willing to pay people up front until the revenue’s in or something, perhaps. Maybe that’s one reason. Another could be, I wonder, is there some legal issue around, can you take money away from people or not? Or is it held in like an Escrow accounts perhaps or something? 

John List (01:06:16): 

Yeah, I think that’s the point is, so we found that it works even when we provisionally give it to people. So if you say, provisionally, this $10,000 is yours, and then we’re going to map your account over the year. And then at the end, that’s going to be the amount that we give you. If people would do that, now you’re in business. Where people have a hard time is physically giving the money. So we have some experiments where we also physically give people the money and then take it away. Like in Chicago, all the teachers had to sign a contract. And the contract said, we will get the money back. That works even better than the provision. 

But if you don’t want to get caught up in the morass of that, I would say the easier route is to say, this is provisionally yours in an account that you can see. And then it ticks down if you don’t perform. And then that constant reminder about the bonus will get people to not only begin to work harder, but to remain committed over the year. So I think if people use provisional rather than the physical, it will become much more popular. 

Bilal Hafeez (01:07:25): 

No, this is great. Great stuff. Yeah. I’m going to have to think about whether we employ that Macro Hive or not. It sounds very elegant. So I might get back in touch with you in terms of how to design this. 

John List (01:07:35): 

Oh, I like it. And I’m doing it right now. So I’m starting a new journal in economics called the Journal of Political Economy Micro. And I’m doing that with my six editors. I’m giving them the money up front. And then if they take too long to get a decision on their papers that they’re handling, every paper that it takes too long, they have a $500 fine. So I’m taking $500 out of their account. 

Thinking on the margin and quitting early 

Bilal Hafeez (01:07:57): 

Okay. Now, I’m definitely going to try to employ this in some way. Okay. So the next secret to high voltage scaling you have is what you call the revolutions on the margins or revolution of margins. What do you mean by that? 

John List (01:08:08): 

So in Economics 101 in college, we always teach these fundamental tenants or boulders of economic thinking. And one of them is, think on the margin. You always hear this all the time. Make decisions on the margin. Now, at this point, I can only imagine what is going through the student’s mind about what does that mean, right? It’s a bit like creative thinking and scaling and all these other words that have multiple meanings. We tend not to teach exactly how you can put that into action. That’s what this chapter does. And it does it in a way that I draw from all of the learnings that I had. I saw people do not make decisions on the margin when they’re making even billion dollar decisions. They just don’t. And this is what I call the Adam Smith memo in that chapter. 

But I won’t go into that, but let me just give you an example so the listeners can kind of understand what I mean. So I was sitting in a meeting at Lyft about a year ago. And the driver acquisition team came in to present to me their results. So you can say, what’s a driver acquisition team? Well, this is a team that’s responsible for recruiting new drivers to be Lyft drivers. Uber has a driver acquisition team as well. So they showed me some data. And in those data, they showed, you know what, when we place ads on Facebook over the last thousand drivers we’ve recruited on Facebook, it’s cost about $500 to recruit the average driver. Okay. And then, they showed me the same statistics using Google ads. It was roughly, the last thousand drivers, it cost about $750 per driver. Okay. And they said, given that, the next tranche of dollars, we are going to use Facebook ads. I said, okay, maybe. I said, can you give me some indication of say the last 25 drivers that we’ve hired on Facebook and Google? 

And they said, oh, we don’t have that. And I said, well, maybe you can get it for me. They sent it later. And what that showed is that, for example, I don’t want to give the exact numbers to give away corporate secrets. But Facebook was roughly $1,500 per driver for the last 25. And Google was like $750 for the last 25. And I was saying, whoa, whoa, whoa, whoa, whoa. If we could turn back the clock, we would’ve done those last 25 on Google, right? They said, yeah. And I said, you have to take thinner slices of the data to make a marginal decision because we want to know where the next steps should ago. And if you look at two coarse of data or data that are too broad, which averages many times gives us, that will give us the wrong steps to take, the wrong forecast. Take thinner slices of your data, that’s marginal thinking. 

So then at that point we said, well, the next tranche of money is going to go into Google ads. Because we’re going to get twice as many drivers, if the indication from the last 25 was correct. 

Bilal Hafeez (01:11:14): 

Yeah. No, that’s great. Yeah. That makes a lot of sense. So what you are essentially saying there is that the averages tend to obscure the dynamics of whatever the input is or the output, perhaps? 

John List (01:11:22): 

Not always, sometimes the averages will be right. But there will be important cases where they’re wrong. And that’s basically a metric of what is a marginal cost curve look like. And you can’t just look at one point an average without looking at the entire trajectory. 

Bilal Hafeez (01:11:37): 

Yeah. That’s very helpful. I mean, when you study economics, you have all these curves and everything, but the way you described it sounds more practical. So the third secret you mentioned is quitting is for winners. So what do you mean by that? 

John List (01:11:50): 

So, wow. My grandfather will probably roll over in his grave. Because where I was raised, we were talking about American football and it was a great old coach named Vince Lombardi. And in fact, for all of you who watched the Super Bowl, the Super Bowl trophy’s name is the Lombardi trophy. Yeah, that’s where it comes from. Is this old coach from Wisconsin, from the Green Bay Packers, his name was Vince Lombardi. They won the first two Super Bowls when he was a coach. So Vince Lombardi was one of these old school chaps who was very gruff and said, “Winners never quit, quitters never win.” This is how I was raised in Wisconsin. Steeped in the culture of you don’t quit. In fact, if you type in Google like inspirational quotes or inspirational quotes on quitting, you will find enough pollsters that if you printed them all out, you’d probably have to cut down half of the Amazon rainforest to make the paper for that. 

Society tells us that quitting is repugnant and we shouldn’t do it. If only we would’ve called it pivoting or calling an audible, we could maybe conform to that norm. So one reason why we don’t quit enough is because society tells us it’s repugnant. The other reason is our own faults. And it’s a bias that we all have. And it’s called, you neglect your opportunity cost of time. So what do I mean by that? So if you’re a data scientist at Lyft, you think about your job as I’m a data scientist at Lyft. You don’t think about it as, given I’m a data scientist at Lyft, I am forgoing the opportunity to be a data scientist at Shopify. That’s not the human mind. And I know it’s not the human mind because when you look at the data about why people quit their jobs, case number one will be my boss no longer appreciated me. Case number two will be, I didn’t get the promotion or the raise that I was promised. Case number three will be, I got cross with a coworker and I just had to leave. 

These are all points that are important, but they’re all parochial in the sense that I don’t look to leave my job until my current lot in life is soiled, and then I start to look. The fact that we don’t appreciate the other side of the coin is, what is my opportunity set? And have my opportunities gotten better? And if my opportunities have gotten better, those should also cause me to move. But I very rarely hear people say, “Wow, the market for data scientists got so much better that I had to leave.” People never say that. 

The reason why they don’t say that is because they are ignoring their opportunity cost of time. So what I’m proposing in this chapter is that understand society says it’s quitting and it’s bad. You call it pivoting. Understand periodically you should be looking at your opportunity set. Whether it’s a job or an apartment or a relationship or whatever, you should be just as likely to move when your opportunity set changes compared to when your current lot in life changes. And when you do that, you will quit at what I call the optimal quitting rate. You quit much more often because right now we just don’t quit enough. 

Bilal Hafeez (01:15:14): 

Yeah, no, that’s a very good point. Yeah. And I guess it takes some practise to think in that way, because everything around you, as you mentioned earlier, is telling you the opposite. 

John List (01:15:22): 

No, that’s right. Everything around us tells us not to do it. But the moment you start putting on an economic cap… One of the boulders in economics is to understand opportunity cost. The minute you do that, you start to say, “Wow, if I stay here, I’m missing out on that.” Like right now, if I stay at Lyft as chief economist, I’m missing out on being the chief economist of another firm. If that choice set gets really rich, that might cause me to move. 

How to scale culture 

Bilal Hafeez (01:15:47): 

And then the final secret you have is scaling culture. This seems almost insurmountable. Like how do you scale culture? It seems like such a big topic. How did you address this one? 

John List (01:15:56): 

Yeah. Well look, culture is another one of those C words that are hard to define, right? Creativity, critical thinking and culture. So this chapter was fun to write because it allowed me to explore and go back to a lot of the old work I had done on the gender pay gap. And in looking at what causes women or minorities to earn less money than men when they’re doing identical jobs. So what this chapter is inherently about is it gives you a playbook from the very beginning of your organisation to say, how can we create a diverse and inclusive environment where everyone will feel appreciated from the very beginning? And for me, that starts with the manner in which you write your job advertisements. 

So in this article I wrote a long time ago with Andreas Leibbrandt, we advertise a job, which we hired people for. And then we put in that job ad, “Wages are negotiable.” Okay. And then we used the exact same ad, but we left that sentence out, “Wages are negotiable.” What we found in is when we say wages are negotiable, women will negotiate their wages and they’ll negotiate as hard as men. But when we leave that sentence out, what happens is women shy away from negotiating. And from the very beginning then, they enter the work making less money than men. 

So this is all about society telling people what’s okay. And in your advertisement, you let them know this action is okay. In ambiguous settings, women tend to shy away and go back to what they’ve always been taught. Be ladylike. Don’t ask for things that you don’t deserve. Where the man is, rush in like a bull in a China closet and go for it. Right? That’s a stereotype and the bias of how many parts of the world raised their children. So it was fun to write this chapter to bring out the lessons for organisations. 

Bilal Hafeez (01:18:03): 

Yeah, and I would urge people to read that, actually read the whole book, but that section I thought was very powerful. Now, having been written this book, what’s the type of response you’ve had from people? 

John List (01:18:12): 

Well, so far it’s been excessively and exceedingly positive, but I have to understand that there’s a selection problem here. So I don’t think that many people will come to me and say, “Your book stinks.” That’s typically not how humans are built, but so far when you look at ratings and how people have responded to me, it’s been all systems go. So what I’m positive about is there are a lot of very, very positive responses in different circles. You have the layperson who’s saying, “This is very accessible. I love the way you wrote it.” You have the VC person saying, “I finally have some science to put behind where I can put some money.” You have many, many governments who I’m giving talks to all the time saying, “Wow, now we have this checklist and we can go over this checklist, and it’s a device that we can use, that we can make sure that we’re adding science to when we try to adapt and scale a policy.” 

So I think in that way, it’s been very well received. Still early. The book came out, what, three weeks ago, and we made a few bestseller lists. We’re up to number two on The Wall Street Journal list, which I’m proud of. So we’ve made several best seller lists and people seem to be confident. So fingers crossed. Fingers crossed that it can keep going. 

Bilal Hafeez (01:19:35): 

That’s great. I would definitely urge all listeners to go out and buy a copy of the book. Now, I did want to round off our conversation with a few personal questions, which I ask all my guests. One is, what’s the best investment advice you’ve ever received? 

John List (01:19:47): 

I would say don’t listen to others unless they have inside information, because I’m more of an efficient markets person. When I wrote about investment advice in The New York Times, I wrote an op-ed a few years ago. My investment advice to people is don’t look at your investments too often. There’s something called myopic loss aversion, which means that when we look at our investments and we see that we have paper losses, it makes us feel bad, and that then causes us to invest much less in equities than we actually should. So the advice there is think about your long term strategy, put the money in there, and make sure that you don’t look at it too often. Maybe look at it every six months or so. And then you won’t have the pain of the natural fluctuations that you see in your portfolio. 

Bilal Hafeez (01:20:37): 

No, that’s excellent advice. The next question was on productivity, whether you have any systems or productivity hacks that you use, because obviously you’re obviously very well read. You’ve got a foot in different worlds. I mean, how do you manage all of this? 

John List (01:20:50): 

For me, it’s all about understanding your production function. First of all, understanding what do you want to produce? And then you have to hire people around you that you can plug into that production function. In many cases, we make mistakes by hiring people like us. I don’t need another person around me like me. I have me. The only way you can scale an organisation that is human centric is to understand what are the bits of inputs you need to produce what you want to produce, and then you have to be really serious about hiring people into those spots. And then you have to let them go. 

I think it’s the worst management style, to oversee every small decision that people are making. I think agency and ownership is super important. Obviously, you watch over and make sure that everything’s okay, make sure you have a reporting line in which people are getting feedback about whether they’re going down the right hole. But to me, it’s always about understand your production function, whether it’s my team at Lyft, whether it’s my team at the University of Chicago, whether it’s I run a team in Australia at the John Mitchell Lab, I run a team in Buenos Aires called the JILAE Lab. Whatever the team is, understand your north star, understand your production function, and hire appropriately into it. 

Books that influenced John

Wealth of Nations (Smith), The Theory of Moral Sentiments (Smith), Anna Karenina (Tolstoy), Principles of Economics (Marshall), Economics (Samuelson), Elementary Principles of Economics (Fisher) 

Bilal Hafeez (01:22:14): 

No, that’s great. And then the final question was what are some of the books that have really influenced you? 

John List (01:22:19): 

I would start with Adam Smith, and it’s not because of his great writing style. I mean, let’s face it, when you have to explain specialisation in 19 pages, or a third of the words you have to look up to make sure you understood what he’s trying to get to, it wasn’t him being terse or being concise. It was the deep level of philosophical thinking, both in that book and the moral book, Moral Sentiments book. I think I was moved by Anna Karenina. I think Leo Tolstoy, the most brilliant first sentence in a novel, “Happy families are all alike. Each unhappy family is unhappy in its own way.” But that’s scaling, because scalable ideas are all alike and each unscalable idea is unscalable in its own way, but my book gives you five reasons why they’re unscalable. So I think that book was moving. 

And then I would say, I sort of am a sucker for textbooks, whether it’s an old economics book from Marshall or Samuelson, or an old experimental design book by Fisher. I think that these have always been attractive to me. And when I read, I tend to read textbooks in various fields, rather than novels today or nonfiction books even, or fiction books for that matter. I can’t remember the last time I read a fiction book. 

Bilal Hafeez (01:23:35): 

That’s interesting. Textbooks, quite unusual, but it makes sense. I can see where you’re coming from. In fact, you’ve read some of the classic textbooks as well. Not just the modern ones. 

John List (01:23:44): 

Oh, that’s right. I think that there are many missing gems in the older. What’s interesting is if you’re thinking about learning an area, I think many of us make the mistake that if we read something current, that’s going to contain all of the future writings or all the past writings in one gem, in one textbook. It’s not true. There are a lot of gems that people may have thought were just unimportant and had been forgotten, and I think when you look at some of the… A lot of the stuff is either wrong or misguided, but occasionally you find that gem and that makes economic calculus all worth it. 

Bilal Hafeez (01:24:24): 

No, that’s great. Well, I’d urge or all the listeners to purchase your book, and are there other ways to follow you? I mean, do you have a Twitter account or where would you direct people? 

John List (01:24:33): 

No, I’m glad you inquired. I just joined Twitter in December, and it’s called Econ for Everyone. So, if you type in @Econ_4_Everyone, I would love for everyone to join me. We’re picking up some steam. I think we have 8,000 or so connections there. And on LinkedIn, I also post a fair amount of my work on LinkedIn. There, it’s hard, because they’ve limited at my connections to 30,000. So they don’t allow me to take more connections. But if you follow me there, it’d be great. 

Bilal Hafeez (01:25:04): 

Okay, great. Well with that, we’ve had a super comprehensive conversation. A lot for me to take away myself in terms of things I can implement either in my personal life or in the work environment. So just a big thank you for taking the time to share all your findings. 

John List (01:25:18): 

Thank you so much for having me, and it was really wonderful to get to know you. 

Bilal Hafeez (01:25:24): 

Now, a word from the sponsors of this episode, Masterworks. Are you worried about inflation? Perhaps you should be. Every month, inflation is reaching new highs, 6.2%, 6.5%, and most recently 7.2% in the US. That’s how much more food, electricity, and housing is costing these days. And if you’re anything like us, now may be the time to consider alternatives. Why alternatives? Well, for starters a survey last year conducted by Motley Fool revealed that ultra-high net worth investors had 50% of their portfolio in alternatives, on average. We’re talking about portfolios valued at over one billion dollars. But one alternative that nobody seems to be talking about is fine art. That’s because up until now, you’ve had to have millions to own fine art, but that’s not the case anymore. Thanks to Masterworks they let you purchase a share that represent an investment in famous artworks, pieces from masters like Warhol and Banksy at a more approachable price point. You can go to masterworks.io/macro to skip the line and join over 300,000 members already on Masterworks. That’s masterworks.io/macro. See important regulatory disclosures, and the offering circular at masterworks.io/about/disclosure. 

Coffee Chat 

John List (01:26:43): 

So where are you located now? 

Bilal Hafeez (01:26:44): 

I’m based in London, in Wimbledon. Wimbledon, London. That’s where the tennis is. 

John List (01:26:48): 

Are you a tennis fan? 

Bilal Hafeez (01:26:49): 

I am a huge tennis fan. I try to watch as much tennis as possible. In fact, actually during Wimbledon tournament, people rent out their houses to tennis players, and the person who is super nice is Djokovic. He’s super, super friendly. If you bump into him on the streets during the tournament, he’ll stop, he’ll hold your camera, take a selfie with you. I mean, he’s super, super, super friendly. 

John List (01:27:10): 

Wow. So that was too bad what happened to him in Australia, then? 

Bilal Hafeez (01:27:13): 

I know, yeah. I actually quite like him, I think he’s really underrated. I think he’s an exceptional player, and just because he is not as smooth and slick as Federer and these other guys. 

John List (01:27:21): 

Or like Nadal, yeah. I think in the end, I mean, you’re better at prediction than me obviously, but I think in the end, Djokovic is going to have the most majors of those three. 

Bilal Hafeez (01:27:31): 

Yeah, I think so. Yeah, yeah. 

John List (01:27:33): 

Right. I mean, he would’ve probably won the Australian, right? 

Bilal Hafeez (01:27:35): 

Yeah, that’s his tournament as well. So I think he’ll get it in the end, and he’s such a good all round player. He’s not as injury prone as, say, Nadal is, and Federer now is just playing one tournament, which is Wimbledon once a year, and he’s not as good as he was. 

John List (01:27:48): 

Yeah. Federer is in trouble, but I think the French, I mean, Nadal might win again. Will he have a lead of two or three against Djokovic? Two. But I guess Djoker would be favourite in the US Open, and probably again next year. And he’s the youngest of the three, right? 

Bilal Hafeez (01:28:03): 

Yeah. He’s younger. He’s a couple years younger than the other guys. 

John List (01:28:06): 

And he keeps in good shape, it feels like. 

Bilal Hafeez (01:28:08): 

Yeah, I don’t think he’s had any major injuries. He keeps in good shape, and just the game he plays, it’s such a good all-around game, whereas Nadal’s a real power player so he really taxes his body, like destroys his knees and things like that. 

John List (01:28:22): 

So tell me, why do you think majors in tennis are so much more concentrated in the past few decades than they were before? So you have three guys who smashed, Sampras and Agassi, et cetera, whereas before it felt like if you picked three people in the nineties or eighties, you picked three people, you probably wouldn’t get the winner right. But now it’s like you’ve got it dialled in. Why do you think that’s the case? 

Bilal Hafeez (01:28:50): 

That’s a really good and hard question to answer, because every year all the tennis pundits are saying the next rung of guys are going to come to through, whether it’s Andy Murray for a period was the next guy, and then Dimitrov. 

John List (01:29:02): 

Medvedev now is supposed to be good. 

Bilal Hafeez (01:29:04): 

Yeah, Medvedev’s a new guy, but they never quite make it. I think one thing is that, I think the style of play of, say of Federer and Djokovic, just their technique and their all round skill level is better than say Sampras, Agassi, Lendl and all those guys. 

John List (01:29:22): 

Sampras is pretty tech, I mean, relative to others, wasn’t Sampras, he may not have been the most athletic, but I think he was by far the most technical back then. Right? 

Bilal Hafeez (01:29:31): 

Yeah, he was the most technical. When I say technical, I should probably more accurately say versatile on different services. Sampras was more of a grass or maybe hard court guy, but he didn’t do so well in other areas. And I think the other thing is, I mean, Federer has shown, is the whole idea of picking tournaments. So restricting the amount of matches you play during the year and focusing on the majors, which has prolonged his career by three or four years. Somehow he’s been able to do that, whereas in the past it just wasn’t a done thing, and maybe that’s monopolised the majors more. 

John List (01:30:02): 

Yeah, so I thought part of your answer was going to be that in the past, there might have been more specialists, like Becker. Boris Becker was a specialist on grass, and you had some specialists on clay, whereas now these guys try to play on all surfaces. And when they do that, it allows the top people to be a little bit better than them on every surface, because they’re not trying to specialise as much as they should. But you didn’t say that, so I must be wrong. 

Bilal Hafeez (01:30:27): 

Yeah, no, it’s hard to say. I mean, people have been puzzled about this for years, and so people are always looking for the next rung of guys. I mean, certainly the fitness levels of these guys, there seems to be something going on, where these guys are just able to last longer and play for a much longer period of time than people in the past. So I don’t know if something’s changed on the nutrition side, maybe what’s happening in tennis is that they have a much bigger staff now than they had during the nineties or the eighties, with a specialist for every single part of your game. 

John List (01:30:56): 

Well, that’s the thing, the specialists have helped the best stay the best, but my question, that why don’t you have more specialisation amongst the players themselves? Like why isn’t there a Boris Becker who will have a shot every time Wimbledon comes, he’s going to focus on grass? You don’t seem to have that as much anymore, do you? I get that Nadal focuses on clay, but he wins the other ones too. 

Bilal Hafeez (01:31:21): 

Yeah, I don’t know. I don’t know the answer to that. It’s a really good question. A book on tennis I read talked about how amateur players, the way you win a match as an amateur is to not lose points. Whereas, professionals, it is about winning points. It’s the other way round. So if you are an amateur like me, I’m terrible at tennis, my main objective should not be to do a smart shot, it’s just to get the ball over the net. And the person who can just sustainably hit it over the net will win in an amateur game. Whereas, a professional, it’s all about the percentages. Yeah, I don’t know. It’s a really good question. Obviously, your interests seem really eclectic as well. 

John List (01:31:52): 

You live once, but the world is pretty rich, so I like to indulge in all of its riches. Whether it’s sports or books or whatever, it’s fun to learn. 

Bilal Hafeez (01:32:03): 

Yeah, I mean, I’m not a huge American football follower or anything, but I remember with my American friends in New York and so on, they would always be debating with each other about the Patriots, was it the coach or was it the quarterback? I mean, just the age old question. I guess this goes to your thing about the chef or the ingredients and stuff like that. 

John List (01:32:21): 

Well, that’s the thing, yeah. So in this case, it’s like a weakest link problem I think. I think there, you need both. So a lot of times people, that argument about which one is it is like a best shot argument, and most technology isn’t best shot. The NBA kind of is, because one great player, the weakest link doesn’t matter as much, but when it comes to the NFL, there’s a fair amount, the front line is like a weakest link problem. The quarterback, coach is sort of like a weakest link problem. A little bit like scaling. You just have to figure out what the technology and the production function is, and then you’re in business. 

Bilal Hafeez (01:32:55): 

Yeah. I mean, soccer, I follow football, our soccer, really closely. I support Liverpool, and the two big teams at the moment in European football is Manchester City and Liverpool, and they both have really good teams, but they have two coaches who are exceptional, who have slightly different styles. And then you have the next rung of teams, in the Premiership, who have the same amount of money, same talent pool, but weaker coaches. It’s like night and day, the difference between the two, where when you join Liverpool as a player or Manchester City, you have to follow the Guardiola system or the Klopp system, and he structures and forces the team to a play certain way. Whereas other teams, they sem to get these star players and the coach doesn’t have either the confidence of the players enough or the systems way of thinking to adapt to the team. 

John List (01:33:36): 

No, that’s a great point. This last weekend, that Man City game was unbelievable, wasn’t it? 

Bilal Hafeez (01:33:43): 

Yeah. 

John List (01:33:43): 

I’m a Liverpool fan, so I like Salah, but the Man City game was unbelievable when they tied it. 

Bilal Hafeez (01:33:50): 

Exactly, yeah. What Conte did, I guess was he worked out the weakness of the Manchester City possession style of play, where you’re basically going to get two or three chances in the whole match and you just have to convert, and he had Harry Kane, who really played really well. 

John List (01:34:05): 

Yeah. Kane played great. I was actually really surprised that Man U picked up Ronaldo, right? And then they bring in the new coach, which doesn’t get long well of course with Ronaldo. 

Bilal Hafeez (01:34:14): 

Yeah, because Rangnick is more of a systems guy. He doesn’t like big personalities, and so that’s going to be chalk and cheese, and Ronaldo, I think to some extent he distorts the team as well, because he’s such a powerful player, that when he comes into your team, it’s like a sun turning up and the gravitational pull, everything distorts around him. 

John List (01:34:32): 

That’s tough. Who’s your team? 

Bilal Hafeez (01:34:34): 

Liverpool. 

John List (01:34:34): 

Oh, you’re like Liverpool too. Great. 

Bilal Hafeez (01:34:36): 

Yeah. I love Liverpool. I’ve been supporting Liverpool since I was a kid. It was a tough 1990s and 2000s, where they didn’t win anything. But recently, it’s been great. 

John List (01:34:44): 

They’ve gotten hot lately, haven’t they? I mean, got a shot now, right? There’s going to be a few big games coming up. I think there’s one more against Man City, right? 

Bilal Hafeez (01:34:51): 

Yeah, one more against Manchester City, so that’s a big one. If Liverpool win that, then there’ll be level on points, assuming they’ve won all their matches in the run up to it all. But it’s just amazing. I mean, there are some complaints amongst football fans in the UK that why is it there’s only one or two teams that just run away with the Premiership for the last five or six years? And so it’s less competitive, but I think it goes back to, I mean, it just shows you the difference that good coaches make. Manchester United had it with Alex Ferguson. He worked with probably three distinctive different squads and won everything, turned over the team three times, which is amazing. I mean, that makes him one of the best managers of all time. To be able to do that is quite remarkable. Now we have Klopp and Guardiola. 

John List (01:35:27): 

Now, it’s funny when you started talking about soccer, you said the two biggest teams in Europe are, you said Man City and Liverpool. So you totally forget PSG. How come? Because they’re pretty good too, right? 

Bilal Hafeez (01:35:39): 

Yeah. Well, I mean the thing is these Spanish teams have really declined in recent years. Real Madrid and Barcelona, they’ve got debt problems, they’ve lost Messi, they lost Ronaldo. They’re not doing so well. Paris Saint-Germain are a star-studded team, but they never win anything in Europe. They’ve never won the European Champions League, and they haven’t got to the final. They haven’t won anything for years and they have a very easy league, domestic league. 

John List (01:36:01): 

What about the German team, then? They’re pretty good, Bayern? 

Bilal Hafeez (01:36:04): 

Yeah, Bayern Munich is good. But I mean, if you look at just betting markets, it’s always Manchester City and Liverpool that are top of all Champions League everything. It’s not Bayern Munich even. I mean, there might be an issue now where I’m not sure who’s going to meet who, which might affect the betting odds. 

John List (01:36:18): 

Right. They might be on the same side of the bracket. 

Bilal Hafeez (01:36:21): 

Yeah, but like ahead of the tournament, it’s Liverpool, Manchester City, or Chelsea even was ahead of Bayern Munich. I mean, the issue with Chelsea is that Lukaku hasn’t been played too well, but it’s unclear whether that’s a coach problem or a Lukaku problem, because he was unplayable previously. But since he joined Chelsea, he’s stranded by himself, the ball doesn’t get to him enough, and it’s like a wasted resource. Tuchel, who used to manage PSG, now manages Chelsea, and doesn’t seem to be able to know how to use him. 

John List (01:36:47): 

Oh, interesting. Yeah, they’ve been having some trouble lately in Premier League, haven’t they? 

Bilal Hafeez (01:36:51): 

Yeah. I mean, they were up there, they were challenging Manchester City, then suddenly they’ve lost form all of a sudden and they’ve been dropping points. And now the interesting thing in the Premiership is who’s going to come third, fourth, fifth? That area is super competitive. 

John List (01:37:03): 

Because top four qualify for some tournament, right? 

Bilal Hafeez (01:37:06): 

Yeah, for Champions League. That’s where the money is. So as a top team, you always want to qualify for the Champions League, so you have to come top four. Liverpool and Manchester City are probably going to get one and two, that’s almost certain, but I think there’s eight teams that could potentially come fourth. 

John List (01:37:19): 

Yeah. Chelsea, Man U, Tottenham. 

Bilal Hafeez (01:37:22): 

Tottenham, yeah. I think West Ham even, and then Arsenal can potentially, there’s a whole cluster of them. So, that’s where the fun is in the Premiership right now. So we’ll see which one of those can make it. 

John List (01:37:33): 

I guess the fun should be at the bottom too, right? Because don’t they get relegated? 

Bilal Hafeez (01:37:36): 

Yeah. They do get relegated. 

John List (01:37:37): 

But nobody talks about that, really? 

Bilal Hafeez (01:37:39): 

No one talks about them, yeah. I mean, there’s a couple of teams at bottom who are probably going to get relegated already, just how badly they’re playing. 

John List (01:37:46): 

Yeah, fair enough. 

Bilal Hafeez (01:37:47): 

So how come you’re into soccer? I mean as an American. 

John List (01:37:50): 

Oh, I like I say, I like to learn about everything. I like to learn about everything. I have a good friend named Robert Metcalfe who is from Wales, and his team, Swansea, a few years ago they had a nice run. Rob was my postdoc and now he’s a Professor at University of Southern California. So he does field experiments and works with the firm, so he’s a nice protege of mine. He’s gotten me more and more into soccer. 

Bilal Hafeez (01:38:16): 

Lots of Americans find soccer a bit boring, but I find it really exciting. 

John List (01:38:21): 

If the play is good. So there’s some sports that you can watch regardless of the quality, but for me, soccer, if the play isn’t top notch, I become really disinterested. My kids grew up all playing soccer, so I’ve watched a lot of really bad soccer and it turns into a kickball game. And unless you’re playing kickball, it’s not very fun, but the top ones I think, artistic, and it’s beautiful to watch. Where the lower ones, just because I wasn’t raised with it, I was raised with football, basketball and baseball and golf, of course. Well, I did golf a little bit, so that’s the difference for me. On Sunday morning, I love to have a game on in the background while I’m working, and it just so happened that this last week was the Man City, Tottenham game. 

Bilal Hafeez (01:39:07): 

Yeah. That was a good match to watch. That was so fun. It was a really gripping match. 

John List (01:39:11): 

It was bad that I killed my productivity, but it was good that I can talk about it now and I can socialise. So I’m not a total… 

Bilal Hafeez (01:39:18): 

You think like an economist in every interaction. 

John List (01:39:21): 

Exactly. So after it, I said, “Well, that cost me an abstract.” Yeah, I think of everything in terms, a lot of times people think about, when I’m in the business world, people think about the opportunity cost is money. But for me, my currency tends to be academic papers, so I watched the end of that game, especially like I said, at the end, it really did cost me a good abstract. 

Bilal Hafeez (01:39:42): 

No, that’s true. I mean, I’ve never really got into American sports. I’ve watched American football, but it seems quite technical, like a stop start, and I don’t understand it. It’s not fluid, and so when I watch it’s like, “Why do they keep stopping every 10, or whatever yards?” It’s like, “What’s happened there? Just carry on.” 

John List (01:39:58): 

Exactly. American football, at its roots, has a really big flaw, and the big flaw is there are so many little rules and things that happen that it makes it really hard to learn it without investing a lot of time. So that’s a deep flaw. It’s like the off side rule in soccer. We have a hundred of those in football. So it’s like, gosh, by the time you have a hundred of, “What’s this? What’s that?” And so then you can ask, “Well, why is it so popular then?” It’s the most popular sport, it’s because of fantasy football. 

What saved football was everyone plays fantasy, and it’s so easy to play fantasy because it’s how many touchdowns does my guy get? How many touchdowns does my quarterback throw? How many yards does my running back run for? How many yards does my receiver have in reception? And then it’s like, bang, bang, bang. That’s super easy to understand. Everyone understands fantasy football and the scoring of it, so they watch it and then it becomes a big thing. But if you had to really take away, strip away fantasy football, that sport would be nothing. 

Bilal Hafeez (01:41:10): 

Yeah, yeah. So what do you make of rugby then, when you watch rugby? 

John List (01:41:13): 

Same thing. It’s like football. My son plays rugby at Harvard, and I took him to a rugby game in Australia because I’m a professor at Australia National University, where it’s like real rugby players there, and in the UK, and it’s the same thing. It’s not as detailed or rule oriented, but it has some of the same features, but it doesn’t have fantasy, so it’s never going to be popular. 

Bilal Hafeez (01:41:36): 

Yeah. I mean, I find rugby a bit more fluid than American football, but again, there’s still a lot of the stop starts. I mean, now they have this thing where you can hear the referee talking and explaining why he stopped the play, but you do get a lot of that. And then, is it a maul, or a ruck, or this? I mean, it gets very technical. 

John List (01:41:52): 

Yeah. So I’ve been getting into it, of course, because of my son, Noah. I went up for a few games at Harvard and it’s fun, but it’s really niche in a way, and it’s kind of brutal too. Two or three guys go off every game and you don’t know what’s going to happen to them. Look, American football’s the same way. It’s just too brutal. 

Bilal Hafeez (01:42:10): 

Yeah. No, I agree. 

Thanks for listening to the episode. Please subscribe to the podcast show on Apple, Spotify or wherever you listen to podcasts, and leave a five star rating and a nice comment and let other people know about the show too. We’d be super grateful. Also, sign up to become a member of Macro Hive at macrohive.com. We’ll be back soon, so tune in then. 

Bilal Hafeez is the CEO and Editor of Macro Hive. He spent over twenty years doing research at big banks – JPMorgan, Deutsche Bank, and Nomura, where he had various “Global Head” roles and did FX, rates and cross-markets research.

(The commentary contained in the above article does not constitute an offer or a solicitation, or a recommendation to implement or liquidate an investment or to carry out any other transaction. It should not be used as a basis for any investment decision or other decision. Any investment decision should be based on appropriate professional advice specific to your needs.)