Richard Hahn – COVID-19 Interview

Welcome to policy, Emma. A data focused conversation on tradeoffs.

I’m Karla’s car value from the Save Them Center for Policy at the University of Texas at Austin.
So we have with us today Professor of Statistics, Richard Horne from Arizona State University.
And there’s going to be a little bit of a different conversation, because Richard not only is a long time collaborator of mine,
but also a friend. So perhaps this conversation to be a little bit more casual then than usual, but hopefully can
fetch a lot on issues that come of it in statistical terms. So,
Richard, thanks for being with us. Thanks. It’s going to be hot. So the way I’m trying to
have this conversation is to try to go back to what we know when, you know, end the beginning at first,
try to remember, recall how we’re thinking and how we’re judging the evidence that was in front of
us then. So first, when did you start to get like, huh? There’s something
important going on here. What was the first sort of like piece of information that that got your attention? So
for me, it was in pieces because I work at Arizona State University
and we had a student from China who got one of the earlier cases in
the US. So I’m thinking that was in January, maybe January or February.
And and it made kind of big news. But at that time, this was when.
If you go and look at the timestamps of your articles online, this was back when The New York Times
was saying it was no worse than the seasonal flu. Right. And so
we didn’t take it too seriously. But it was kind of interesting. And all of my Chinese graduate
students were wearing masks. So they sort of like early on were like had been
talking to people at home. They knew they knew something. And so that I thought was weird. The biggest
thing for me was that my classroom. I started seeing students with masks. And then there was a student.
But what happened was that student got better. They did, you know, tracing or whatever. They found
his contacts. They put them in quarantine. He was a young guy. He recovered. And I was kind
of the end of it. And then it started hitting the US. I had a trip planned
to Brazil in March and mid-March, I think 13.
No, I must left on the 9th, something like somewhere around them. And
my biggest concern. So I went. So that tells you what my personal risk was. I didn’t think it was a big deal. I
wasn’t scared of getting sick. I wasn’t scared of being a vector. I wasn’t going to get my family sick. At that time. I
was my biggest fear was they were talking about closing down some airports. And so I was really worried that I might
get stuck in Houston, which is where my connecting flight was. So anyway, this is just my boring personal story. But
the idea was I didn’t have models. I wasn’t worried when I was in Brazil.
The Italians didn’t show up to the conference. And I was like, oh, OK. Because because all.
But not because they were sick, but because all air travel had been canceled. But then a lot changed. Right. By the time
I got back on the story on March 13th. It was a whole
a whole different world. We talked that day. We’re like, what’s going on? I
think 48 hours after that, California had their shelter in place, order
in place. So I was bewildered. I mean, it’s funny. So not only am I the less
the least illustrious guest that you’re speaking to, but I’m also probably the most clueless one. This is just going to be like, what?
What an average do to it? Yeah, I wasn’t scared at all. And then I was like, oh, this
is weird. I didn’t really get engaged. And and sort of intellectually
until the Ferguson paper, Ferguson at all, which was like a circa March
And in particular I had a good friend of ours. The onhe was at
the conference in Brazil. She lives in London and we were talking at the time about how London
has been at war. St. Peter College did. Did. Right. Yeah.
No. I don’t want to be quoted on that. And so
I was we were saying like how great it is that London is keeping schools open. So at that time, I adamantly
believe the school should be open. And then that paper came out. And
Boris Johnson changed his mind overnight. There was a paper saying just that. Everybody,
Ray. So. So this paper this paper basically said everyone’s
going to die unless we shut the world down. I don’t even remember the numbers. I remember
reading it and thinking, well, this is obvious hyperbole. So he said
if we don’t do anything in the US, 2.2 million people are going to die. Two point eighty eighty
percent of the people gonna get it. And it 2.2 percent of people are going to die by the summer. By July,
I think was the number and and five hundred thousand people in the UK. That was
the number they had. Yeah. So if somebody gives you numbers like that and you think that they might be right.
You start making different decisions. You start closing things down. You start hunkering
down with your family and so forth and so on. I
never liked that analysis from the from the get go.
What about it in particular? And the thing I didn’t like about it was that their decision making was clearly
pessimistic. Right. Statisticians or economists might call it Mini Max. They were worried about.
They were worried about the worst possible thing that could happen. And that’s generally not a great way to make decisions.
A classic example is driving and we drive all the time. But if we had a
worst case attitude about the driving risks, none of us would ever be able to drive a car. Worst case,
get in the car again. Because the worst case is that you end up in a head on collision with a truck and
it’s over. But on average, it doesn’t happen. No, not on average and not
very often. Right. So it’s important when dealing with decision making under uncertainty that you figure
out what is your criteria. What is your criteria for evaluating this? Do I want
it to be. Go on, be right on average? Or am I going to have a lot of regret if I make a decision,
wrong decision theorist study this stuff. By and large, that’s
not how policymaking is done in policymaking in the real world as lots of stakeholders. And
it’s a lot more based on gut gut reactions, I guess, and it can go both ways. Sometimes people are
too risky. I think sometimes they’re none of that. This case I think was very unique
in that a scientific body wrote this report.
In a very aggressive way. To make their point
now, if they had been right and this had been Ebola level deadly or something like that. And plus the
contagiousness, the Navy would be saying that they were heroes because they sort of overstated
things and saved lives. But frankly, the science wasn’t there. I mean,
by phrasing it in terms of the worst case, they were making a value judgment in terms of their
emphasis. Right. And when you say the science wasn’t there. So I think there was a paper that came
around the same time by another group of experts in
Oxford, I believe. And what did that paper say? So
that that paper came out a bit like a week later or maybe less. And
it basically said it said the opposite. Or at least that’s how it was reported in the press.
So I want to talk about both of those things because I think they’re kind of interesting. So what did the press say? What did the paper
said? So this paper came out and it basically said, look, it could possibly be the case
that actually way more people are already infected than we thought. And if that’s the case, that
the denominator of the number of people that have this is much higher than we thought. Meaning
that the death rate or the infection fatality rate is a lot lower than we thought.
In which case maybe one, we’re closer to immunity than we thought already because more people have it.
And also, it’s not as scary as we thought it was. So that
paper that paper said this is a possibility. And again, it’s a matter
of emphasis, because the fact of the matter is, neither of the papers knew for
sure. Nor is it knowable. So the technical idea for this. Think I think this is why
you got here to talk about this is there’s this idea called identify ability or identification. And
it basically says based on the data that’s available to you. Can you
rule out certain stories or explanations to the exclusion of all others?
Sorry. Can you can you or can you say that this is the right one for the people here? Is this set
of plausible things? Well, so that’s the thing. Like, generally speaking, you can’t so like if you’re doing physics,
if you’re doing mechanics or something like that, you can do an experiment. And on the basis of that, you can rule
out theory. And you can therefore adopt a theory or something like that. More complicated
problems in social sciences and epidemiology. What will you do instead as you rule
things out, you say this data is not consistent with these possibilities, but it is consistent. So
this is what I mean, the data that we had available at that time, it was a mixed bag of stuff coming out of
Suhan and Italy and early cases in London and stuff like that.
Not good data. I mean, it had all sorts of problems, not a lot of it. And the tests were the
test or a perfect. In fact, the tests are still not great. And because of all of
these factors, there were not able to rule out lots of interesting possibilities. And so
I think that they think that the scientists both knew this, but the two
papers basically put a totally different slant on it. The one paper said we can’t rule
out the possibility that 2.2 million people are will die if we don’t think so. And that number
was based on it could be the case that very few people are infected, but it is highly infectious
and then it kills 3 percent of people. Yes. The other paper said, well, it’s also concerned the data
that we have available is also consistent with the case where many more people are infected. And it’s not as tough
as we thought. So forth and so on. And so, you know, in a sober world, what we would like
is that the media didn’t pick up on either paper, didn’t put them in juxtaposition. The scientists
wouldn’t talk about it. You know, it wouldn’t have sort of foregrounded one over the other.
Yeah. So the fact is the data did not resolve things.
And so this is this has been a real case study that people think of science as being a collection of facts.
But really science is a method for ruling out scenarios.
And in this particular case, the science method did not allow us to rule out
enough scenarios so that the decision making was obvious. And when that happens, you
have to have to guess. Well, you have to guess. But then you have to think about what gives them
the other side. Right. So so when we talk a lot about tradeoffs in our discussions in our group here,
the classes that this made is gonna be shown, too. So here we are looking at
a few different scenarios, a possibility associate with a progression of disease. There’s one path that is compatible,
the data we’ve seen so far that would lead to something like two men in destiny. There’s another path.
That’s a path that’s more like mile that maybe leads to two hundred thousand. That’s us now
have to decide what to do. We can go in and say we’re going to lock down our economy, going to shut everybody
house arrest, or we’re gonna do what a country like, let’s say Sweden did, which is say
we’re not gonna do it. We’re going to just tell people to be careful, you know, close some obvious things and then see how
this thing goes. So, you know, there’s possibilities and there’s real costs
in one. If I’m wrong, go one direction. But also real cost if I’m wrong and the other direction. Yeah. So one of the things that got lost
in all of this. So I agree broadly with what you said. I don’t know if I’d use the word house arrest. It’s a little loaded.
Not in the U.S., but in places like Italy. You were in house arrest. Yeah. Anyway.
Yeah. So. So typically in the in the academic scholarly literature on decision
making under uncertainty, there’s a big component played by the utility or the costs.
And those play a strong role. And ideally, we would be balancing those. And you can’t
do it. You can’t do it exactly right. Let’s write the rules. But you want to at least be aware of it. And then,
you know, you and I have been collecting articles. There were people that were sort of glaring this horn
saying, hey, look, there’s costs. Economic costs, unemployment
and stuff like that. But I guess it’s just it’s hard.
And what I noticed, the thing that I personally was turned
off by what I saw in policymaking was that people
were not being open about the trade offs and about
the interpretation. So it wasn’t as if the
ICL paper came out. I see it as I said I was. I get a bricolage.
Puracal, isn’t it right? I see. I do not UCL University College on it. Let’s do this. Why? So
I’m a pro. Right. So when that paper came out, it wasn’t. It wasn’t like
they were like, well, this this, this, this. And worst case scenario and there’s
these other costs, like it was a kind of a long report. But the tenor of it for sure
was like we should do this thing. And they really never backed off.
So it makes it hard to make decision making because the people downstream from that hear one story
and they don’t know what to do with it. I guess what I’m trying to say is
I think people have their own incentives for the way that they interpret this stuff and
they they do not necessarily explain themselves. So you you know,
you’re tiptoeing around this a bit, but I can say it more openly that that
our biases when we scientists, you know, generally speaking, you and I are scientists
and enough in Alaska. And when we write something, it’s really
hard not to let your your beliefs come in and say, oh, here’s what I think is
right or wrong with his analysis or so. Right. It’s really hard. And we have a process of peer reviewing
that oftentimes is very frustrating. It sort of tries to mitigate that a bit. And, you know, it’s best
done in its best format does that. It takes away some things that maybe I think that’s not a generic
genetic enough claim to be made worth mentioning. Right. That neither of these two papers we’re talking about were peer reviewed
and they were very rushed. The one paper was from a government panel.
So in that sense, it was a large group of people and they were sort of enlisted to do this
type of work. The other group, I think it was a response paper because they saw the ICL paper
and they were like, wait a sec. Well, let me get those papers were not great scientifically.
I mean, I’m willing to go on the record and say that when I saw those models and when I saw what they had done,
how they adapt with the uncertainty. I was not overly impressed. It looked tasty.
And it was you know, there was well documented. The code was not readily
available in all the sort of standards that you typically hold for scholarly work.
We’re sort of not met in either paper, which again, in some ways it is also is also understandable
given that the it was rushed to be produced and now that. Right. So
yeah, I don’t I mean, this is not coming across as a clear narrative. You were just asking me to talk about
the sort of issues that are defined. But what I think is what I do, what I want what I want to say is that is that
you mentioned is that people like to think about science and scientists as being somehow these
arbiters like this, people in wearing robes, you know, are calling balls and
strikes without any kind of. And every time you read a paper by an economist or by
me or by you or by them geologist, their preferred path is
there in one way or the other. And I take that into account, because when a governor of a state
as big as California says, the decisions are going to be dictated by science. It’s
him not doing his job because science cannot tell him what to do. Yes.
Yeah. Yeah. I mean, I not thought about that. And I certainly don’t want to be I’m not on the bandwagon of like a scientist
subjective. There’s certainly no scientist that’s objective. But but it has to be.
Let’s go this way. Science done well does eventually lead to facts. I mean,
I’m not a philosopher by trade like I’m a student of philosophy more than I am a philosopher. But, you know, they
call this up. What’s the word? I swear that they use truth filmic. Like the idea is like the method
you wanted to be leading to the tricks. And we think that it does. And that’s why the scientific method is so
great. Prior to arriving there, you
get to these places where you just can’t tell. And that’s especially true in fields like epidemiology.
So, yeah, I don’t know like that. I agree that the governors are, you know, the people in charge.
Saying we’re going to let that. We’re going to follow the scientists. Yeah, it’s
weird. You’re going to follow which scientists write, if they follow the Oxford scientists, they would have made one decision. And if they’re
gonna follow the ICL scientists, they’re going to do another one. In a way, it seems like just a way to take cover. You
pick a scientist that’s willing to say they think you want to do it and then you do
it. Scientists are people, right? It’s not like they don’t have agendas, incidentally. And I
don’t know Professor Ferguson personally, but I read that article by
Matt Ridley, who pointed out that he’s made a career out of saying
that the next pandemic is going to be the big one. He did it with mad
cow, did it with swine flu. It with Ebola. With Ebola. So so he has a
track record of like this. Clearly, his world view now as an economist and
you’re more an economist than I am getting our station. But like we talk about incepted, you know,
kind of stuck with incentives a lot. Somebody said to me, you know, epidemiologists don’t get
in trouble with their pessimistic. If an epidemiologist says everybody is going to die, we should
shut down and then nobody dies. Everybody is just happy that nobody died. And because what
if the epidemiologist says, oh, it’s going to be cool. Not a big deal. And then everybody is a dying,
you know, that epidemiologist is on the hook. So I think it’s pretty interesting,
though, that you’re going to advice from a group of people who have every professional incentive
essentially to say that things are worse than they are. You know, it’s not just
time. It may have changed this time because I’ll tell you so. So it’s not big
like. We did something that you heard a large amount of defense. I mean, we
were still going to spend years trying to debate whether, you know, was what was the cause of a fact here,
of what if we hadn’t shut down? What if. You know, and and relative to that,
we’ve seen that already. That’s actually a lot of variability happened to us. Right. All states, they’re still closed.
They’re open. We’re gonna actually learned a lot from from from experiments that we run.
All right. So that was done anyway. So so recap. First point, those are these papers. Those are these
two hook sort of high-Profile papers, these pretty similar models. And those models were not
able to do everything we wanted them to do. And that’s right. That’s just
completely plausible within those models. A battle of a radically different. Radically different.
OK. So. So for people that are familiar with thinking this way, let me give an example that I worked on. So it’s sort
of like factoring in. No. OK, so five times six is thirty. Right.
So let’s say I tell you that it’s 30. You don’t know if it came from five times six
or 10 times three. Those are very different things, right? So
so let’s say that, you know, that the revenue was 30. You don’t know if you sold 10
units at $3 of $3 or 5 units at $6,
etc., etc. And that’s it. That’s all it comes down to. You’ve got some data and then you’ve got an explanation
for how that data came up and sometimes you’re not able to uniquely resolve it. If the number happened to be a prime number.
That’s right, Tony. You can do it right. But you can’t tell us,
so I saw that if the first issue that nobody talked about, then fast
forward a bit. And now we are here in June 5th, 2020. And
what do we learn now and how do we think about those models and those predictions based on what we now now have
more information about? So let’s talk about back testing. This
is going to place to do that. Is any back testing is a term that I first heard.
We’re talking about financial markets. So let’s say that you’re building a stock picking algorithm and you
want to know if it works. So what would you do? Well, you could take the past data
and then you would look at it and you try to predict the future date. Right. That’s that’s what this thing does.
So what you could do is you could pretend that you collected the data in the 70s
and then used it in the 80s. Sitting in 2005. Right. So you get
this why they call it back testing you basically you use your your algorithm, you place it into the past
and then run it for it. It’s a powerful method because it tells you if the patterns that you’re seeing would have
persisted. It’s just it’s a smart thing to do. We can do that with these epidemiologists.
Right. So everybody was trying to do forward prediction and they were taking those
four predictions and they were saying this is what we should do now
that we’re here. It would be really interesting to go and compare what those predictions
were to what happened. Now it’s complicated. And the reason
it’s complicated is because there’s an extra degree of freedoms, which is the fact that we did things like
crackdowns. Right. The degree is still the degree of social distance is a enormous
degree of freedom. So, again, we’re in this case where we’re unidentified again.
So why is that? So they have predicted that blahblahblah would have happened. The
curve would have gone like this. Right. And then it didn’t. The curve kind of went like this.
The question is, did the curve grow like that because the world wasn’t as bad as we thought it was? The
disease wasn’t as bad in some ways. Or did that happen? Because we were all very,
very good boys and girls. Stay inside talking. It’s probably some of each.
Right. Nonetheless, that’s the sort of analysis I’m excited about going forward. I think
we have enough data to look at it now. And in particular, you’ll be able to say things like. Let’s assume
that we had very high levels of social distancing. If the disease were
as bad as we thought, would it have still been? Would it have been slow? You know, I think the answer we’re starting to see is no
light coming in from various quarters, sort of. It doesn’t seem like
it’s as deadly as it was. And it seems like they’re contagious. Contagion
mechanism is distinct from these models thought it was, which was that everybody is equally likely to.
I mean, that’s one of the implicit assumptions, right. Some of these models now
that is anyway that there’s a particular in particular that what we learn is when you say not badly, we’re talking
maybe an order of magnitude less deadly than people thought at first. Yeah. So. So it’s it’s a very
peculiar to my understanding. I am not a medical doctor, obviously.
But that said, from what I’ve been reading, this is a disease that that hits
older people harder than the flu. For various reasons. On the other
hand, it actually seems to be less bad than the few for younger people,
and that gives you a good. What happens because of that is that if you aggregate up over age group, you actually get a
section fatality rate. That’s about five times higher. It’s
getting to be less than that. Well, know, last I added more information. Yeah, it’s been follow it too closely.
But. Overall, it seems to be from a couple of places that I’ve looked and unfortunately
I don’t have references. But when you do the age breakdown below 50,
it seems to be the case that it’s actually not as bad as the flu. Right. Which is one reason
that I’m upset about the significantly larger. But above 50, significantly worse than
the flu. Yes, yes, significantly. So, yes, it’s.
And then the super spreader thing. Apparently, it appears that not everybody. You know, there’s some weird mysteries
that kids don’t seem to give it to their parents or to each other. In addition to not being symptomatic.
Some people seem to be super spreaders, meaning both that they come into contact with a lot of people, but also that they have
a high degree of viral load and sneeze a lot or something. I don’t
know, adult all the mechanisms are, but it changes every day. But the point is that has a lot
of that. It seems that a lot of heterogeneity there and not something that is homogeneous as they use it as a lot of those models.
Yeah. So the models basically assume that everybody there’s some fraction, you know, it’s based on averages.
These simple models are essentially estimate an average and then assume that the average
holds for everybody. So everybody in the model is equally likely to die of it. Everybody equally likely
to get it in the simplest models. All right. That’s what it looks like.
So what we’re finding is not true. I don’t know, I like like
I guess the tenor of I am not going to come down and tell people that they should not be scared of this.
I just don’t feel comfortable doing that. But I do feel comfortable saying is that I’m not scared about.
Just my smites synthesis of the information. You know, sort of informally. Are you willing to
take it or are you? I will. We want to put your children back in school. They go back on June
Yes. So I don’t think I’m going to be doing a lot like. I’m a little bit. A little bit. Do you
want to go to a crowded bar? I wouldn’t go to a bar anyway. Yeah. So sporting events?
Probably not. You know, a large, large, dense public transportation.
Probably not. We’ll just say that from a personal risk aspect or from like I don’t wanna be a vector.
Because if I go to a crowded bar or to a sporting event, I’m being a vector. I’m more
likely to become a vector when I help. That is right. I think both. I think
both. I don’t I don’t consider this this kind of an interesting point. I don’t consider those two things
as distinct as some people. So this came up with with the discussion of masks. So people were talking about art
and the CDC screwed up. Right. Because in the sense that they was hold and they weren’t clear there when the W.H.O.
Yes. Sort of across the board. The rationale for masks all
along. If you’ve been paying attention to the way it used for decades in Asian countries is that
it’s a courtesy gestures to prevent you from getting other people sick. All of the scientific
studies on this have been about does it protect you? And
I think the CDC rationale early on all that they can never say this and I don’t think they ever came out said this
was that. They wanted to keep the personal protective equipment
for the frontline workers so that they didn’t get sick, which is a totally reasonable thing. But rather than saying that apparently
there was this paternalistic response where they just told people the masks aren’t effective.
Don’t use them because we need to save them for the. I mean, it just on the face of it, the logic of it doesn’t
make sense. Don’t use the masks. They don’t work because we need them for the
doctors. Like if they don’t like what? So there’s two debate. You know, this is a persistent confusion.
Two different things that the mask do. Right. So the question is, when you go to a grocery store, should you wear
a mask? And this gets to your question of, well, is it because you’re scared of infection? If it’s because you’re scared
of infection, the mask doesn’t help. But it kind of obviously helps
you because it prevents you from sneezing on things. Right. Right. So if you’re the vector.
So that’s kind of an interesting. You know, these are the sort of subtleties that. Have become
commonplace in life now, but people are bad at it. People are bad at passing this sort of thing.
And it makes everybody anxious and then they fight over the masks. And then on top of that, there’s misinformation. People saying
that the mask. Make you sick. Right. And he comes in and becomes a political
statement. And it’s a mess. Anyway, going back to the point, I think
about it as a community. And so I think about getting my kids teacher sick.
So it’s I mean, it’s sort of self-interested in that way. It’s like I don’t have this very abstract notion of being elected.
But but I do think about. I don’t want to get the bus drivers sick. Who then gets the teacher sick, who then gets
my kid, you know? So I think that’s the right attitude. We’re kind of in this together
and we should just do what we need to do to keep people safe. But
that doesn’t extend to question. One question I’m asking pretty much everybody that
I’m talking to is that what you’re talking about is that there’s an externality where there’s a cost. That’s not to you
directly. Personally, that’s B associate. So when you go out, you might have be facing
your personal risk of deciding to go out to a bar or whatever. But that cost is
is you can decide to mitigate your cost, the weight that the cost of you getting sick, whatever you want, but
by you being out. That increases the transmission rate of disease out there. And that’s a cost to others that you
are Detroit creating. So we don’t internalize that would call that economics. We can talk about that in this class and externality.
Right. Interesting thing about this problem is that is very different than a typical example of action now
and pollution. I sent some stuff in the air. The air is polluted, not costing me anything directly.
But everybody else, I’m not paying the price for it. So, you know, we need to somehow create a mechanism
to fix that. Right. But unlike pollution, the people there are really at risk here
have a choice of not going out. So, you know, it’s not like I have a choice of not reading
the air. Right. So in the series. Are you talking to captain? We will later. Yes.
OK. OK. Captain had a nice comment about this. But but. But here’s a part of the action now that I think. So,
you know, I think that I’ve heard that part of the discussion and some people elaborate very clearly on that. Well, one thing
that I’ve not heard yet is the fact that we also use vaccines. And again, I will talk about vaccines,
not GLAAD’s, as an example of a positive action ality. By these vaccinating, I become
now immune person in a system that therefore slows down the progression of the disease. Right.
By the same logic, my kid. That is not at risk. Cap. Getting the disease
and becoming immune is actually a generator of a positive externality. So the action,
Ali, there’s a third there’s a phase transition between when it’s negative versus when it’s possible in
a world in which I could isolate all the people, they are really at risk and let everybody else hang
hang out to mingle. We’ll get to herd immunity and then they can come out. It ended
now that people can come out. This was this this was the Sweden plan. Right. But they did a bad job at
step one. Step one, which is hard. It might be impossible. Right. It might be impossible. That’s why I did a bad
job at step 1, which is that their nursing homes got super sick, but everyone stayed. Right. I mean, nobody
took a job. It turns out to be very hard not to. And if you didn’t know if the things were
asymptomatic in the beginning, nobody knew and is already getting in. So, you know, it might be too late by the time we figure out it might
have been too late. Right. We could have avoided sending Kovic positive patients
back into nursing homes. Like a lot of states. That’s different. Yes.
Yeah, I think New York, New York and of course, thousand three hundred patients positive were sent back to nursing
homes, which again, I don’t know what it what about. I mean,
it’s funny because the area in which I have any professional expertise is in is in this
this idea of quantifying uncertainty or trying to do in some sense, there’s
nothing much to say. Like like there was a lot that we said already. I think some of the decision making part,
I think. I think you use a sentence before. To me that I think is interesting is that I think a lot of the ways people
are talking about it is they’re confusing the evidence with our values. Yeah. Right. And
I think in decision making, we have the evidence that has a lot of uncertainty. And then we’ll have to put that through a utility
function that says, OK, now let’s make a decision. And that’s what you’ll find is common. And it’s totally fine. You might have the
same evidence. You and I might get to a different decision. That’s a big component of what we’d like to emphasize in this
glass. Right. So I need to get on to get on a soapbox here. I think this is super important for
people these days in particular, because, you know, there’s this
phrase of like, you know, post truth world or post facts
or whatever. And I think that this is that problem. When you’ve got scientists putting out papers
and the papers are written in such a way that the value, their value. It makes
it seem like the facts are up here. And to some extent, some facts are up in the air, but
not not. But they don’t disagree with each other. Like it like a proper reading of those two
papers is that they said the exact same thing. And that one group emphasized
one thing and the other group emphasized another thing. So at least at the end of that interchange, like the
correct way to look at that is they analyzed data. They found a range of possibilities
that were the truth. We don’t know which one it was. And then on top of that, you layer on the values.
Right. But at least in that context, people aren’t pointing fingers at each other saying you’re making up facts.
Right. Furthermore, it means that you can have cases that are not ambiguous.
The polio vaccine is an amazing vaccine and it works. And to the best
of everybody’s knowledge, it doesn’t cause autism. You know, the idea is there should be
and we can have this like as long as you it like if people were
open about when they didn’t know they would be more trustworthy when they
said they didn’t know. Right. That’s true. But if scientists go around saying
every single time. I know, I know, I know, I know. Because science and then they’re wrong
nobody’s gonna believe them. And I have a huge fear here with epidemiology
that’s happening is that, you know, they have a role to play whenever the play comes back.
Right. We’ll get to that. You guys that are you guys are at it again saying
those things. They’re not going to happen in this book, which so that
I mean, on the on the other hand, you want it, you don’t want a broken clock is right twice a day. So if
they keep out, Ferguson is going to be right. Maybe it’s time, maybe not in his lifetime, but
he’s actually going to write eventually. Know not for the right reasons, but. Right, right.
Yeah. It’s crazy. So one question I’m asking a lot of people is a question of,
yes, we all got into somehow for one way for one reason or the other. I don’t know. Maybe the Ferguson paper had a huge
influence on this. The majority of the Western world. Let’s focus on the Western world, the
developed countries, the rich countries in the world. All of them except one decided to do the
same thing. And, you know, as we spend a lot of time talking
about here, it seems that he was not clear from the evidence that that was the path forward. It was not clear from the
tradeoff evaluation that was there, right? That’s right. So I think if we have any idea of why,
a lot of thoughts on. So I think there are two interesting points here. So the first one,
which is not super controversial and it’s not really an opinion, is that in cases of great uncertainty,
if you’re interested in the global welfare, it is actually beneficial that
people do different things, because if they do different things, you learn
from the places that did it wrong and you learn from places that did it right. And then you adapt.
And then in the sort of medium term, everybody is better off. So it’s the term of
that in our science, right? Explore, exploit. Or
MOTY Army Bandit’s, you know. I mean, so this idea is like if you don’t know and you want
to learn, you cannot learn if everybody in the world does the same exact
thing. All that’s gonna happen then is everyone’s going to have roughly the same outcomes or not. Yeah. And then
it’s very confusing. Whereas if everybody had done sort of different things,
we would learn faster about it and we could have had better policies in months, too. I mean, hell, we’re on. But
three of the four and a half. You know, if everybody had been doing slightly different things, maybe we
could have this lot figured out by now. I would argue that that me, because of an example
and by you, kind of can rule out certain things by what we saw happening. So we did.
Well, I think that was right on last is a different virus. I think what happened swedens shows us that
by not locking down, by trusting the citizens to do reasonable things, they can
avoid an exponential growth of disease. Yes. And they did not have
a exact. That statement is uncontroversial. Whether that’s going to lead to a higher death rate after
the whole epidemic is over, not I don’t know. I cannot argue that. But I can argue for sure that we are able to avoid a
exponential growth of disease by just crossing the city. So anyway, so that’s that’s the one angle which which
I have heard many people talk about. You know, from an ethical perspective, the claim might be, oh, well,
everybody should do the thing that they think is best. And if they all agree that this is the best, even though they’re not sure,
you know, it’s understandable, you know. But then but then you can make the argument that will in the
long run. Right. Yes. Antibiotic or learn if anybody’s going to be better. So
anyway, anyway, then the second thing, which is a little bit more loaded is just the amount of groupthink
and sort of social piling on due to social media, just due to human frailty, because
people are weak at social is decreasing the number of people
that all of a sudden were. One hundred I’m trying to think
of a good example of how crazy I think this all is. I mean, look, we know scientists,
right? Like like we have colleagues, they’re scientists and we know that they’re just people. You’ve seen the sausage made.
It’s not always pretty. After a decade or two decades,
things get polished up and they get presented. But in the short term, science
is a little messy. And I think because of that, people that do science are a little bit skeptical. Usually
I did not see that here. All of
a sudden people got scared and they were very willing to say, like angrily, maybe this is just my little
bubble. I’ve got a bunch of well-educated friends. But by and large, everybody was like, trust the experts.
Flatten the curve. There there was not a lot at home.
If you dare not to believe that stay at home was the right thing, you’re you’re you’re somehow. Oh, there is. There was a lot of value
to shaming going off for people that had differing opinions to the point where people
I know weren’t quite on board. They want to say thing.
So I just I know that regardless of whether or not they’re right or wrong,
the fact that they they felt chilled. You know, I heard about this in history books
about this sort of phenomenon. This is the first time in my lifetime that I’ve really see shop
where I thought I thought that was kind of scary, just that people weren’t.
So so let’s talk about expertise. OK. So I naturally and sort of
authority averse, I guess. And so I’m a jerk. If somebody says do this,
I say, why? And that’s just how I’m fired. And I don’t think that’s necessarily right.
But it’s also not right to just trust with the experts blindly. Right.
So go back a few years. Do you remember when Nate Silver was crushing that prediction, all predicting all the election
results and so forth? And he wrote in his book, This Big Thing about
the death of experts and how data data driven was. Now
the you know, you didn’t need an expert, you didn’t need a talking head. Around 2012
or so, there was this whole thing where people on the news would get on the news and they say it should go one way and
then go the other way. Right. So screw the experts.
Trust the data. Trust the data. And boy, did that go out the window.
All of a sudden, I was hearing the same people saying stuff like Danielle epidemiologist
had been studying for this. Unless you have a degree in epidemiology, you can’t have an opinion on on the
social media platforms are filtering out clothes for people that did not have a degree in epidemiology or some
like. That’s about that’s like scary. So is it you know,
it’s a theme of our times. That information is widely available and
there’s more talking heads than ever and there’s credentials in it. It’s a sort
of side note of all of this is like when do we trust the experts? Like how how expert are the experts,
in my opinion? I guess I don’t take I’ll take some flak for this, but I have not
been overly impressed with epidemiology as a field. So I’ve got a few epidemiologists
that that I follow professionally. Just a namedrop
Sanders Greenland is Andrew GREENE, that is an epidemiologist I follow and out of UCLA
right at UCLA. Jeannie Robbins You know,
there and there are people that that work in this area that I know there were professionally
I would say that are outliers in the with their careful. And and so,
you know, simple differential equations model of a virus, not good enough.
Not up to the task, which is not really it’s not an indictment about I mean, it’s it’s hard,
right? It’s hard. And I’m not going to say it’s useless, but good God. Let’s not pretend that these things are
anything. I think that that forget about a judgment on on their field, on the models specifically.
I think that they got a good, as you said, a careful reading of any of those statements of those papers in
the early days would say that we don’t know enough. Yes. Right. And then again.
So for the purpose of policymaking, there are situations that the policymaker doesn’t know enough.
You’ve tried their best to corroborate the information you have. And now we’re going to try to think about the tradeoffs you face.
And you might not know enough or the trade off took place either, because in this case, you don’t know what is going to be a better
one. But you have to make a decision. You have to consider. Charles means Charles Minsky,
the economist at Northwestern, has a line. We are talking about decision making under uncertainty.
And, you know, most people would say you want to do best on average. But in a case like this,
you don’t get to realize the average is an individual. Right. And so he’s got this line that really stuck with
me. He says, we don’t want to maximize
our primary probabilities. He’s excellent. We don’t want to maximize ex-ante probability.
We want to maximize ex-post outcomes. But you can’t
do that. You can do that. Right. But it’s an interesting it’s an important thing to remember, because what it means
is that you can make a decision with the best of intentions, using the best of the data, using an honest utility function
and still get clobbered. But, you know,
that’s of errors. But I think I think going back to just to close it up, one thing that
you said in the beginning is the idea that people decided to to do plan on the worst case scenario
in one dimension, which was that let’s do everything it takes to reduce the number of
deaths, whatever it is. Right. And and people tried to portray that as as
a precautionary principle. Oh, we don’t know enough. Let’s be super careful. Right.
But but, yes, what’s forgotten here is that, well, by doing this incredible
credible measure, sledgehammer on people’s lives. Right. I mean, I’m not I didn’t know at the time
that all of a sudden supply chains wouldn’t break down, that there’ll be no food in the grocery store. I didn’t know that. Yet
somehow we know. Thank God for I mean, at least companies that kept doing their job like
did we know that would happen for sure. So anyway, this goes back to this goes back to my statement about
how the the incentives of a profession because epidemiologist
by training narrowly focus on deaths due to the disease. Right.
Disease specifically just sort of what they’re trained to do. So you can hardly fault them for it for when
they write up the reports, even if they wanted to do a worst case analysis, which they should have been more honest about the worst
case. But what you tell us, the makers are the ones the policymakers then need to recognize
that that’s a narrow metric. And health economists are supposed to be the ones that sort of do the second step,
which is to say, OK, here’s all these other things. But I think also. But also psychiatrist and also sociologist.
And also like there’s a lot of different input that I think you’re I think that you’re right. There’s a logical fallacy
there about how to interpret the data, but really focusing on
these numbers. And by the way. We should not be reading these numbers daily. It is ridiculous
that it’s like it’s posted up there and people. It’s like reading tea leaves people without training or watching these
numbers going up and down and then deciding whether or not to go to work that day or or what
sentencing. And I have too many thoughts, too many
thoughts on this. I just one just left. So what do you think happens next? That’s more like now hour, just to
put you in a bad spot of making a prediction to close it up. I mean, I think.
Well, it certainly is. I’ll stick to my guns. I think that what’s going to happen is people are going to try to be very judicious
with a lot of caution. And it’s sort of great inconvenience. They’re going to make four year olds wear
masks and they’re going to open up schools at 40 percent capacity
and stuff like that. And I think that they’re doing that because there’s some regret driven decision making, which
might be reasonable. And my forecast, as imperfect
as it might be as an informal as it is, is that people are not going to get sick
and that pretty right as much or as much as or as much and that and that pretty
rapidly people are going to stop adhering to the
measures because they don’t see them. Is doing anything, you know.
So maybe I am an optimist. I’m I’m actually pretty optimistic about stuff. And I think that it’s going to sort
of. Go back to normal faster then I mean, faster relative, though, because this is already
dragged on longer than I don’t know what’s to stop it. But
yeah, we’ll see how it goes in the fall. I mean, I think higher education is going to be. I mean, this is me
just talking about. I know, but I think one of the more affected industries
is going to be higher education just because there’s a lot of flexibility. And
the troops were already circling on some of the traditions. So kind of interesting.
I don’t like that. I don’t want to talk about models a little bit more. I want to yes, though.
Five more minutes. Otherwise, it gets too long. Nobody watches it. So there’s a famous there’s a famous quote in our field by
statistician George Foxes. All models are wrong. Some models are useful. Right.
And so it’s a it’s an interesting quote in context, what he was saying was you’re probably not modeling
reality perfectly, OK? And that’s OK, because you don’t want your model
to be exactly perfect because there’s another saying this is the map is not the lamp. If you have a map that says
that’s to scale as big as Rome, you know, it doesn’t do anything right. So you want something that you can understand
that that captures the sort of key features. And then he says some models are useful. So
in other words, you can come up with a model there’s accidentally too simple and doesn’t capture
the key features that you want. So people use this a lot as a defense
of their favorite simple model. In particular, a lot of times mathematicians
and statisticians would pick a model that they know how to analyze, meaning they know how to do the math to get it
to work. And then when somebody challenges them and says, well, you know, that’s not actually
accurate, they’ll say, well, all models are wrong, some models are useful. And actually what
they typically do is they kind of elide. They just stop and say, well, all models are wrong. Dot, dot, dot,
dot. That’s OK. So I think in this case, we had
a lot of that going on. People, how did these epidemiologic models, these infected,
expose susceptible Assad? Is that the right tactics both
recovered? I recovered more removed from what they can
die? Exactly. You know, I saw some people talking
about these and they’re like, yeah, they’re simplified. But to do it realistically is too hard.
You can’t do it. That’s just that. So so in particular, what you need to try to do is elaborate the model
just enough, so that captures the features that you need because you literally learned nothing from a model
that is like literally what you learned about as the model. And you
want to make sure that your mom is qualitatively right. And but, you know, if you have
if the mechanism is is that a few super spreaders do it and you’re trying to evaluate
the efficacy of a lockdown, things are going to look very different than that. And that’s actually
that’s the difference between economic models. And you’ve been a critic of a lot of the econ models that
we’ve seen before. I actually have a lot of similarities in its composition
to the geneology models. But the difference between the economists and the epidemiology is that
economists are very careful when thinking about the policy variables in their models and
trying to figure out what were the levers, the levers that actually, you know, you’re gonna
be simplifying. But let’s have the levers in the model, at least the levers that we’re trying to understand. What happens if I do this
or whatever I do that. That’s not in the models of epidemiology. They didn’t have any leverage. There were meaningful leverage.
That’s why the only thing they had the only thing they had was
social distancing. Let’s reduce it because the blindly across the board, because they didn’t have
any mechanism or maybe I mean, I’m not going to let the economist the models. I’ll go and say the models
that I critical parts for the dynamics of plastic generalisable models,
because they’re like they’re so complicated that they’re almost as complicated as the world.
And yet we don’t know the ways that like. And so I’m going
to quote somebody, but without attribution, as I said. But somebody said, you know, those models
should do two things. It should make good predictions or it should be simple enough that we can understand it.
And it doesn’t make good predictions that we probably wouldn’t trust the explanations.
And if they’re so complicated, we don’t understand them. We don’t know how to use how to use it. You
know, some of these models have. So you can be wrong based, but so.
OK. That’s maybe a good place to close it up. The models that we’ve seen in this pandemic,
they seem to fail on both. I can’t go if they’re not good enough to predict and
they’re too complicated for anybody to understand. Yes. On that positive note,
and maybe I’m trying to think, you know, what does that leave us to do? Right.
And in cases like this, this data is data. I think the data I think that
what I’ve been what I’ve been focusing a lot my efforts on and this direction has been trying to learn
from just data on coming out of things, that data that allows me
to then help potentially inform the overall sledgehammer decision. But like
more point wise decision making. For example, the fact that kids are
now we have overwhelming evidence doesn’t rely on any model. There’s overwhelming evidence that children
are not at risk of dying. But some will, just like some kids, die of
many things. That’s tragic. Every single death of a child is tragic. I have two little kids and I can’t wait to and have
them go back to school. So so the overwhelming evidence that kids
are not separable and there’s plenty of evidence already of the fact that they are not big contributors to the spread
of the epidemic. So regardless of any other thing, those two pieces of evidence that are available
don’t require a model and to have a huge outsized impact on people’s. So
I think to me, that’s enough. And I don’t understand why no government in the West, by the way, in Europe, every single
country near the schools are open right now. Every single one. Oh, I see. Well,
nobody’s willing to go and say it. No, I don’t think there’s so much to talk about. So. So New York is
so different than Austin. Austin is different. It’s geographically, politically,
socially, demographically. Right. It is crazy that
we’re looking to the federal government. In some sense, or even the World Health Organization for
like Blink, give guidance on how to deal with this stuff as if.
I mean, Tucker, that’s a one size fits all policy, which seems like a disaster. Right. Clearly, like maybe
New York should shut down. Right. Here’s the. York City. Me and New York City, maybe. New
York City. Full on. Shut down, as you said. House arrest. No public transportation.
Don’t answer the door for anyone. All right. Maybe that’s the right move there. Is that the right? Is that
the same move in Arlington, Texas? Probably. Probably not.
Yeah. So. And going back to a point of view of yours from before, right, this
idea of experimentation is a huge thing for us to learn about what to do in
the US. One of the one of the greatest things about our system is the system sedimentation fermentation. It comes from a federal system
that we have. The 50 states have autonomy to do things. I love that aspect of it. I love that Texas can do things different
than New York. But we went into groupthink. Everybody did the same thing. Now, it was actually interesting because
now you have their ability. So you have you know, I am not in house arrest anymore in Texas.
You know, my friends in California are. Yeah. So, you know, you can see whether
whether what are the dynamics for those two different policies put in place that we’re going to learn a lot of things by the fall,
by the end of this year, there’ll be a lot of information that will rely on complicated black box
model information that we can just look at. And and, you know, yeah, some assumptions
are needed always. But but we’re going to be able to get I mean, if you call if you call these models black box models, they’re not black
box. Understand? And while it’s like kinda right.
You don’t understand the mapping back to the root is the problem. Like that. That’s the thing. Like the model it needs to overlay
the the reality. Sort of in a reasonable way, in one way, it’s another
way to think about implications that you’ve got the map that you don’t know which or injection, you don’t
know what transmission would say. That’s right. So, yeah, maybe you understand your map, but
if you don’t know how to make it now. Rambling. Thanks for having me. That’s
great. Thank you for joining us. And we’ll do it again. Thanks for listening to
Policy McCombs.