Jay Battacharya – COVID-19 Interview

[0:00:00 Speaker 0] Mm. Welcome to Policy and becomes a data focused conversation on trade offs. I’m Carlos Car value from the Salem Center for Policy at the University of Texas at Austin. A pleasure to have with us today. Jay Bhattacharya from Stanford University professor of medicine and professor of economics J Thanks for joining us at the Salem Center for Policy. It’s my pleasure. So, J you know, this interview that I’m conducting, I’m asking people to to try to go back in time and think about the evidence and the sort of information they had in place and start thinking about the covid back in March. So So what was the first sort of a set of information and and and and and evidence that really, you know, started getting attention to the problem. So, uh, should I go back to March? My I’ve been doing work on economic technology for a long time, and I did some work on the on the H one n one flu epidemic in 2000 and nine, and in March, I thought back to what we knew about the mortality rate, h one n one and how that evolved over time. So very early on, there were case reports in the H one n one. This is 2009 epidemic about enormously high mortality rates. Case fatality rates of what we call him now. All right, So, uh, from H one n one. And in fact, like Argentina, those reports 14% typically worldwide was like 123% very, very high, you know, 3% isn’t really high. 300 people died from a disease. It’s enormously high. And what happened in h one n one was that Syria travel study started to come out That found that that there were about 100 times more cases infections than there were cases 100 times more. You know, Sarah Palin studies is your audience probably knows is our studies of antibodies in the blood of the patient of people. And, you know, Sarah Palin studies attempt to measure a temporary population number rather than just looking at people who come down with the illness and, uh, identified by doctors are then tested to see if you have the virus. Uh, your province looks for antibody. Evans says Okay, Yeah, Even though you may not have shown up to the doctor with the disease. You you had it because of the antibodies that are specific to the disease. Say that you had it. Uh, and in the case of H one n one that was turned out to be 100 times more cases, infections in cases they started to up that, uh, so when you say 100 times, that’s that’s probably a lower bound, right, because our ability to detect antibodies is also restricted, Right? Takes a while to because they go away right after a few months. Yeah, that’s that’s right. In fact, that comes out of you. I think even more important, the case of covid than even than in D H one n one case. Um, and not everybody reacts to antibodies with with antibodies as strongly as other people. Uh, you know, so it really is. It is. It is a lower bound is probably lower than it’s probably more than 100. Um, in any case, with that hundredfold that the infection fatality rate turned out to be about point number one point oh, one and 10.2%. Okay, one in 10,000 to 2 and 10,000, whereas the initial estimates were on the order of one in 100 too much higher. Okay, so I remembered that. I mean, that was my very first thought when I saw the case fatality rate estimates that were out in the in the public in covid. And so I mean, I suppose it’s not that unusual to wonder, and that’s what I did. I just wondered, Do we have that situation now? And I looked around and no one seemed to be talking about that, which was really odd. To me. It seems like the most obvious One of the most obvious signs that I remembered it from the H one n one epidemic. Why should we not consider that as a possibility? Now, uh, everyone’s attention was focused on the people who were dying in the hospitals with the with the severe viral pneumonia, which was absolutely terrible. Um, but we didn’t know. I mean, I didn’t know how widespread it was. I mean, you could you shouldn’t extrapolate from a previous disease. Could be 100 to 1. It could be 10 to 1 who knows what it is. So I worked, uh, with some of my friends. We, uh, colleagues here and elsewhere to start up some of these Syria prevalence studies to try to get that number. Um, that’s so And I think a lot of the discussion that led to the lockdown was it seemed to be struck me, as in the absence of that vital piece of information, How widespread is it? And how deadly is it Unless you have that, you can’t get any reasonable discussion about the cost of benefits of lockdown. And just so we’re clear, the lockdown itself was an extraordinary decision. I don’t think that I can think. I mean, I don’t I can’t. I don’t. You have to go back to 1918 to find some evidence, some something similar policy of such widespread, uh, cessation of, of, of all activity as a as a way to combat the disease spread. Just it’s unique. In my experience. We even then I think it wasn’t as extreme right in in In the in, the spread of the lockdown was like, specific some cities and and so on, right? Yeah, it was a universal lockdown. It was I just followed one after the other one country after the other basically decided on the basis of, I think, almost no evidence that it was. This was the only way to address the epidemic. Um, I think when you do an extraordinary policy like a lockdown, you need extraordinary evidence. And we did it with the absence of that kind of evidence. So So let me, Let’s go back a little bit. Right? So when when we’re looking at the evidence that you you you had experience with H one n one and presumably the epidemiologist at a time involved with h one n one, which I’m sure they types of, uh, sort of what they call it s I R. Models that are put in place to study viruses. They’re not that different. The ones that were used for H one n one. Right. So one of the inputs for those models is the infection fatality rate they have to make assumptions about. Okay, if this thing infects as many people, how many people are gonna die and then make those projections? So from from your description of what? What? The difference between the initial case fatality rate on h one, n one and, uh, the actual higher Fars infection. Fatality rates are found later on. That probably was a huge discrepancy in the epidemiological predictions at first and quickly in a span of, uh, I don’t know, what was the span of time between the initial months in the case of H one n one to get to the right number. That’s why I partly that was that was what motivated trying to go fast. I wanted the conversation to move very quickly to actual numbers, using the prevalence studies as opposed I want. That was my whole goal, actually was to get the people who were populating these models to populate them with parameters that had some evidence behind them, as opposed to make the things they were just guessing, right? So that the time during h one and one. So what I’m trying to say is that how what would you a sort of pinpoint as the difference between Why didn’t we freak out and lock the world down under H one n one? Given that there was some indication of this is a flu that goes around pretty quickly, it’s pretty aggressive in terms of its contagious. It might be killing 3% of people. What what was different then? I think. I think the the single biggest difference in the early discussion was was the possibility of very quickly getting a vaccine. I mean, they rushed the vaccine and people worried. Okay, this is the vaccine safe. But we have a lot of experience flu vaccine, a lot of strain with developing them with, uh, assuring that they’re safe. And we have a mechanism to deploy them in a wide scale setting that none of that was true in the case of covid. Uh huh. I think that the ability to develop that vaccine so quickly in case of H one N one really did calm down the public policy response. And then the Sarah Palin started studies started coming on like, Okay, we got very lucky, right? The thing was not as deadly as we thought it was. Thank God. Right? And we have a vaccine quickly. It’s all over. But in some sense, it was, in some sense, it was It was happenstance, right? We know now it was happenstance because the thing turned out to not be so deadly, and it happened to be the flu. So we have a vaccine platform H one n one. Uh, so different from covid. In that sense, No vaccine platform. We have this deadly number 3% number. In fact, I think the imperial college model they populate their initial I fr was, like 1%. And that’s what led to the two million deaths. That’s right, right. Uh uh, And they use these, like, you know, these beige in kind of Yeah, like they’re trying to like these models that they’re just they’re under identified like there’s all these parameters and their fitting into not very many hard endpoints. And there, you know, like they have some some Beijing fit to, like, try to get to get the parameters to work. But who the hell knows if those right, You really need to populate some of those numbers with actual data on. You know, you’ve drawn from studies like Sarah prevalence studies to have any confidence in them. Um, well, and on top of it, I think those models, uh, just the lack of the human element to it. Like once, you know, we understand things. Once we started reacting to it, the exponential growth that they assume and those things were just assumed and they’re like, yeah, of course, we That’s not how the world works, right? I mean, that’s a That’s a classic problem that economic epidemiology addresses, right? So the idea is that people respond. So the the elasticity of demand for self protection is a function of the prevalence of disease, the dangerous, the perceived dangerousness of the disease. So when the when the danger is going up, people will automatically voluntarily engage in self protection, hand washing, other kind of mitigation activities, you know they won’t go to that won’t go out to baseball games or whatever, So I mean that that is not in any of those models. They’re all these very, very detailed compartment models. But that kind of feedback of how prevalent changes behavior, not in any of the models and, of course, there. As you pointed out there, the models tend to be these homogeneous models of mixing, and we know for certain that there’s no modules mixing that. In fact, there’s like people mixed with, uh, very heterogeneous ways. And in fact, even it’s even you can go further. People who believe that their highest risk our respond more dramatically with reductions in, uh, social mixing. Uh, and the models don’t have any of that in there. But that’s just that’s a perennial problem. Um, but I just But I I f r number drove everything. It scales up and down with that fr number of those models. If the I phone number really was 100 times, that turns out not to be 100 times. Let’s just can we jump forward, I think turns out to be about 5 to 10 times less than that. Probably five times less than that, uh, that that the number of deaths scales with the fr in those models. Um, so let’s go back to two, then. Where when we used you, try to push forward to some, do something fast, right to get a Syria prevalence study done as quick as possible. Um Then tell us about the Santa Clara study. Sure. So, yeah, that was the first of three Santa Clara policies have been involved with the Santa Clare study. We organized it, uh, in at a speed and all the credit goes there on Dr Baron made David. Actually, one of my very first students ever said he’s down the hall from me. Um, and he organized it in in a incredibly rapid weight. And it was actually a really nice feeling that there was all kinds of folks around the Stanford community that came together, volunteer their time, medical students offering my family offer what we’re volunteers, Iran’s family were volunteers and in helping set up things and, you know, that enter data, all this kind of stuff. Um, it was it was, and normally a study like that where we drew blood from, you know, 2000 more. A whole bunch of very, very large number of people in the community would have taken a year or more to set up and get permissions. We managed to do it in a matter of weeks. Um, partly this this to happen because we were using a test, um, platform called a lateral flow assay test. It’s a little tiny. It looks for all the world like a little pregnancy kit, but it measures antibodies to the specific sars-cov-2. Now, the FDA had provided a authorization to commercialize those kids in the US in March For research purposes, they weren’t authorized for clinical use in the population. But for our use. That was perfect. And we knew a fair bit about the test that we used. It looked like it had pretty good sensitivity, specificity, properties. We turned out we needed, we needed. We’re going to learn a lot more about the kids. But in March, we know they look promising in terms of sensing the specificity properties we powered up, our study said, Okay, we don’t know the prevalence, but let’s say like and we made a bet Me, the main authors of the study about like what we thought the prevalence is going to be. I thought it was going to be much higher than it turned out to be. Um, you know, that’s why you do the number. That’s why that’s when you run the study, right? Um, my co authors thought it would be lower, Uh, and we powered it with their guests because we’re going to be conservative and lower means you need a large sample size. And so we probably the very pretty large sample size and we collected over two days. We collected thousands of samples. The test kit, the nice properly about the test kit for fielding it in the context of a lockdown was we could just draw blood off a finger stick. The people just came up, drove up. They stuck their finger out the window, didn’t expose the person pulling the blood or to anyone to any any respiratory droplets. We don’t know. The risk is so we have everybody mask up and where gloves and all that kind of stuff. And having that essentially the car window shield was another level of protection. Uh, if we had tried to get there are there are another set of tests called Eliza’s lives. You have to run in a lab and you need a venous blood draw. At least you did back then. A venous blood draw two. And it’s getting a venous blood drives like it’s a big deal. You gotta pull you got. It’s hard to do in a setting like that. It would be impossible to run a study like that. So presumably the second time. But that would be more precise, right? So then So, yeah, I mean, although in retrospect, it turned out it’s not as bad as I mean. The little we use turned out to be fantastically precise, which I’ll tell you about it. So yeah, so we ran the study uh, and found a prevalence of of, uh, two point, I think, like 2.8%. Um, which turns out to be about 50 50 positives out of, uh, I forget the number, but the denominator is, in any case, the the we had we Now there are a couple of aspects of the sampling. We’re talking econometrics and stuff, right? So we can talk about that. Uh, the sampling, the study. We did this very, very rapid sampling through Facebook to sign people up, and it turns out that that is not a representative. Not surprisingly, um, so we re weighted the population to make it look like the demographics of the county. And so if you just do the unweighted, it’s like 1.5% with it turns out like 2 to 1 women signed up relative to men. Uh, the understand Peled. Some of the zip codes where there were poor populations backed by a fair bit. So we re weighted so that it it matched the zip. Uh, sex and race sort of mix of Santa come, and that’s where you get the 2.8%. So with that 2.8% that turns out to be about 50,000 cases. If you if you extrapolated the county at large, Uh, of course, that excludes jails and other institutional populations that weren’t sampled up to date in the in the county that up to the had there been 1000 identified cases, 1000 identified cases. So that means we had a multiplier by 50 to 1. If you, you know, you allow for a death lag. It turns out that that the death, the infection fatality rate, was somewhere on the order of 200,000 after you after you did all that. Whereas the case fatality rate at the time was about 10% in Santa Clara County. So 50 to 1 difference in the in the case fatality rate of infection fatality rate. So we’re talking here, just summarized, right? So you’re talking about a 10% case fatality rate in the county, and you’re looking at an I F. R. Estimate of about 100.25 or something like that, actually, at that 0.2 Okay, Yeah, uh, in l A county, we ran a very, very similar study. It was again. It was the most part was like 40 rather than 50 but almost the same I fr almost the same I for little bit hired, I think LA than in Santa Clara County. But nothing nothing to write home about. Um, okay, so we we we also did. I also worked on the Major League Baseball study, which we can talk about if you want, but these two are sort of the central studies of drawn focus. Okay, so we release the we send out the L A County study, which we kind of the week after Senate. Their study to Jammeh for review. It gets accepted almost immediately. It’s published almost no notice. And that’s the JAMA, the premier medical journal World Journal, American Medical Association. We decided we wanted to get out the message quickly on the Santa Clara study, so we put it on. A pre prints of economists are used to N b. R. Uh, that’s normal free content because it takes so many, so many, so long to get any people that they were published. And so we’re used to essentially what is called in medicine pre print service. We put up working papers. We discuss them ad nauseam long before they see print by the time they see print. It’s really all do is write in medicine. That’s a new thing in medicine. What the norm is you don’t say anything about the paper until it’s published. In recent years, this open science model has started to, like take over medicine, these open model. And actually I think it’s been really healthy for minutes because having these three prints, it allows there to be a much broader set of people discussing these things. You know, it’s mixed. I mean, sometimes the discussion is not that productive, but like that’s true in economics, too, I think so. We released it through this reprint and Twitter and the whole world exploded. I’ve never had 10,000 referee reports in my life up to that time way, I literally within a day I had 10,000 referee reports in my inbox all at once. So a lot of hate posts out there. I remember reading something from colleagues that I think are very thoughtful. General, I was like, Whoa, I’m not going to name names here, but there’s a prominent at this blog that was very mean to to to to U T yeah, I mean, like the statisticians and, you know, some of it was justified. So we made a mistake in the calculation of the standard errors and which we corrected, right? Uh, yeah, actually, so it was It was literally just like that. So the formula, we have two. So the citizens involved this. So you have to correct for how sensitive and specific the test is the false positive rate in the false negative rate. So we did that with a formula that corrects for the sensitivity, specificity, and it was my fault. The sample sizes for the sensitivity and specificity are not the same as a sample. Sizes for the for the for the the study at large. And I didn’t take that into account in the initial initial thing. Um, but a week later, we issued this is how free French work. You can update them. Not published, right? Yeah. I mean, uh, and my view of that was this was a really productive thing. I got the 10,000 referee reports. We took the important parts of it, we revised it, and we updated it. Right. And if you learn, uh, the other thing that happened, um And to be clear, that didn’t change the estimate. Right? Change your uncertain about maybe a little bit wider uncertainty on that train. Exactly. And I’ll say this. The other thing that happened was that because of the open science model, a very large number of people turned out had worked to evaluate the sensitivity and specificity of these of that specific test. We’re using the premier biotech kit. Um, so we got, uh I mean, we went from about 400 samples for the specificity number two about. I’m over 1000 over 35 over 1000 I can’t remember, just marry the number. But, like a very large number of samples, all of a sudden about both the sensitivity and specificity just greatly decreasing the actual standard. And, uh so even after so after we corrected, it was the standards were a little bit, you know, a little bit lot wider. The point estimate was the same. Maybe not that surprising, right? And it turns out that the kit is incredibly sensitive and incredibly specific. So 99.6 or 99 points, five specific on that order. So one a five false positives out of 1000 And, uh, you know, it’s that turns out to be the main number because they fall could have low prevalence environment. False positives can kill you. So we we It was very low, false positive rate at the point estimate. And after we got all those all those independent tests of the test kit, we learned that the standard air for that was much smaller than we realized. We released version one. Um, I mean, there were a lot of, like, false things on, like there was a very prominent post on medium arguing that all of our all of our positives could be false positive, which turns out not to be true. I mean, it just I think that science was fine. I was actually happy with that. I think that’s how open science is supposed to work. Um, and I learned stuff, and I think we low more like that. We have more confidence in the study as a result of that discussion. It’s good. It’s good science. Uh, what I didn’t expect was the media. BuzzFeed attacked my family, which was really annoying, like so my my wife. I told you to participate in the study. She She’s a doctor. She’s an oncologist. She wrote to her friends because she was all excited about the study to get to participate in. Somebody leaked the email she wrote, which had not someone there was. Some information wasn’t corrected in it to BuzzFeed, which who decided to make her into a national story. The funding for the study was small donors, all to Stanford. I mean, I took no personal money for the study, and one of the donors who gave money after the study was a JetBlue executive, David Neeleman, who was interested in the results of the study. I mean, he’s interesting results of the study, okay, He had $5000 to the to Stanford, and BuzzFeed wrote an article accusing me of conflict of interest outrageously, Um, I mean, it was really, really annoying to have to deal with all of this. And in the midst of it, Stanford, uh, itself actually acted in ways I think that attacked my academic freedom, which which we can go into in a bit. I mean, I’ve just been very, very disappointed with Stanford leadership on this front because I had always thought of Stanford is a place that respect. I’ve been here for This is not My undergrad was here. I’ve been I did my graduate school here. I did my MD here. I’ve I’ve always loved that. We have a motto. Is that let me in German to let the winds of freedom blow? I don’t believe that anymore. No longer the case, right? Yeah, I don’t I don’t believe that’s true about Stanford anymore. At least not in their current leadership. And I would I would point out to Stanford specifically on that problem, right? I think that have you been in a lot of different places? My guess? No. But this is high now. University, been pretty, is home for me. It’s just it came as a shock, right? It’s like you anyway. So enough about me. I mean that. But that that was that was not That was really a stressful time As a result of it. I wish it was just about the science. The other thing I didn’t fully expect was the political angle of it. Like, why is the fr of this disease a political fight you became? You became a rabbit republican. As a result of that paper. I’ll tell you, like my colleagues, my senior colleague Johnny, and who worked on the study with me, I have no idea what his politics are. I really don’t I never Why would I talk to Johnny Needs about politics when I could learn from him 500 other things that are way more interesting and important, right? And and Iran. I’m pretty sure there’s some a lot of politics we don’t share, but I don’t care because he’s a brilliant scientist and I learned from every time I talk with you about something important, So I I just It was really, really weird, Like it doesn’t matter. Like I’ve always thought that, like I write with everybody, you can look at my my CV. I’ve written with people of all different political stripes. It doesn’t I don’t Frankly, most of the time, I don’t know what the political stripes are because it doesn’t make any difference. And and and to be clear, right, uh, one of the things that your study was providing the potential evidence for us to be able to make a more informed decision about what to do and what not to do, and it was not at all pointing out. Oh, here’s what you should do or should not do, right? That’s I guess, the realm of politics. I mean, like you the fr Okay. So, I mean, obviously, the policy is going to be different if you think that the death rates three and 300 than if it’s 200,000, right? Right. You have to have a different policy. Otherwise, why you just you’re not doing it right. Um, but shouldn’t be informed by that. So any question, we put the study out and it creates this, like, huge firestorm of attack on us. And, you know, the economists are writing me saying, Oh, no, no, you can’t. You can’t be that low. Look at New York and the promise like it gets, it gets the medicine wrong. The I f r is not just a single number. That’s a feature of the virus. It’s a function of the host and the virus and the health policy setting in which which the cases are managed. So if you’re in a setting where you can get good medical care, you’re going to have a very high fr. If you’re in a setting where the bug hits old people in nursing homes. You’re gonna have a very high fr right if you’re if you’re if you’re if it’s if it’s hitting basically young, healthy people that actually it turns out that the death rate from this is really a stratified. So, like if you’re over 65 this is the experts from the much, much worse than if you’re under 40 it’s better you’d rather have. You’d rather have this than and it’s not false. I mean, for instance, among among kids in the U. S. I think somewhere in there were three times more kids have died of the flu this year than of Covid 19, which by any measure, has been a very mild flu season, which is hard for us to separate because a lot of it, you know, covid and flew versus flu. And although there’s just like emerging event so you can get both the same time, I don’t know there’s some some some interesting sense there, but any case, like it’s on that order, it’s not. It’s not that deadly as far as it’s not. It’s not like super, super deadly the way it is for older people. I mean, for young people, it’s, you know, and so that a stratification matters for policy also right? So because now we know who we have to protect, who have to work really hard to protect and who who is. It’s sort of, you know, we don’t. It changes the balance of cost benefits, right When you think about thinking, thinking about that, a stratification in your l. A or Santa Clara study, uh, you probably didn’t have enough power to get a good read on the fr per different groups. I didn’t, for example, didn’t sample enough Children to get a sense of LA We didn’t do Children in Santa Clara. We did do Children, so the prominence looked pretty pretty close. We didn’t end up reporting it in the main paper, but we’ve now done that calculation. I don’t know if you tell. I mean statistically. There are states that a different difference between old and young, even in the Senate, because the actual point estimates are so large, the difference is so large. Um, actually, at that time, not a single child had died in Santa Clara. From it, I think that’s still true. It’s still probably true, right? Yeah, the number of so low. I mean, it’s almost like it’s hard for us to even pinpoint that took over. If you look at the number of being so low across the world on Children, I mean and that’s, you know, it might be a unhealthy Children dying for various reasons. And I have to say, like when I when when the thing first hit like I worried, What what do I will? I bring the whole covid home because I had to go out and interact with some people. Occasionally I’ll bring it home. And after I saw the study results, I thought, Okay, that’s that’s That’s a huge blessing that we don’t have to worry about that for our kids. But just imagine, if that were the opposite. You have the 1918 flu, which which young people? That’s a hugely different optimal response there. I would think so. Let’s fast forward a little bit and then think about all the other fr studies that are now studying and they find the same. Okay, I’ll be a little careful because I’m talking to someone who knows statistics really well. So our number is in the median aware of these I forms of these studies these here, but now they’re 50 plus of them. Done. Um uh, in places all around the world, in places like Japan, it looks like the FR is lower in places like Spain. It was higher. Uh, and we’re the media. We’re pretty close to the meeting 0.26 So somewhere between two and 2000 and three and 2000 is the I f. I think. And so just to just to nail that home, that means that if you get infected that somewhere between 997 times out of 999,000 times out of 1000 you will survive the disease. Okay, that’s of course, the caveat be the big stratification for age, right? So So So we know now. And also there are other things that I didn’t mention but are really important as well. So they’re like, if you have some common conditions, it seems like you’re it makes you put put you at higher risk things like that, which I mean, we’ll learn more as kind as on, but I think we already know quite a bit about. Right? Right. Um, actually, yeah. So my my my my brother in Brazil got got diagnosed with Covid. And and he has two of the comorbidities. He’s a little older than me and and, you know, high blood pressure prediabetes. And we’re like, Oh, wow. But then I, uh, So he actually got hospitalized for a couple of days for no reason, just for observation. And I remember talking to my mom was like, Listen, I look at the numbers, even his condition. We’re talking here at maybe one in 1000. I mean, a little bit more, a little bit more, a little more, but that it’s still less than 1%. Probably. Yeah. Certainly. Less than 1%. That’s which is I like the hero college model said, Look, I I think, uh, you know, 50 studies that find roughly what we found. I think we’ve got the science, right. I think Twitter got it wrong. So it was just I mean, not not to say Twitter was bad. I mean, it was actually said productive to have that scientific discussion, so we can know if this, uh, we’re finding something robust or not, But the real test is replication, and we now have replication studies from around the world that found the same thing, right? And in addition, right, I think we have the sort of, like the assumptions about number of deaths associated with desire for 1% you know, where Maybe 2.2 million deaths. And we’re like, order magnitude away from that, right? Of course, there’s two things going on, which is the infectiousness. That and all the measures that we’ve put in place and all the things that are still doing that clearly slow down the project, the progression of the virus. Um, but I think the best case scenario with all the mitigation measures of those models were still talking about an order of a million people dying. Yeah, I think I think those models were grossly mistaken. Um, And there were people like Giannini’s saying that at the time, right that we don’t have enough data, and that was my That was my instinct as well. We didn’t have enough data to make the call. Why does it matter? So I think, um, if we’re going to talk about policy, we have to account for the uncertainty in in in this decision making. We can’t make the decisions as if there’s only one side to it. What I think we said we did, We said we saw the models. We said, Okay, this is saying millions of deaths. Well, what’s the harm in shutting down for a little while? What’s the harm? Right? Um, as if there were no cost to that whatsoever. And I think this is where economists failed at that point, I think economists as a whole should have risen up and said, No, you need to think about the costs and the benefits. I think it was a very easy rejoinder that drove many people away from that argument, which is that the cost as the costs as they were best, like argued, as specified in Twitter and other places, was on economic costs. Oh yeah, your GDP would get hit for a little lot. Who cares? Lives are more valuable. GDP. What’s true? I think lives are more valuable GDP, but the point is that it wasn’t just GDP on the other side. It’s lives on the other side of that as well. We’re going to have seen that and we’re going to continue to see that I think for for a very long time. Actually, um, so I think at that, at that point, we should have been saying things like, Look, if you shut down the economies of the whole world, which is essentially what we did, the number of people who depend on those economies functioning well is absolutely enormous. I think I think I saw some numbers. We’ve raised a billion people out of poverty in the last 20 years with GDP growth in developing countries worldwide. Right? So I I grew up in, uh, in the United States, but I was born in India. Uh, when I was born in 1968 my, uh, I looked this up at one time. My life expectancy birth was 48. Wow. 48 52. A kid born in India male born in me today is like 60 60 something. I mean, enormous progress. Um, that progress is not magical. It comes because of economic growth. Reversing that economic growth has consequences. So, like the U. N, for instance, are saying that 100 million people worldwide will start as a consequence of the lockdowns and the economic shutdown. Gabby which is this program that does immunizations worldwide shut down its operations. We will see a resurgence of polio and measles worldwide as well as a consequence of this, uh, definitely diseases. Tuberculosis is going to have to come back already has come back million extra deaths of tuberculosis potentially because, you know, in order to treat tuberculosis, it’s not just the one time you get antibiotics to keep coming back, time out, week after week or whatever to get the new dose. Um, you stop that program and those people will die from tuberculosis. They’ll spread it. Cancer diagnostics Probably right. Cancer, cancer. So we haven’t actually had success story the last few years, where cancer mortality United States fell for the first time in living memory that is going to be reversed almost certain, because not only that did you get people not coming in for their their cancer screening, so we’re going to see later stage breast cancers. We’re going to see later stage prostate cancer colon cancer than we otherwise would have seen. But we’re also we also saw people who were more afraid of covid than of cancer. They had cancer. I mean, it’s just It’s sad to think about, but it’s exactly what happened. Um, there was a little bit of a mystery for a while. Why weren’t people coming in for heart attacks? Looks like covid cured heart attacks or something. Right? Strokes as well, Right strokes. And that’s what it was like. The number of of stroke patients disappeared. Yeah. So what was happening? People are dying of strokes and heart attacks in at home rather than going to the hospital. Get treated, Um, where we have excellent treatments for heart attacks. Now, you know, just it’s just much safer. I mean, if I don’t think I would go to the hospital, I mean, you have to. All right, So those are costs. Those are lives. There is no risk this option, which is what economists should have been saying at the time. But instead, what I heard economists saying was, You know, we have to we have to. It’s okay to start. There’s this worst precautionary principle. There’s this worst case. Let’s focus on that. That is no guide for action because, especially in a situation where there’s massive uncertainty and you don’t know and there’s risks on both sides if you. If you take the precautionary principle, you don’t go out and drive ever. Yeah, well, I like, but even, But it’s even worse than that. Like that’s true. I mean, there’s this reproductive absolutely assertive and kind of thing. But there’s also when you have a situation where there really are risks on both sides, which which risk you mitigate. Precautionary principle doesn’t tell you anything about that you have to make. You have to use a good old fashioned economic analysis like think about the cost and the benefits Carefully assess the risk. I mean, just it’s hard, but you have to do it and you can’t let people who are that are pushing a particular policy because of fear, basically intimidate you into doing those kind of careful calculations on both sides. Yeah, and, you know, the sort of like, uh, fighting words of like, No, you need to trust the science as if the science is a settled thing. And as you pointed out right, there was no settled thing at all. And the science Here’s how the science was not settled is a bit of a joke. I made my students today. If I had predicted If I was a flat earth ER and I predicted in March zero deaths in Zero Hospitalizations by Covid, I will be closer to write an epidemiologist in Texas. That’s true, actually. I mean, it’s a my mean square error will be smaller then the predictions made by the experts. I mean, that’s that’s atrocious, right? I mean, it’s as if there was this, like sense of, like any any mistake on the upside is harmless, and any mistake on the downside is going to kill people. That is, that is just a mistake, an enormous mistake. And it’s an enormous mistake that economist should have known better should have been crying following from the beginning, you know, and I you point to economists. I’ve been very critical to them as well. In a sense of like, where are they when we needed the sort of adults in the room making this case is very strongly, they disappeared. Statisticians also disappeared, And John knew are I think, you know, John is somebody that in the statistics field is incredibly respected. And my profession somewhat, you know, was very quiet throughout this whole thing. We know the badness associated with the epidemic, the electoral models. We know we look at them in five minutes like Whoa, hold on. And we didn’t say anything. We did not go, but they were too busy attacking the standard areas in the Saint Clara study. You got it. That’s all right. So let’s go to the next topic that I think it’s also incredibly important to what we’re living right now. To some extent, you know, I think the preference of people one over the politicians, at least in the U. S. But like even though in California is still more in a lockdown setup than than I am here in Texas. For the most part, I think people some said, like, uh, no, we’re seeing the risk out there and we’re living our lives, adjusting to our best to do that, even though they’re being bombarded constantly that Oh, no, we haven’t done enough. We haven’t done enough like Well, I don’t know what enough would have been. I don’t know what the options were and so on, but here we are at a point where where lots of things are open, lots of things that can do but lots of kids in the state of Texas and most states in the country cannot go to school. Still, and and so what do we know about the evidence associated with kids? Not only we talked about the fr only as being much, much lower, like perhaps close to zero for kids. But what we know about their their sort of contribution to the spread of the disease. Yeah, so I mean, I think that you started with is the most important point, which is that it’s safe. As far as Covid is concerned to send kids to school, the risk they face from covid vanishingly small and, in fact, two kids themselves. The risk of not going to school are higher than going to school by an order of magnitude. Kids are where so, for instance, like kids schools, I study school, school nutrition. So, like it’s very, very large. Fraction kids 3rd, 3rd, higher get get their meals, subsidized meals or free meals through school. A lot of the nutrition happens in schools. If you are special needs, that’s where you get your help. Like socialization takes place in schools. It’s where, like, people get a psycho psychological help when they’re when they’re going to kids who are going through tough times. It’s not just the education as well. We know that even short duration stoppage in educational activity can have very long run health consequences. There’s an enormous econometric literature that establishes this. Um, so it’s not just that you’re reducing, you know, job prospects in the long run or education. You can’t make it up. Like, how do you teach a kid? How do you stop first grader to read on Zoom? It doesn’t mean it. Just think about it just doesn’t make any sense. Right? Um so, yeah, I think schools are enormously important for Children. Develop mentally, psychologically, nutritionally in so many ways, whereas like having them stay at home is no substitute for that. Even learning from home is not not not developmentally appropriate way to teach. Okay, so that’s the first thing. So the it’s safer for the kids to be in school than not then the question is, how about the staff there? You have to look at these studies of spread of the disease. So let me just tell you about one study, which I think is my very favorite study in the whole epidemic. I’ve seen this study published in New England Journal Medicine, Um, done by this group in Iceland. And what they did there is They got a sample of 12% of the Icelandic population, a representative sample of 12% very, very large scale study. Um, they they tested everything. One of them, And they identified the virus from from all the positives in that 12%. Not not. Not actually. All of some subset of them were positive. And then they sequenced the genome of every single virus. So the virus itself mutates all the time. The mutations mostly don’t change the function of the virus, but they serve as a sort of a fingerprint of, you know? So what virus you got? Um And so, for instance, from that you can tell, uh, like, I have mutation A and my virus. You have mutation and being your virus, I could have passed the virus to you. That’s possible, because you had to share a. But it’s unlikely that you passed it to me. And if you have a, uh, if I had a and you have B and I don’t have being, you have a then I couldn’t have passed it to you. Right? So based on a careful contact racing study with this representative population and this mutation analysis, they determined exactly who passed the virus to whom? For every single person in Iceland. For this 12% nicely. So and it turns out that this is absolutely shocking result. There wasn’t one single instance of a child passing you to revolt. Not one. A lot of instances of adults passing into Children, not one single instance of a child passenger to adult Children do pass the disease on, like other Children can get it from them. But they seem like much less likely to spread the disease than adults. And part of it is is that for the most part, when Children are infected, they get a mild form of this. Many of them are not symptomatic at all. If you’re asymptomatic, you can spread the disease, but much less efficiently. The diseases spread by droplets. Um, and if you’re sneezing and coughing on somebody, you’re more likely to spread it to them. If you’re amount of virus you have inside of you also, if you’re not having any symptoms there, I mean, they’re the so does turn out their studies that say that the kids have have the virus there. But it’s not causing symptoms that all islands spread it. So it’s not that they have some degree of immunity. They they’re able to fight it off earlier. Just that they don’t develop the symptoms. Yeah, but I mean, probably that’s because of their immunity right there. That’s one theory is that there’s like this cross reactive T cell immunity that they have it preferentially over adults. Um, that protects them from getting severe symptoms. I mean, they still can get sick, but not really sick, so that I think that’s one of the theories. In any case, the empirical fact is, they don’t spread it. And now, at this point now, no one’s done as careful study is that Iceland study, Um, but now they’ve been contact tracing studies around the world around kids, and they find monotonously the same result over and over in the UK, a single largest study found the same result. Ireland, Greece, uh, South Korea, which actually there was a There was a New York Times report reporting the opposite, but that when there was a second revision of that study that found that in South Korea the kids don’t pass, pass, pass the virus their studies of school openings and closings, which seem to have almost no effect whatsoever on disease spread. Schools are not the knights of the spread of infection. So what do we learn from this? The teachers are at more risk from other staff and other teachers than they are from kids. And, um, there’s a study comparing Sweden and Finland. Sweden. They left the schools open all the way through the epidemic. Uh, update 15. No, no, no mass, No restrictions, no nothing. Um, and what? They what? They What? They found there was no kids by the zero kids dead from epidemic. I’m going to quote here from from because I have the number is zero kids death out of 1.8 million kids going to school through the whole pandemic. Correct. Um, so I mean, that’s that is stunning, right? So it is, uh, it is. I don’t want to say that there’s no risk to teachers because there it’s there are risks. I mean said from other and you know, and I think we but we kind of know who’s at risk if you are older teacher, uh, you know that there ought to be some accommodation for people who have high risk? Absolutely. So maybe the older teachers can teach for the assume. Or they can. They can help the younger teachers who can be in the classroom together. They can get reassigned to places where there’s things like that I think are reasonable accommodations to account for the fact that risk, we should take the data and act on it in that way. But to to decide to not have in person school at all, I think is immoral. Yeah, I know it’s hard, hard not to think in those terms and because you mentioned something about the sort of short term interruption to school being problematic and causing health problems. The Children, right? The we have a large body of literature. Also economics about the long run impact of the summer on poor kids in particular. So our kids in the summer to get a bunch of enriching activities, they don’t lose any reading time, any math time, and so on, right? Any math ability before become lower income kids suffer a lot from the summer. So now we’re just said three summers in a row for some people. I mean, you know, I used to love summers, but, like, you know, the, uh, the the the the this summer was harder than within the typical. And I think, uh, the the inequality affect Carlos. You’re absolutely right. I mean, it is mind boggling. Uh, whatever progress we’ve made in closing those gaps, inequality are gone overnight. I saw this picture of the San Jose Mercury News, which just is, uh, it’s of two Hispanic Children. It couldn’t be more than seven sitting outside on a curb. It said, like, outside of a 7 11, with the little Google Chromebooks that their school given them because they didn’t have WiFi at home. They’re, you know, sort of leaching off the WiFi of the 7 11 or whatever Taco Bell. I mean, it’s just, uh how do you How do you square that with our commitment to reducing educational equality, inequality? I mean, you just can’t You have to say Okay, I care way more about what the kids well being. No, you don’t care about the kids Well being you care about something else if you’re going to allow that to happen, Yeah, you in the transfer is a transfer of a cost right to to, um to people that are benefiting. Might say that perhaps the very old and vulnerable benefiting from this at a cost that we typically don’t think about that transfer going that way so clearly. Right? And Carlos, we did the exact inverse is the right policy implied by the evidence We quarantine the healthy and the young, and we’ve exposed the old, old and vulnerable to the disease. You know, think of 40% of deaths for 45% is a very, very large fraction of deaths are nursing homes. Um, Jefferson might say that it was just like, you know, we didn’t know enough. Perhaps by the time we realized what was going on, nursing homes were infected already in a lot of places, I mean, I mean, to be fair, the United States, I mean, that’s the same thing, actually happened to Sweden like there was exposure in Sweden and nursing homes. Uh, what’s happened to many, many places around the world where it was in early days, difficult to protect. But I think now we know a fair bit about how to isolate. Uh, I think we should be using our testing resources there where we would actually make a difference. Like if you want to enter a nursing home, you should have a rapid test done on you. And if you’re if it’s positive, even if it’s a false positive, I don’t want you in there. I can’t tell. That’s right. That’s right. So let me let me go to the commission testing because then you wrote recently, uh, not bad at Wall Street Journal, where you basically advocate for stopping testing healthy and young folks. And that goes against a lot of a lot of I think, what’s been conventional wisdom, even a sense of Well, if you test, the more things are gonna be a problem. In some ways, if you think about, you know, testing could be akin to a vaccine. If everybody was testing themselves every day at home before leaving the house, that could be it could essentially work as a vaccine, right. Um, but but so why? Why go through your argument? Because I think that’s important to make the distinction. Okay, so I think that first we should actually take on that alternative because I think that has driven a lot of discussions incredibly misleading. That testing regime where we test everybody every day is not possible. It’s not in the feasible space, not just for technical reasons. Just getting the capacity to run that many tests is not technically feasible, but also just behaviorally. If I really have to go outside and to feed my kids, you know, to am I going to take that test? Is someone going to monitor me to make sure I take that test? It’s It’s not behaviorally consistent, like you’re going to voluntarily ask people to pay a cost where, especially where. Like they don’t feel sick. They just feel and they get a test. Now, if you did do these kind of things, you would have to have a once. These rapid tests that are actually have a fairly higher rate both of false positives, false negatives. I don’t I feel fine. The test says isn’t positive. If I miss work today, I don’t get I don’t know. I don’t get a paycheck. Am I really going to say Okay? Stay. I’m gonna stay at home voluntarily. You’re asking people to pay costs for an external benefit that is uncertain. It just doesn’t as a foreign policy, one of you to be very difficult to put a regimen like that. And I think we’ve focused on that because we think about, uh, test as this perfect thing that’s cheap and easy doing. My God, how can we? We, the richest country on earth, can’t get 100% of the population tested every day into into eternity. That has never been reasonable. And that alternative has driven a lot of policy. So, like this, this dogma of test contact rates and quarantine has driven policy. Now that dogma works in very limited settings, right? So it works in very limited settings. It works when there’s very few people with the disease, the disease in venereal diseases. It works great because the disease last pretty long. You don’t get cured of it by yourself. Um, you can test and trace you can. You can figure out who the full network of people you’re. You know, that’s exposed to exposure to disease and isolate. Treat those right. So that works great in that something when you have a disease that in the early days, 40 times more common than than you’re seeing in cases now, probably 10 times more common than seeing in cases. How does that work, especially when a lot of it’s asymptomatic? It makes absolutely no sense in that setting. Whole life, even the premise of contract. Tracy, By the time we realized what was going on, that was completely out of the back, right? Yeah, I think it’s useful for studies like the Iceland study and other settings in order to understand that ideologically, the nature of how the disease spreads. But there’s a very different thing than using it to control the disease, to control the spread of disease. I think even by March it was too late for that. The countries that were successful in controlling that we had done it very, very early on. But they were still very few cases, most famously South Korea. They found a super spreader event, identified everybody. They control this contact tracing thing to control the spread with this app. But you know what? The case is just come back. Their New Zealand famously identified every case shut down the borders altogether, this island nation, and there were no cases for 100 20 days, and yet all of a sudden okay, pops up, and now they’ve shut down again. This disease is very infectious. It’s very unlikely you’re going to be able to stop it from from spreading. The question is, who’s getting it? But risk is posed to them, and what are the costs of shutting them down? We also talked about false positives with the PCR test that I think is important, but even I don’t need that argument for this. If you test in college campus settings, you will find cases guaranteed. We found them. But what you won’t find is very many cases that result in hospitalizations and deaths. Kids are going to be kids. I mean, I teach college kids, and I was once a college kid, and, you know, I mean, that’s just normal developmentally for kids to interact with one. You can’t ask them to isolate themselves forever and behaviorally. Also, they, if they understand the risk to them, is incredibly low. Yeah, they’re not going to act on that, right? So, So more risk taking anyway. And if the risk is told to them to be very low then well apart. And we can’t lie to them. I mean, we just We just have an obligation to tell the truth, right? We’re basically saying, let’s test and then what we have. Okay, 10,000 cases on your college campus. What am I supposed to do with that political pressure and the media pressure is going to be to shut it down, right? Yeah. I mean, that’s exactly what we do with that. But the harm from shutting it down is probably is certainly more than the harm from letting keep going. Now, it might be important information, I think, for instance, for college professors we should take. I mean, I personally would be perfectly fine teaching, You know, in person. I’m actually okay with that. You know, I can understand professors who don’t want to take that risk. We should make accommodations just like we do with for high school or, you know, sort of elementary schools. We should We should have accommodation people who you know, for that fact. So, you know, teach outside if it’s if it’s feasible. If you’re not comfortable, then you find do zoom classes for for especially in cases are spreading. I mean, you can do this without denying. Are the kids who go to college an actual college, an opportunity for actual in in in person dorm education and anything that that’s that’s, That’s, uh, so far at U T. At least I think we’re looking at a prevalence. About 10% of the students, I think the test being conducted. Um, and so far, you know, the our administrators have been produced pretty steady on on like, Well, we manage this. This is fine. This is not a it’s not a problem. And meanwhile, the public health officials in Austin which don’t have control of our decision, is kind of an interesting, uh, your governmental like we live under the governor governor inside of the U. T campus. Uh, they’ve been, you know, yelling bloody murder for months on. No, you can’t open. You can’t have traded down. We had a football game this week here with 15,000 people in the stadium, and they’re like screaming at the top of their lungs. Perhaps that’s not a great idea, but but but the campus being open like unless there’s something, as you pointed out, unless there’s like a huge impact on the progression of disease elsewhere. Um, and there’s no evidence of that, right? And I mean, I think, as I said, I thought that there isn’t a symptomatic spread there is, but it’s much less efficient than symptomatic spread. So you know, you tell kids who are symptomatic stay in your dorm self quarantine. They’ll do it because they’re symptomatic. You tell an asymptomatic kid. Yeah, you got this positive. I’m feeling you’ve had no symptoms at all. Stay for 14 days and isolate yourself. I mean, it’s it’s harder, it’s much harder. And I think, you know, I think that like taking human nature into account when we’re doing these kind of policies is really important. And I think a lot of the policies are just It’s as if we’re going to automatically obey when we don’t. People just don’t like they are going to consider the costs and benefits to themselves very carefully when they’re deciding whether, uh, let’s go ahead. Go ahead. Yeah, just close it up, just, you know, so don’t don’t take more than just let me just close the loop on the colleges and asymptomatic testing, I think I think, um I think the balance that I wanted to return to the false positives because there’s actually now is some evidence the PCR test themselves have a false positive rate. So the PCR the way that the test works, is it, uh it amplifies the genetic material by doubling right. So you put a little bit of a you run one cycle, and it doubles the genetic material each time you run the cycle. If the virus is present or the part of the virus you’re looking for is president will double 248 16, 32 right normally, like you would run it until it comes. The virus comes up with detectable level. Now, if the virus isn’t present, you do the doubling zero times 200 times, 2 to 0. So you get nothing, right? It turns out, it turns out there’s an emerging literature that the number if you if you double enough times, you will magnify amplify non infectious viral particles that you will get essentially, even though it’s not a false positive. You get a functional, false positive positive that’s not actually likely to be infectious. So dysfunctional, false positives really important because it depends on the number of cycles that you do this. So if you go 32 to the 30th, it seems like actually turns out to be pretty good balance. You amplified things that are you’re still going to pick up some functional positive positives, but false positives. But you’re not that many. But after you do about 35. 37 you know that many cycles, it turns out, almost all if you, if you’re negative 2 34 and then the 35th picks up a positive that 2 to 34th picks up nothing. And the two of the 35th picks up something. You actually have almost 100% very, very high rates of functional, maybe 40 cycles. I mean, there’s some some The literature still is, uh, at least to be unclear, not exactly where that threshold is. A lot of the cycle times for the PCR tests are higher than 30 it throughout and labs throughout the United States countries. So I think it’s I think it’s Paraguay that seems to have done well with the epidemic that their cycle times or 30 in their labs are lower. And so it looks like there are not many cases because they’re not picking up cases, which were only shows that positive after 35 cycles after 35 doubling. Um, so I think there is some increasing evidence that there’s that there’s false positives from these p. C. R s. That’s true It it makes the strength argument against testing asymptomatic even stronger, I think, because the probability if you’re looking at somebody, the probability is low enough to begin with, right, you just like the probability of the posterior probability going even lower than than you think. Right? Which again, uh, if the decision making associated with that right to impose a huge cost without much of a benefit. That’s exactly what’s missing from from the calculation, right? So it sounds like a counterintuitive argument. The questions like, What do you do with information? Use the test to save lives. Use the test to make sure that no one enters at a nursing home and find if you have a false positive and you leave someone outside the nursing home, that’s better than than exposing somebody that balance the costs and benefits. There a point toward excluding people who have any chance of being positive indication of being positive. Uh, colleges? Can kids go 18 to 20 year olds hang out with one another that are asymptomatic, Even if I get some positives? I mean, I’m just not going to It’s hard for me too excited when it doesn’t result in in case people ending up. All right, so where we go from here, what do you think? What’s your sort of general diagnostics were? Uh, I know that’s that’s a That’s a That’s a difficult question. But like, you know, uh, okay, let me start with I use the word wrong. I use the word wrong. I think I think just with your forecast, I think, What do you think? You know, in terms of the disease progression or types of things from us, that’s that’s so there’s two schools of thought on disease progression. So one school of thought is that we’re not going to see a second wave. We haven’t seen a second wave anywhere yet. Really, What we’ve seen is regional epidemics with a big burst, essentially harvesting of, of of of lives. Unfortunately, uh, and then a decline. We saw that in New York, and the questions like how high does. The peak reach is, will that and and there’s one school of thought that says, Look, look at Sweden. They have had no second wave. Their deaths are down to zero. They’ve they’ve not locked anything down. If it’s going to run through, the population is kind of run through the population. They’re done. They’re done. Um, that’s one school of thought. Another school of thought is that look at seasonal. There might be a second wave. I don’t know enough to tell the difference between those two views. Uh, if you if you pin me down and say that the first school is more likely true. But I don’t have a lot of uncertainty around that, I just say, um, I think, um what we’ve learned about the lockdowns. Even if there’s a second wave, I do not think there should be a lockdown, not a general lockdown. We should use what we learned, which is we know who is vulnerable and focus the lockdowns on those populations and let everyone else live their life. Controlling community spread as a way to protect the vulnerable is too costly. And I think we’ve seen and I think we’ve seen that even in the very hard hit areas, the sort of, uh, we never overwhelmed the hospital capacity, which was one of the arguments in the beginning that well, we’re not gonna be able to manage this. The onslaught of cases and we think like Italy did you might make You might make some argument bad. There was, like, a few days or real bad. And then you went away, right? So so, yeah, it was not great. I’m not going to. That was that was, you know, not dramatic what happened in Italy or even in New York City, Uh, for for a week or so there, but but again, balancing through the cost we we just saw in Texas for the last since May as we reopened here. Yes, the wave came because we didn’t have it. We locked down before anything. The wave came and in fact, peaked much earlier than then. I think what you know, much more than any epidemiological studies would expect. Uh, and it’s hard to explain. Why would it going away, given that the behavioral changes are not in place. So, you know, I mean, so what they did in may, we heard immunity is a dirty word. What? What does it actually, So herd immunity just means that, uh, on the margin an additional case results in one or fewer additional cases so that the total case numbers daily are coming down rather than up. There’s not a single herd immunity number in that sense, because that has to do with the structure of the social network in which the disease passes to one another. So if you see cases declining, people start interacting a lot, you may see cases go back up and then, you know, then they’ll take more precautions that goes back down again. I mean, that that kind of idea about her one herd immunity number, But, uh, you know, sort of a dynamic response to the risk. Privately, I think that’s likely to happen. Continue to happen in this, Um uh, so I think that that in that sense, Sweden is not in her community. It’s given the set of social circumstance. So social interactions that currently have they have they have very little spread. Texas is on its way down, it looks like, uh, but it’s cases go up. Maybe maybe people change their social interactions again. And in Sweden, you can say that. Well, once winter comes in, people start being side. More people start being close proximity to each other more, and therefore the social interactions change. And then, you know the thing can creep up again, right? So I think I don’t think I don’t think we’re done with this in that sense. Um, I do think we have to learn to live with it, right. So we have to learn to live with it, actually live not just hide, um, which is what we’ve been doing. I think, uh, I think that that we have to understand that there is no safe alternative and that’s going to continue to be true. Now there’s going to be a vaccine, Uh, should be careful. It’s likely that there’s going to be a vaccine. It will be statistically difficult to prove that the vaccine works for deaths because the setting is the vaccines tested in a setting of declining cases and deaths. Um, so if we run the vaccine trial in a setting where there is no cases at all, you’re not going to find is a significant difference, right? Uh, it’s gonna be harder to find. You need much larger samples to find us. It is a significant difference. Um, so they’ll most likely be an intermediate endpoint. Um, you know, rather than you know. So rather than deaths may be cases or rather than cases, maybe protective immunity, t cell immunity or some some, some some sort of measure immunity. I don’t know. Um, the key questions where vaccines is, how safe are they going to be? And I think that that there’s there’s going to be and I just I just know from from, uh, personal involvement, that there will be a very rigorous process to see if the vaccine is safe. In fact, I think that AstraZeneca’s trial just got stopped because they found one case of transverse myelitis, incidental virus vaccine, um, platforms. So I think I’m like, If you’d asked me six months ago, would it be possible to get to this stage where we have all these vaccine candidates in in September? I would have said said that there’s no way. So in that sense, like we’ve done Yeoman’s work till I get to get get to this point. Um, I’m hopeful, but we’ll see the vaccine could change things, Especially if there’s a safe and effective, somewhat effective vaccine. I don’t think it’ll solve the epidemic, but it will make a lot life. It’ll be like, more like an H one N one. Let me start the conversation. And Ray J. Thank you so much for joining us. This is wonderful. Yeah. Thank you, Carlos. Pleasure. Take care. Thanks for listening to policy of McCombs. Yeah. Mhm. Mm mm