Jennifer Doleac Discusses the General State of Policing in the United States

[0:00:00 Speaker 1] Mm hmm. Welcome to Policy and McCombs, A data focused conversation on trade offs. I’m Carlos Car value from the Salem Center for Policy at the University of Texas at Austin. All right, so we are on. So our guest today is economist Jennifer Dahlia from Texas A and M University, where she is also the director of the Justice tax Tech Lab. Jennifer studies crime discrimination in various other aspects of the criminal justice system. Jennifer, welcome to policy. McCombs.

[0:00:39 Speaker 0] Hi. Thanks for having me.

[0:00:41 Speaker 1] So when we scheduled this visit was actually was checking on my calendar was over a year ago when we scheduled, scheduled to visit. And, you know, at the time I was very excited to hear about your work and your research. But since then, things changed quite a bit. And we are. We’re here. Maybe at even more even more excited about having you join us today because your your expertise, given a lot of the rhetoric out there these days, is something that, uh, really glad to hear from you. So let’s start with some big picture questions. Um, and you know, I don’t know how to ask this question in an easy way. But, um, how would you assess the general state of policing in the US In particular, I’m thinking about in regards to a lot of the current focus on proposals that the protesters had brought to bear since since May in particular. Like, you know, Are we over police? Are we under police who spend too much on police? Uh, is our police to violent? I don’t know what your general take on.

[0:01:35 Speaker 0] Yeah, great question. One that we have a good amount of research evidence on but could always use more. And they’re certainly topics within this that are understudied, I think, and where researchers are focusing their attention. So I think big picture, um, we know that there are too many incidents that unnecessarily escalate between police and local residents that unnecessarily escalate to either violence or even arrest. Too many people are arrested for kind of low level offenses where it doesn’t do any good, uh, to public safety, to to take those people and, you know, book them in jail. Um, And then, of course, the the incidents that have been getting a lot of media attention are those where, uh, someone is shot or killed, apparently for doing nothing wrong. Right where we have these videos that wind up going viral, where it seems very clear that the person was no threat to the police officer at the moment. Um, and so clearly there are problems. And I think you know, this is a somewhat challenging policy space to, uh, to discuss because the extent of those problems is more difficult to measure. And the reality is that we just don’t have good data on what police do and how much violence there is. Um, you know, there isn’t data on police use of force that’s collected in any systematic way. Uh, so I mean, my general sense is we know that, uh, you know, racial discrimination in particular is a problem and basically every walk of American life. And so it shouldn’t be surprising that it’s an issue in policing as well. Um, and so racial disparities in police use of force and and arrests are surely a problem. And there is some recent research evidence showing that that is true. Um, so so that’s one piece of it. Uh, the other piece that you asked about is kind of like what you know, to police, Are we under police are over police. And that question is also sort of complicated. To answer. Most of the research on this is really focused on what’s the impact of police on crime? And there is just overwhelming the evidence showing that hiring more police officers, putting more police on the streets, increasing police presence in general does reduce

[0:03:49 Speaker 1] crime rates.

[0:03:50 Speaker 0] So what we’re focused on is reducing crime. Then, uh, then the best evidence suggests that our U. S cities are dramatically under policed and we should be hiring more police officers. Um, but of course, that’s you know the

[0:04:03 Speaker 1] question. Let me ask you a question on that. So So the evidence of being under police is that a question on is a measure on the number of cops a number of number of actually people in the police force, uh, does that How about the money that we spend? Is that is that any indication that we spent overwhelming amount? They don’t have enough resources or actually don’t relative to maybe other places that teaches about the value of having more police do spend enough.

[0:04:29 Speaker 0] So most of the studies on this are focused on often money spent on police hiring. In particular, the biggest piece of police budget is salaries and pensions and, you know, and even training, which all like, you know, part of hiring more cops. Um and so, uh, I’m not sure that we have many studies on kind of like other pieces of police spending and to what extent they’re cost effective. But the biggest, the biggest expense of police.

[0:04:59 Speaker 1] I was just thinking about the sort of like the U. S. Police. Generally, when you look at a police car or police person in the U. S. Is overly equipped where they have, like, a bunch of stuff in here and where is like, I feel that elsewhere in the world is not so I would guess that we spend a lot more money on equipment as well. There’s, uh, in other places. Perhaps.

[0:05:16 Speaker 0] Yeah, we do. And that is something that you know, a lot of police departments get federal grants for those for that type of equipment, Um, or they get it for free through the 10 33 program. I believe it is, which basically is like, um, used to be military equipment but now is is kind of passed on to the two police departments and law enforcement when it’s no longer needed by the military. So I I’m sure police departments are also spending a good amount of money on that stuff, but they actually get a surprising amount of it, not from local dollars from somewhere else. Um, So, um, yeah, So there’s kind of this question about, you know, what are the benefits of all of that spending? And there were able to quantify what the benefits are in terms of crime reduction, at least the hiring piece. Uh, but there are costs to and that, you know, as economists like, that’s something we think about. But we don’t have really good measures most of the time of what the costs of policing are. There’s certainly lots of lots of qualitative evidence and ethnographic evidence suggesting that there are real costs to, uh, you know what people think of as over policing, um, and negative relationships between police and communities. But that’s something we’re just beginning to be able to quantify. Um, and we have to quantify it if we want to be able to weigh the costs against the benefits right? We have to have. We have to be able to put numbers on it. Um, and so there is some evidence that is beginning to come out. There’s an economist at U C L. A and moneywise. First is actually a UTI Ph. D. Who has a nice paper. Looking at the impact of putting more police in schools, she finds that when more school resource officers are higher, these police officers stations in schools, um, graduation rates fall, particularly for black students. Um, there’s another paper by an economist named Desmond Eng who looks at the impact again on I believe, also looking at, uh, like school completion, but of having a local school shooting or local shooting of of a student or young person on students who live in that area and finding the negative effects. Um, and this

[0:07:24 Speaker 1] is going different directions, right? We’re

[0:07:26 Speaker 0] going to go well, so yeah, so basically like it seems like there are big costs to policing, so that’s what these papers are trying to quantify, and that’s going to go in the other, you know. So there are costs, and but we also know there are benefits to having police officers there. Um, and so figuring out, you know, what’s the optimal number of police officers is something we’re just at the beginning of, I think, um, in terms of figuring out those costs to weigh against the benefits. But this is all to say. It’s complicated,

[0:07:56 Speaker 1] I guess that’s good for for for our

[0:07:58 Speaker 0] jobs.

[0:07:59 Speaker 1] Uh, so so now turned into somewhat more specific proposals that you’re here. So one of the sort of major crimes in the country has been, at least from a very vocal, uh, set of set of, uh, colleges protesters. But there’s a There’s definitely a vocal group demanding that police be defunded and and the funding police becomes this sort of major cry for reform. I think there’s also all sorts of different questions. Uh, yeah, demands for reforming police, but I think that one is the one that gets the most attention, and I think that you spoke about already about the fact that we don’t know necessarily what’s the optimal level. We know that more is better to reduce crime. That is clear from the data. We don’t know whether there there’s some externalities costs created by by that increase. Um, when you look at the proposal on the table right now in terms of I think BLM in particular, is a group that has put forward some very aggressive proposals on on on, uh, reforming police and defunding police. Do you see some good ideas in there? Do you see or at this point, you think that we just don’t have enough evidence to really be careful about about waiting the pros and cons?

[0:09:10 Speaker 0] I think both. I think there are some great ideas going around, and we don’t know which ones are going to work yet. So, anyway, it’s a really exciting time to be a researcher who who thinks about these issues, right? Because it’s there’s gonna be a lot. Hopefully there’ll be a lot of new stuff that’s tried, Um, that will then be able to see you know, what works. And the upside of our very decentralised criminal justice system is we have, uh, you know, at least depending on how you count 12,000 police departments across the country. And so if they all do something different than those are those are some great experiments that we can evaluate. Um, you know, so I think the defund the police movement is, um, you know, on its face is sort of a call for cutting in police budgets. Um, the actual proposals tend to be a bit more complex. Um, and really are pushing for reimagining what police are doing and not making police, you know, the only people we can call whenever we have a problem. So a lot of the the the calls the police departments get or for, um, situations that are, you know, surely better handled by people or social workers or medical professionals. You know, they get a lot of mental health calls and stuff like that, and you ask any police officer and they’re like, Yeah, we don’t want to go on those calls either. Like, you don’t want us there. We don’t want to be there. Were not trained for this. Um and so I think there’s a broad consensus that that, uh, you know, rethinking how you know what types of resources we make available for those kinds of situations is a good thing. Um, but frankly, it hasn’t been tried in many places yet, so we’re gonna need there’s going to be need to be some experimentation and iteration to figure out what works best. Um, I mean, the other piece of this that that sort of makes me a little bit nervous as I watch these calls across the country is often it’s going to wind up, you know, they’re they’re sort of the true underlying policy proposal that these groups are pushing for. And then there is what the politicians here in the moment and what they actually do to kind of get their constituents to stop yelling at them and that could wind up just being cutting budgets and then walking away. Right? And if we do that, then I think you know, the research evidence has shown quite clearly that, like, what we’re gonna see is prime rates go up. If basically police departments just cut their hiring and we don’t do anything instead, we don’t move that money somewhere else. Um, And if we don’t have a very clear plan, like, you know, the budget cuts can be very quick building up these alternate system is going to take awhile, and it’s not it’s not gonna be an overnight fix. And so, uh, even if we cut the budgets and say are well, next year we’ll put that money in the social services budget instead, or whatever. You could wind up seeing a big increase in crime rates in between. So making sure we have, like a transition plan is going to be really important, and I hope people on the ground or thinking about, But it’s not always clear from the public conversations.

[0:12:05 Speaker 1] So in thinking about that plan you mentioned, it’s sort of like opportunity. We have to run all the experiments and you, you you could study a lot of experiments that actually been put in place before. Uh, there’s a number of like R C T s like randomized controlled trials that have been done on things like Body Cam usage or or I guess that’s the one that comes to mind. But there have been others, right? And you’ve written a lot about about those Can you? I guess two questions one is that do you see more of an appetite in different localities to actually engage in the R. C. T. S and to, uh, what are the ones that we have maybe learned something about already? I think there’s a few things that are sort of like, potentially encouraging on on on some ideas.

[0:12:48 Speaker 0] Yeah. Um, yeah. So I think in general, there’s a lot of openness in the criminal justice space broadly, but especially within police departments. And engaging with researchers and running these kinds of experiments, uh, certainly varies from place to place. Some of them are open to it than others. But, um, you know, there’s some really great criminologist who for quite a while now, have been running experiments with police departments, particularly around things like hot spot policing. Like figuring out if it matters. We put this police officer on the street corner rather than that street corner like to what extent That police presence matters. Um and so those were some of the earlier versions of these randomized controlled trials? Um, that really started changing the culture of police departments and their willingness to experiment. And so again, there are plenty of police movements that are, you know, not interested and not on board yet, but it is quite encouraging. How many, especially big city police departments have research departments have, um, have data people who are there and ready to engage with academics. Uh, and so I always tell students like, if you have a great idea for something, you think the police, local police government should be doing differently, like start like start that conversation because it’s very possible that they’d be willing to try that. Um, yeah. So what do we know so far? So, uh, there is so body worn cameras, you know, it’s something that comes up every time we have one of these viral videos showing a police shooting. Uh, and there’s a call for police accountability. There was this real sense, you know, the last time we had this big national conversation and there was a real appetite for change. Um, there was a real push to have, uh, police departments around the country adopt body worn cameras and basically have every cop where a camera all the time have it turned on any time they’re interacting with a local resident, um, so that you know, we can go back and see how they how they acted and if they did something wrong. Um, and so unlike a lot of different technologies in particular, but police practices more generally, we actually have a lot of evidence on the impact of the body where cameras have been a lot of randomized controlled trials were randomly assigned some cops to where Cameron others not, uh, Turns out, the punch line is on that they do nothing. They do nothing to police behavior. Like if your goal with having cops where body worn cameras was that they would kind of know that people are watching and so be deterred from doing something bad that they know is wrong. Then we should see that in the data. We should see a change in their behavior. And we don’t see that. You know, there’s some variation across different departments. Some places, you know, behavior gets better. Some places, behavior actually get to worse. But on average, it’s just no effect. Um, and one of the bigger experiments was done in Washington D. C. A few years ago. Huge randomized controlled trial. It was like quite a feat that they did it and pulled it off. And it was just, you know, no effects across the board on everything. Um, and you know, when we can, we can talk about the possible reasons for this. But the punch line is that if your goal is to change police behavior. Body worn cameras are not a good investment, and they’re expensive. So it’s good to know, uh, you know, if that’s your that’s your goal, you should be spending that money on something else.

[0:16:00 Speaker 1] Huge costs no clear benefit right now that that question was whether there was any kind of changing behavior. You no known that changing behavior, but any change on accountability. So, for example, it might be the case that we don’t change behavior, but we never see a bad thing happening. Perhaps evidence that can be better used for either training or punishment. Or is this that an indication of that? Um,

[0:16:22 Speaker 0] So, um, you know, the those types of events are relatively rare. Um, and so it’s really difficult to pick up, you know, something, Uh, the use of that kind of evidence or whether the person was actually, um, you know, fired or charged with a crime or something like that In the moment, there just aren’t going to be enough of those situations. Um, but I think the way I think about this is that if cops on the ground have the sense that they’re going to be those kinds of consequences, then we should see a change in behavior. Um, if if it’s a conscious choice to be used force in the moment when they shouldn’t be. Now that, you know, it’s totally possible here that the cops who are using unnecessary force or behaving badly in general don’t necessarily know it’s wrong. They might, like, legitimately be afraid in the moment. And that’s something that maybe we should be training them differently. Or maybe those people just shouldn’t be cops, right? Like, I probably would not be a very good cop. I would not be like running toward dangerous. You know, not everyone is well suited to this very difficult and dangerous job. And if you’re the kind of person who, like, gets easily freaked out imposing on anyone, you probably shouldn’t be a police officer. So one thing that these that cameras could do is potentially help us identify who those cops are. Um, you know, the on the other hand, it turns out like there’s some research that is out there. That, like tries to, you know, use fancy machine learning techniques on, you know, existing data to figure out who the cops are. There any problems down the road. Turns out it’s not hard. It turns out that, like, if you get a lot of once, you’re on the job. You know, you get complaints past complaints from from citizens predict future bad behavior feature like real problems. And so the problem here is, uh, seems to be more about, uh, you know what to do with those officers. Once we’ve identified that, they’re going to be problems down the road. The problem is not identifying them, so the camera’s really aren’t necessary for that either.

[0:18:28 Speaker 1] Mhm. It’s okay. Body cams then it’s something that I think pretty much every well funded police department the country has adopted, Uh, some level of body cam, uh, where usage and and again we are seeing. No, no particular A particular effect there any doing of that after after the after the, you know, the findings, which is our cities. So we learned that the policy might not be very effective and it costs a lot of money. Any trend of moving away from it

[0:18:56 Speaker 0] or

[0:18:58 Speaker 1] or that learning has been sort of,

[0:19:00 Speaker 0] I think, um, yeah, not that I’ve noticed. I think most police departments are still and a lot of citizens are still calling for cameras. And I think you know, one way to think about this is that, um you know, what people are hoping for from cameras is not actually behavior change. What they’re hoping for is having the footage on file in case something goes wrong and they have. Then they can prove it right that they can prove that something. And so it’s sort of like again sort of preparing us for the situation. You described where that footage can be used to get that cop off the force or to charge them with a crime or something. Uh, now, you know, we hear plenty of stories in the news that suggests that is not done very often currently. But you can imagine, I think a lot. There’s a lot of appetite for moving in the direction where there is more accountability for bad behavior. So I think there’s a sense of like cameras provide transparency, and even if they don’t change behavior on the job or in the moment, um, they might still be worth it, because we could still have the footage and sort of the proof that someone shouldn’t be a cop anymore. Um, and, uh and that’s a totally reasonable thing to want to pay for. We just need to be honest with ourselves. But what we’re paying for, uh, not not kind of hoping for stuff that we’re not going to get.

[0:20:16 Speaker 1] And it also points out to something that is is difficult for us in our research right, which is Sometimes we try to focus on short term outcomes because that’s what we have. You know, if I think about what is the effect on current use of force by having the body cameras or not? Right now we have one RCT. We check, we don’t see anything. But perhaps there’s, you know, 10 years down the road to sort of continue where usage of those cameras might lead to more indictment of people that need to be indicted, even more evidence in court for even the criminals, not only against the cops that do bad behavior, but the availability of that evidence might be useful for having the correct outcomes in the criminal justice system later on. And that’s very difficult to measure and wait for and so

[0:20:54 Speaker 0] on. Yeah, yeah. So I mean, what you’re describing is sort of like culture change, right? They can start ushering in sort of culture change. It’s very difficult to quantify in the moment. I guess I think of it. Uh, this situation is being a little bit less about kind of short run versus long run and more about, like, measurable versus much more difficult to measure. Like what I was thinking, I was thinking about something

[0:21:15 Speaker 1] measurable in the long run as well. Which is which is, you know, that’s now you have perhaps better evidence or certain things. So whenever you’re trying to adjudicate an issue in court later on,

[0:21:23 Speaker 0] those

[0:21:24 Speaker 1] footage might be helpful in terms of whether it’s a cop, it’s in trouble or as a criminal, that’s in trouble. And we’re trying to figure out the put this person in jail

[0:21:31 Speaker 0] for how long?

[0:21:32 Speaker 1] Even all the sort of the implementation of the justice part. Now, after somebody gets arrested, my improve as a result of this, and that’s that’s measurable to some degree, but it’s probably you’re going to see in the long in the long term, right?

[0:21:44 Speaker 0] Yeah, and probably is, uh, you know, a big piece of this is going to be how it interacts with other policies, like given current policies and current practices. It appears that body worn cameras don’t do anything or don’t have any benefits ever hoping. But, you know, even within a very short time, feel like there’s a lot of ongoing policy conversation about changing those policies and practices. And so maybe if we do, you know, a similar RCT next year a body worn cameras under a different set of policies in a different set of accountability practices, you could see very different effects on behavior, and that would be really cool to study.

[0:22:19 Speaker 1] Yeah, I think you point out one interview that I heard with you, uh, that a lot of the sort of, uh, sort of things that go viral in terms of police bad behavior. Something like that is also coming from cell phone. So we have we have a solution, are important, like everybody has a camera everywhere all the time, right? So in some ways, that that might be it might might be actually contributing to the fact that, um, the effect might be no, because, well, you know, people know there are cameras everywhere in D c you walk around you being filmed 100% of the time. So

[0:22:48 Speaker 0] absolutely, whether it’s a cell phone camera or just like security cameras around the city, Absolutely, like I think most of us are probably just you don’t even notice the cameras anymore. We just sort of know that. Well, probably if we’re out in public, there’s no presumption of privacy at this point.

[0:23:05 Speaker 1] So So just wrapping up on on this notion of the sort of, like police reforms and ideas, anything else that you can remember or you can point to, that that seems encouraging at this point that that, um, we know something about

[0:23:16 Speaker 0] Yeah, So there’s some really good evidence from changes in practice in, like hiring practices back in the eighties early nineties to increase the diversity of police forces. Um and so, uh, those basically they’re a bunch of court mandates that ain’t that required affirmative action that you know or required in some way that police departments increase the number of black officers and, uh, female officers. And both of those moves not only led to more hiring of black officers and female officers, so, like the court orders worked and the departments are able to find qualified people. Um, but it actually improves the policing. So, um, so crime rates fell in those communities, and people were more likely to report crimes against them to the women were more likely to report domestic violence and so on and and black residents were more likely to report if they’ve been victimized by crime. When there were more officers on the force that looks like them. Um, And I think you know, this kind of makes sense if you think that a big piece of this is just trust between police officers in their communities and if people you know have a sense that you are going to care about them and the types of problems that they have and they’re getting more likely to not only tell you about bad things that happened to them but probably cooperate with you to help solve those crimes and keep them and happening in the future. And so you know that evidence from the past does not automatically mean that if we encourage more diversity now we’re going to see, you know, similar effects. Maybe, you know, already we’ve got police departments that are much more diverse than they were in the past. Um, but I think it’s worth trying. And there’s some really there’s a researcher named Elizabeth Lee knows that u C Berkeley, who does some really need, uh, field experiments with police departments to around hiring and basically just trying different messages to get new and different people to apply to police forces. So if we think part of the problem here is that we want different people than currently are signing up to the an officer, um, or even just more people to choose from so that we can be choosier and we can fire some people without worrying about how we’re going to fill their spot, Um, we’re gonna need to run those kinds of experiments. And she’s finding that actually, there’s some really good strategies that really aren’t hard. It’s just a matter of changing the way we talk about the role of a police officer.

[0:25:31 Speaker 1] So that’s that’s that’s encouraging and and something that you don’t hear very often. I think you’re talking about before right, the discourse between what we can learn and know from the data versus what politicians are reacting to and how things are portrayed in the media are sometimes they are different.

[0:25:47 Speaker 0] Yeah, let’s let’s let’s get

[0:25:48 Speaker 1] to To To your work and the justice tech lab that you run at A and M tell us a little bit about, I guess, your research agenda generally, but also in particular, what the justice that lab is

[0:26:01 Speaker 0] sure so so broadly. I studied crime and discrimination, but within that, I’m very interested in the impacts of technology and public safety as well as prisoner reentry. What happens? People when they come out of prison and how to help them reintegrate into society? Uh, and those might sound on their face like very different topics. There is a decent amount of overlap, but I’ve kind of come at them for different reasons. Um, and the justice tech lab, you know, really started based on my initial interest on the role of technology in public safety. But what has expanded, um uh, to to include research on a variety of criminal justice topics, um, particularly on, you know, interested in, um, uh, increasing fairness and effectiveness of of our policies in the criminal justice system. And so I’ve got a great team of research assistants and students that work with me, Um, and, uh, doing a variety of projects that are, you know, often often we’re going out and trying to get data that people might not have made public in the past. And, um, and trying to use those data to understand what works in communities across the country. And so that’s that’s what those

[0:27:08 Speaker 1] those students and our is mostly economists or or a variety of different backgrounds,

[0:27:14 Speaker 0] mostly economists. Yeah. So, um, you know, I try very hard to find whether it’s undergrad or these pre docks, uh, recent college grads that are interested in getting a PhD trying to find people that are interested in a career in academic research. Um, and they may not want to be economists. They might want to go into public policy or be considering a variety of options. But for the type of work I do, having some sort of economics background is usually help.

[0:27:42 Speaker 1] We’ll talk a little bit about that later on, but continue here on the on the on the sort of technology aspect of the types of things that interest you in terms of technology effect on on on the criminal justice system. One of the things that you that I don’t know more interesting these days, they’re having more of an impact. You know, that makes us maybe hopeful that that there’s there’s lots of improvements to come.

[0:28:06 Speaker 0] Oh, um, I feel like so much of my job right at this point is about like, uh, bring the bad news to policy conversations like, actually, this technology isn’t going to work as well as you think it is, but But I Questions like this remind me that, like, you know, one of the reasons I bought into this the reason that I, you know and study technology in the first place is it’s just so cool and exciting to think about the potential

[0:28:30 Speaker 1] work. DNA works very positive results.

[0:28:32 Speaker 0] Yes, yes. One place where actually do find very big benefits. Um, bigger than I think. I certainly expected going into this. This line of work is in DNA databases. Um, and so, uh, just a little bit of background on this policy, Basically, every US state, um, and many countries around the world have databases of criminal offenders, D n A. As well as crimes. Uh, DNA samples from crime scenes. Um and so basically, the idea is that if you are charged or convicted of a particular crime, and this is depends on state law which crimes are included here, um, you’re required to provide a DNA sample and then the government uses analyzes that DNA sample just to provide an identifying string of numbers. It’s just just to identify you. They’re not doing any sort of like, um, you know, checks into your health or anything like that. But then then they upload, upload that identifying string of numbers, the database, and it’s compared with strings of numbers from the crime scene data. And so the idea here is that if there’s some sort of cold case out there where they haven’t been able to identify you as a suspect and you committed that crime, the DNA will match you to that. And you’ll be more likely to be caught in these cases where you would otherwise would not have been a suspect, um, for police, and so this would increase the likelihood of getting caught

[0:29:47 Speaker 1] for your crimes.

[0:29:48 Speaker 0] As an economist, I think of that as a kind of a very standard Gary Becker style model where, uh, that I should have a big deterrent effect on criminal behavior. If I know that I’m more likely to get caught, this increases than the expected cost of committing crime. And I should therefore commit less crime because it’s more costly. Um,

[0:30:07 Speaker 1] that’s when you say Gary Becker type is like, you know, assuming rationality on the on the involvement, which was something that was very controversial and first started the idea that an economist could study crime, right? Like no criminal is not going to be thinking rationally. Well, it turns out that it do. Incentives matter almost everywhere.

[0:30:21 Speaker 0] Incentives matter, even for criminals. Yeah, uh, not necessarily every criminal, right? There are lots of people out there who commit crimes because they’re drunk or high or something else, and they’re not being super rational in the moment. But even the decision to go get drunk or high with your friends if you’re no, you’re you know there’s some likelihood you’re going to get into trouble. That is a rational decision. The decision to kind of put yourself in that situation. And so, um, yeah, so it turns out people respond to incentives and they respond to this incentive in particular. So I have research from the U. S. Where the data is much worse. But it kind of did the best I could with it and found that, uh, for those who are who are, you know, charged or released just just on either side of the expansion date for for DNA database. So a database a state decides to add convicted robbers, you say, for their database. If you’re convicted of robbery just before that date, the where the law goes into effect, you don’t go in the database. If you’re convicted just after the law goes into effect, you do go in the database. Now you have very similar people who, just by the wreck of it’s essentially random, whether people in the database or not, it’s a beautiful national experiment that economists love and so that I can compare those people over time and see what happens. And it turns out that the person who goes in the database is much less likely to re offend, are going forward. Um, and so that was, you know, saw that in the U. S. Data, but again, the data super messy. And there were a whole bunch of caveats to that. Uh, and then I wound up linking up with some economists in, uh, in Denmark, where they have much better data but very similar policies and studied, Um, a big database expansion there. And we find with their much better data, we can have a much, much cleaner measurement of this. We find that people who go into the database are 40% less likely to be convicted of another crime than the people who, you know, were charged just before, Uh, and don’t go in the database. So it does seem to have people. People do seem to know that this,

[0:32:23 Speaker 1] um and I assume that that the number is controlling for some sort of level of service they received by being arrested the first time. I think that there’s a sort of like results coming out of this kind of neighbor. For example, you go to prison is kind of behavior. You come out as a much better citizen as, for

[0:32:39 Speaker 0] example, quest,

[0:32:41 Speaker 1] right? So they figure out that system and I don’t know if they figure out there are other issues, but that’s all controlling for that, right? So that this decrease

[0:32:47 Speaker 0] is these are people who have been charged at least once before, so they there has to be an initial charge that puts you in our sample. And then and then those people like, based on your first charge, you’re either you’re convicted or whatever in the deep going in the database doesn’t affect that. So all these people are similar on all those types of dimensions. Um, but But the people charged

[0:33:08 Speaker 1] just after the

[0:33:09 Speaker 0] expansion date their identities at the database so that going forward, they’ll be more likely match the crime scene evidence. And they’re the ones that we see if they drop in recidivism for

[0:33:18 Speaker 1] its 42% which is like

[0:33:20 Speaker 0] a huge large. I mean, I just Yeah, I mean, this is just one of those papers were like we I’m not sure we believed it at first. It was like, Let’s just go at this every which way and and make sure that our results are robust to like anything we can throw at it, and the check that we do that I find the most convincing is we kind of run do the exact same thing we do in every other other year like, let’s pretend that this policy change It happened. You know what actually happened in 2005? Let’s pretend it happened in 2000 and 4, 2000 and six or 2007 or 2003 and run our exact analysis. And maybe there’s just something weird about the structure analysis that picking up a fake effect, right? And it turns out we just get, like, really precise zeros in all those other years. And then we get this huge effect in 2000 and five. It’s like, Okay, there’s something like actually happens.

[0:34:11 Speaker 1] Yeah, um I just thought about a question that who was it? Uh, so yeah, so So you say 42% Is the relatives that sort of relative risk? Right? But but what’s the baseline of recidivism? And in Denmark, anyway,

[0:34:25 Speaker 0] to start? Oh, my goodness, I don’t know, off the top of my head, but it’s not that different from in the U. S. Um, people do still re Synovate at pretty high rate, the high

[0:34:33 Speaker 1] rates, right? So therefore, the 42 is actually very important because we’re talking about something very low to begin with. Who cares right, But no, it’s actually pretty high. So the 42 becomes very, very relevant,

[0:34:43 Speaker 0] right? And we can actually see. And it’s a much less clean experiment. So we don’t, uh we don’t write home about this result, but we can actually plot out reported crime rates in Denmark through this threshold. And if you know, if this effect is real, we should see a drop in crime. And we do see a drop in reported crime starting in 2000 and five. So, um, you know, there might be other stuff going on that’s contributing to that, but it does. It’s consistent with this idea that, like, this is a real change in behavior that’s improving public safety, Um, and is in line with my US data had also shown like there was a reduction in crime rates when this stuff happens. So it’s

[0:35:16 Speaker 1] a big benefit. Big benefit of these policies of expanding databases of DNA material. What are the costs? I mean, how How about the costs of this, how how expensive they are? Are there any costs that you worry about having those databases available?

[0:35:29 Speaker 0] Yep. So, uh, so there’s certainly financial costs. Um, the financial cost of setting up a database and setting up all the crime labs that you need are you know, it’s somewhat expensive. So it’s not. It’s not cheap, but at this point everyone’s got those labs. And so the question is like, you know, and the questions I’m looking at in this papers is what happens when you add a marginal offender. You add one more offender, you add one more group of offenders, and so they’re The costs are really cheap. So it’s like, You know,

[0:35:57 Speaker 1] you have infrastructure in place. You’ve

[0:35:59 Speaker 0] already got the infrastructure. So it’s really just the cost is a saliva swab and like putting it in a machine. Um, at this point self standardized. And so um so the financial costs are really pretty negligible relative to the big benefits that we’re seeing in terms of reductions in

[0:36:13 Speaker 1] crime. The

[0:36:14 Speaker 0] cost that people talk about, um and that I think are are more more relevant to policy conversations is the perceived privacy cost of these things. So the government is taking your d n A and analyzing it in some way and probably storing it, um uh, in most cases, because, you know often the standard. The FBI updated standards every once in a while to make sure you know, we’re at the cutting edge technology will have to re analyze all the DNA again. Um And so So, you know, depending on your level of trust of the government, you might not want them to have a sample of your DNA on file. Uh, and even though under current law, they can’t do anything with that DNA other than just come up with identifying string of numbers, some people do worry about sort of a slippery slope where? Well, now they have it Imagine. You know, it’s not that hard to imagine that in 10 years something happens where they start analyzing all the d n A. Or to see who’s predispose to schizophrenia or substance abuse or other things that can predispose you to criminal behavior. Um, you know, my general view is like, you know, we’re we’re not there were not there and, you know, we got to look at the current law, and we do have a choice about whether to to cross that line or not. But at the same time, I think you know, there are lots of other technologies out there that are quite invasive. Uh, you know, putting cameras everywhere, for instance, or having people were GPS monitors or blood alcohol content monitors or, um, yeah, just a variety of stuff like that, Uh, and so relative to, you know, having a camera, having cameras everywhere, having someone’s d n A. And just getting this identifying string of numbers off of it to me seems relatively non invasive. But people have very different perceptions of this, and, um, and so reasonable people can disagree. And unfortunately, it’s something we haven’t figured out how to quantify, yet kind of what those perceived privacy costs are.

[0:38:06 Speaker 1] Yeah, I think one of the cost is the misuse of the information as well. And I think that the prosecutor’s fallacy of a match you need to understand what the probability of that person committed a crime given the match versus the other way around, which is something that people there’s a lot of problems with that when you have a match that you know, depending on how the match come about, Uh, if you find a match in and it turns out to be something on the other side of the country, There might be the probability that the actual match is much smaller than something that No, no, no. You live in the house and like

[0:38:35 Speaker 0] the example,

[0:38:36 Speaker 1] right,

[0:38:37 Speaker 0] Uh,

[0:38:38 Speaker 1] probability that

[0:38:39 Speaker 0] an issue with DNA evidence more broadly, right, Uh, DNA database and thinking about, like, how do we understand the problem with

[0:38:47 Speaker 1] the ability to scan DNA databases all over the place for for like, I think that there is a family issue of situation that I think we have seen some issues in in England. I think about about searching.

[0:38:58 Speaker 0] Yeah,

[0:38:59 Speaker 1] that’s what I think, something that could lead to more false positives. Uh,

[0:39:04 Speaker 0] yeah. So they’re essentially looking in those cases, you’re essentially looking for a partial DNA match, and so you’re just gonna you’re gonna cost a much wider net. And most of the people are not going to be the person who actually committed the crime. Um, yeah, there to kind of, you know, I think people who are thoughtful about this it’s like, you know, in any given case, the DNA evidence should never be the only thing that gets you right. You also have to show that the person you know, doesn’t have an alibi and was there, you know, all the stuff. And then also like, the d N A. Just in general, like DNA can prove that you were in a place. Um, but it can’t prove you committed a crime there, And so there is still other stuff going on. But that’s

[0:39:42 Speaker 1] the problem of of of, uh, you know, sort of creating a sense of the expert. The scientist says that this thing is infallible, and it’s not something that we have to put into context. And so that’s a cost. It’s more like educational. How to use this information program not necessarily cost information. How to process that.

[0:39:59 Speaker 0] Yeah, and that that is a super interesting aspect of all of this. With with forensics, I think more problems, especially when there are a lot of forensic DNA. I think it’s like pretty solid, but like there are a lot of forensic sciences that are not real. Science is, uh,

[0:40:15 Speaker 1] random Ballistics are pretty bad, right? Until

[0:40:18 Speaker 0] it turns out, Yeah, it’s just like it’s entirely visual, like people think of it as being, you know, you could imagine a world where we do this fancy machine learning algorithms. Maybe over the last year, someone’s done that. But like, as far as I know, that is still not the

[0:40:32 Speaker 1] way.

[0:40:33 Speaker 0] And, you know, the most recent thing I heard on fingerprinting is eventually, like, we just we don’t even that worried because within not that long d n A. And like, the analysis of touch DNA to be so much better than when we take fingerprints were actually not going to look at the fingerprint. We’re gonna be taking DNA. So, like, forget the problems with fingerprints because it’s just gonna be obsolete in a couple of years. But yeah, like fire science, like all this stuff is just, like, kind of nonsense. Um And so, uh, yeah, thinking about how you know. So there’s a science piece of this and the conversation about what should happen and what You know what is effective, like in terms of just like for scientists in the lab. And that’s new, not the kind of scientists I am. But then there’s also the piece of how it’s portrayed and understood in a courtroom, both by a jury and judge, and and they’re, you know, even just like basic probabilities aren’t necessarily understood well by the general population. So it all becomes much more complicated in that context.

[0:41:37 Speaker 1] I guess it’s a good segue for us to get to your paper that you’re presenting today at UT, um, on the role of of sort of machine learning ai techniques in predicting recidivism, uh, and how that gets used in the criminal justice system right now. So So tell us about that.

[0:41:53 Speaker 0] Yeah. So I’m talking about risk assessments and criminal sentencing context in particular. Um, and yeah, there’s a lot of interest in whether, uh, um, machine learning in particular, but just sort of like computers and and regression technology and just

[0:42:11 Speaker 1] probably just

[0:42:13 Speaker 0] even a better basic progression could, uh could help us make better predictions about who’s going to go on to re offend. Um, and that could help us, perhaps use incarceration in a more effective way. Um, you make sure we’re only locking up the people who are highest risk and also potentially, you know, if we do this in a standardized way across all defendants, uh, use incarceration in a more and and sentence in general more fairly across different people. So we don’t have you know, a judge treating the black defendant differently. The white defendant, because they’re using the same risk score. You know, that’s calculate the same way for both of them. You

[0:42:48 Speaker 1] re right a bit Rewind a bit. So typical of the people might not necessarily know how these things work. Right. So you go to court, you’re charged with a crime, you get convicted now and now a judge has to make an assessment decision on whether to whether to put you in jail or not, or how, for how long and so on whether to release you on early. There’s lots of decisions the judge has to make right. And at that point, the sort of before these techniques were available, the judge would just look at evidence. May be from the case, maybe some some depositions from family members, circumstantial evidence about that person to make a consideration. And we’ve been trying to automate some of that, right?

[0:43:27 Speaker 0] Right, So they have a lot of information. So there’s sort of a question here about, like, does this actually provide more information? Like often the judges have a big stack of papers. There’s some sort of pre sentencing report that could involve,

[0:43:38 Speaker 1] um,

[0:43:38 Speaker 0] interviews with the person, psychological analyses, maybe testimony from yeah, the witnesses or family members or so on. Um and so And they know their criminal history, obviously. And so they are deciding, You know what sentence is reasonable for this person standing in front of them. Uh, but there have been some some policy simulations that show that if we just had, like, um, if we had a computer, use that data to run to run a regression and predict the person’s risk And just, you know, allocates allocate incarceration only to like, the most risky people based on the data. If we did that instead of having the judge locked them up and you know, whatever order they think is appropriate, we would actually, uh, it appears from those simulations, like, the judges routinely make mistakes in who they lock up, their routinely locking up people that are much lower risk and would not have gone on to commit crimes. Um, and so some of these studies have just, like, really eye popping results that you could, you know, you could reduce crime rates by 25%. Holding incarceration constant. You can reduce incarceration rates by 40% with no changing crime rates. You just let the computer decide instead. And so that’s sort of like the motivation for all this that maybe, like people are just really bad at making predictions and were easily distracted by information that is relevant. Uh, you know, So I’m hungry or my football team lost this weekend or something like that, and that puts me in a bad mood. And so I take it out on the person in front of me. Um and so we know that’s true. There are studies showing this is true. This is like, not a surprise to people who study psychology. And so maybe the computer can do better. And that’s the motivation. Um, so in this paper with Meghan Stevenson, we used data from Virginia where they implemented risk assessments and criminal sentencing for nonviolent offenders back in the in the early two thousands, um, and and study like what actually happened in the real world, where we implementing these things. This isn’t just looking at a policy simulation. We’re not looking at just how accurate the prediction is or whatever were saying. Let’s see what happened when they’re used in the real world the way they’re actually used. Like the judge now has one more piece of paper in front of them that has the risk for and has a recommendation for diversion or not based on that risk for and what do they do with it? And it turns out that they pay attention to it, but it doesn’t lead to any of the benefits that we had hoped for. And the punchline really seems to be that most of the the advantage of the risk assessment is it, uh, identifies all of these young people as being very high risk because it turns out ages like one of the best predictors whether you’re going to go on to commit another crime. Um, young men in particular are very high risk. And so the computer says, Hey, here all the young men will go lock them up. And the judges, for good reason, don’t want to lock up young people. There’s a long tradition in the legal system for leniency and, uh, for young people, because there’s a general sense they’re less comfortable for their crimes. The brains aren’t fully developed, and so on, and so they don’t lock up those young people. They basically just ignore the risk assessment most of the time. Um and so we don’t see the big gains that people like thought we would get. Um, but it’s for this very good reason. And it suggests to us that the judges aren’t actually making mistakes. They have a different objective in mind. Um, and you know, as economists would say, they have a different objective function. They’re optimizing a different objective function. And so it really points to this bigger picture for criminal justice reform. It really points this idea that if we want to change the behavior of the actors on the ground in our criminal justice system, we’re going to need to understand what currently motivates them and what their current objective function is. And until we understand that, we’re not going to know how to change their behavior. And so this is a perfect example of the Legislature Was like, we’re going to get the specific goal of this policy was to divert the 25% lowest risk, nonviolent offenders, incarceration, and there was no change in incarceration rates fall because the judges just said like no Thanks. I don’t want to lock these people up. So, um, are they, you know, they just didn’t do what the risk assessment said. So, um, yeah, so it’s a bit of a bit of bad news added to the conversation, which is which was much more comfortable.

[0:47:56 Speaker 1] So there’s some press, and I haven’t followed this very, very closely. But I know that there’s a lot of people study now, the sort of potential problems with the algorithms themselves of how how very biased they are. And And my understanding, for the most part, is that is that the bias associated with them is not with the mechanisms or the algorithms in place is more like with the data that we gather to train the algorithm

[0:48:19 Speaker 0] and so on. So

[0:48:20 Speaker 1] did you. Do you have anything to say about that? What? Your understanding generally, about that part of the picture here?

[0:48:26 Speaker 0] Yeah. So they’re Yeah. As you said, lots of computer scientists who were thinking deeply about these issues. And like, if there are ways how one can deal with kind of biased data, um and, uh, kind of like the basic example of this that we would worry about here is that if we know that black men are more likely to be convicted, arrested and convicted for committing a crime and then a white man who does the exact same thing as and we know that again, there’s racial bias in the system. Um then an algorithm that is trying to predict future risk is going to accurately say the black man is higher risk because he’s more likely to be convicted of a crime going forward, right? That’s not because he’s more likely to actually re offend. It is more likely to be convicted conditional on the same behavior. So So we worry about, you know, both for future behavior. And the same thing happens in the past. The black man also has a longer criminal record for exactly the same reason. Right? Um and so So we worry about those kind. That’s what people mean when they say that there’s there’s this group. There’s bias kind of baked into the data. Um, the way Meghan and I approached this is again as economists, social scientists who said like Well, you know, people are biased to the judges are biased to, and they have all the same information that the computer is using. So the question here isn’t. Is the algorithm biased or is the data biased? The question is, is it more or less biased? And the judges and what actually happens when we give the judges information? And so it turns out, when we actually give the judges information when they implemented risk assessments and sentencing in Virginia? Um, there was no impact on racial disparities in Virginia. Now I went into the study really actually expecting it to to reduce racial disparities. I was one of those people who was, like, really bullish on the potential here to reduce bias by judges. Um, and it turns out, I’m disappointed there was no benefit. But for those who are really worried that these risk assessments are going to do real harm and widen racial disparities, it turns out there too pessimistic. And the judges, they’re basically just do justice just as badly as the judges do. Yeah,

[0:50:33 Speaker 1] it speaks about about the sort of, uh, general notion of trade offs on the on the so the information is there, and people are trying to make the best assessment of information. Whatever it is, the way right and and whether to to not focus on a particular new tool. In this case, the result was no. But but like the possibility of the new tool, potentially reducing problems even though it’s not perfect, it’s something that’s not thinking through the trade off very carefully.

[0:50:56 Speaker 0] You think? Well,

[0:50:57 Speaker 1] you know, if it’s not perfect, I’m going to do it like, Well, that’s not the game. The game is margin improvement and and thinking on the margin is something that you and I are trained to do. But it’s not what I think. A lot of folks in charge of the decisions are right, so I can think about the same situation happening in in banking applications. I mean, there’s just sort of like I think there was a What was it? There’s a company that tried to, uh, sort of optimize whether to give student loans are not based on data, and they got some grades

[0:51:25 Speaker 0] from people and so

[0:51:25 Speaker 1] on and then fit folks like Oh my God, that’s terrible because leading to bias outcomes, uh, in terms of race is like well, is it any more biased than filling out the fastest score and going to the bank and whatever. Like they have some information, just crunchy differently and access to that information. If the system is already has contained, some biases might be actually reducing. Get improvement. Don’t stop them from doing it.

[0:51:45 Speaker 0] And we do have examples like I have previous research on Ban the box. But

[0:51:49 Speaker 1] I was going to say I was going to go there

[0:51:51 Speaker 0] American kind of motivation here, Which is part of the reason I was really optimistic about the potential of these tools in this context. So the question, you know, uh, there’s sort of this general human impulse. I think we have that, like when we know that particular information is biased, Uh, then, like, we know that, like, you know, your past criminal record probably includes some racial bias because it includes how police officers treated you in the past, Then a lot who are going to say, Well, just don’t let them look at the previous criminal history anymore, or in the case of and the box don’t don’t let them don’t see. Let them see the criminal record until the end of the hiring process. Um, and that is sort of like our first impulse of a policy. But as economists, we hear about that, and we think, well, but what are they going to do when they can’t see that information anymore? They’re probably, you know, you haven’t changed their incentives. They still don’t want to hire people criminal records if that was their initial, you know, uh, goal. And so now they’re just going to try to guess. And if people if black men are more likely have criminal records than any black man they see now, they’re going to guess he’s probably got a criminal record. So it actually improves employment outcomes to provide that information because the black men who don’t have many, many black men who don’t have

[0:53:06 Speaker 1] criminal records can

[0:53:08 Speaker 0] signal they’re clean record and they get a job. But when you take that information away, none of them get jobs. And that’s basically what we find in the box research and was totally possible in the risk assessment context to like maybe even if these algorithms are biased against black defendants. If in the absence of this risk score, the judges assumed even worse about the black defendants than providing this information could actually, uh lead to some of those black defendants having better outcomes. Um, it’s not what we want up seeing in practice, but it was possible.

[0:53:43 Speaker 1] Yeah, and and and that’s sort of like mindset of unintended consequences is again something that people they are putting forward the proposals of Ban the box, for example. I think a lot of those they came out in and even through ballot initiatives in some places, right? Um um, it’s just like I think that a lot of those votes were coming from a good place, but

[0:54:00 Speaker 0] absolutely, but

[0:54:01 Speaker 1] but without thinking through the fact that you know there’s, there’s a you’re creating a problem with information and I might create actually more severe problem. As a result, there’s a good motivation for an employer to say maybe good or bad, But the point is that I understand somebody depending some jobs, saying I don’t want to hire somebody, the criminal record and, um, they’re not going to stop thinking about that by the by the bento box,

[0:54:22 Speaker 0] right? And much like I said, you know, the you know, my takeaway from this paper about criminal justice reform is we need to really understand what the objective functions are of these actors of the judges, and the prosecutors are actually using this information and doing something we don’t want them to do. If we want to change their behavior, we need to fully understand what’s motivating them. The same is true of the employers in the ban the box situation. If the goal is to have employers hire more people with criminal records, we need to understand why they currently don’t want to hire people with criminal records. And that’s something we don’t fully understand yet. And that is a major challenge to policy. In this in this space is that they’re worried about legal liability. Are they worried about safety? Are they worried about productivity? Reliable like what is it? Because whatever it is, we can then go address that concern. But until we figure that out, we’re kind of

[0:55:08 Speaker 1] stuck. All right, So let’s wrap up with the basically touching on on this notion that you work on an area of study that is traditionally populated by sociologists and criminologists and some folks in public health. Uh, and we touched upon this notion that that economists come at these questions in slightly different ways they have a training to take very hard about incentives. Think very hard about unintended consequences. Basically, the notion of January Taliban took some over here, Move something over there and finally focus on on, like, always very clearly focused on cost benefit analysis. Um, so in your experience, and I can share a little bit about my experience and work with some folks here, but I want to hear from you and your experience in dealing with those folks. Is that Is that a resistance generally, about having that kind of mindset when working with economists is there? I feel that we at least and I totally by looking at how things are described in the media or in in in public discourse in terms of policy making. It’s never sounding like the Economist wants to make it sound. It sounds like sociologist are having more of an influence their, uh, than the public health people have more of an influence there to how an economist would portray those issues. So I guess I don’t know. I don’t have a specific question. More like your your experience on this and how you think the role of economists, how can improve the role of economists actually helping with those issues.

[0:56:28 Speaker 0] Yeah. I mean, obviously I’m biased. And I think the

[0:56:31 Speaker 1] book

[0:56:32 Speaker 0] it economist

[0:56:33 Speaker 1] okay, for

[0:56:35 Speaker 0] important. Yeah. I spend a lot of time talking with practitioners and policymakers and funders about why they should be talking to economists when they’re interested in these topics and even to students who are interested in criminal justice policy and have never thought that they should study economics, right? Like I and I think part of it is this general perception that economics is all about money. And so when people hear that, I’m you know, and I used to travel and occasionally being a taxi or something. And the drivers like So what do you do? And, like, I’m an economist and their own your banking stock markets, like I studied crime and they’re like, Oh, white collar crime. You must study,

[0:57:12 Speaker 1] you know, bank

[0:57:13 Speaker 0] fraud or something. And like, no, no and so, yeah. I mean, I think a big part of this is people don’t know that economics is about you know, how people respond to incentives. Um, and and that’s really I’m thinking through these trade offs which are so fundamental to any policy decision. Um, including in criminal justice policy actually started my own podcast a little over a year ago. Where for exactly this reason? Like, I kind of got tired of having these conversations and just like, trying to, you know, explain to people over and over again why they should be listening to economists if they care about criminal justice policy. And I figured I just start showing them. And so now on this, on my podcast probable causation I interview every every episode and someone who studies who you know who studies crime and criminal justice policy. And we could talk to a specific paper and and basically trying to show people how economists think about these very interesting policy questions. Um, and and the goal is really to hopefully pull some more students into studying economics, but also to showcase the really cool work that economists are doing in the space. Um, so, yeah, I totally hear you. And, uh, you know, I mean, I think this is an area where a variety of disciplinary perspectives is super useful.

[0:58:28 Speaker 1] Um,

[0:58:29 Speaker 0] criminologist tend to have a lot more institutional knowledge than economists do. They spend a lot more time actually in prison sins and jails and talking to people, so that’s super valuable. But I think our empirical toolkit is top notch. And if you want to know what the policy, what the effect of the policy is going to be, you could be talking to economists about that.

[0:58:49 Speaker 1] I mean, I don’t have to. My shirt tells you everything. I think

[0:58:52 Speaker 0] about what

[0:58:54 Speaker 1] we do. We spend time here trying to educate our students, create opportunities like this to talk to people that work hard to understand trade offs of policy making. Uh, I think my efforts have been a lot of the dedicated to teaching classes to folks are not necessarily economics majors and just the power of economics and statistics and giving them tools to think about policy is not about how it sounds, not about how they intended consequences in terms of like, you know, we’re going to make the world a better place, like we all want to do that. We disagree sometimes on how to do that, and a lot of the disagreement comes from the evidence, and that’s what we’re trying to do. So,

[0:59:27 Speaker 0] Jennifer, thank you so much for

[0:59:28 Speaker 1] joining me. And I’m really looking forward to talk this afternoon.

[0:59:32 Speaker 0] Great. Thank you so much. This was great.

[0:59:36 Speaker 1] Thanks for listening to policy on McCombs. Mhm. Mhm. Yeah,

[0:59:47 Speaker 0] yeah