Episode 121 | May 19, 2021
In the world of economics, researchers at Microsoft are examining a range of complex systems—from those that impact the technologies we use to those that inform the laws and policies we create—through the lens of a social science that goes beyond the numbers to better understand people and society.
In this episode, Dr. Hunt Allcott, Senior Principal Researcher at Microsoft Research New England, talks with Dr. Evan Rose, Postdoctoral Researcher, whom Allcott describes as “one of the most engaging and talented researchers in applied microeconomics today.” They’ll discuss how Rose’s experience teaching adult learners at San Quentin State Prison has resonated throughout his research, and they’ll delve into what his and others’ work is uncovering about the criminal justice system today, including the effects of incarceration and parole, impacts of ban-the-box hiring practices, and racial disparities and discrimination.
Related:
- Research Group Microeconomics
- Research Group Economics and Computation
Subscribe to the Microsoft Research Podcast (opens in new tab):
iTunes (opens in new tab) | Email (opens in new tab) | Android (opens in new tab) | Spotify (opens in new tab) | RSS feed (opens in new tab) | Microsoft Research Newsletter (opens in new tab)
Transcript
TEASER (EVAN ROSE): To me, what it means when you have an idea like systematic discrimination or institutional discrimination is not that somebody’s behavior would change if you counterfactually changed somebody’s race or all the affects that are typically associated with race, but simply that the tools and policies are not well-calibrated for one group relative to another. And I think economics is well positioned to answer these kind of questions because we’re good at asking about targeting. This is a question about whether or not the tools we use to target policies, admissions decisions, welfare programs, et cetera, are targeted in a way that makes sense across groups.
[MUSIC BREAK]
HUNT ALLCOTT: Welcome to the Microsoft Research Podcast, where you get a front-row seat to cutting-edge conversations. I’m Hunt Allcott, and I’ll be your host today as we explore the relationship between economics and criminal justice reform. Our guest today is Evan Rose. He’s a postdoc here at Microsoft Research in Cambridge, and I think he’s one of the most engaging and talented researchers in applied microeconomics today. His work on racial disparities in the criminal justice system—which has always been important—has received additional attention with the events of this year, and that’s why I’m so excited to have him on and be able to have this conversation with him. So Evan, thank you, and welcome to the podcast.
EVAN ROSE: Thanks. I’m very excited to be here.
ALLCOTT: I want to start with your background. What made you want to become an economist?
ROSE: I took a long road to economics. I didn’t start college intending to do this. In fact, I was studying Classics at the time—focusing on Greek and Latin—which was very nerdy and very interesting, but ultimately, I sort of ran out of stuff to read. There’s not all that much extent Classical text out there, so I had to pick up some more coursework in order to graduate. I cast it around and started taking some more economics classes, and I discovered this interest that I had in sort of quantitative social sciences, and statistics, and math, and rigorous quantitative methods; I really started to explore that in college. And when I started looking at what economists were doing in their research, my perspective on what economics was totally changed. And I traditionally thought about more, classic macro-style monetary theory questions in economics; but when you look at the research that’s being published and the research that was being published when I was in college, they’re tackling all sorts of questions across a huge set of really important problems, and I wanted to do that, too. I wanted to be able to apply rigorous methods to answer questions that were going to make a meaningful difference in people’s lives across a whole bunch of different domains, and economics seemed like an awesome way to have the opportunity to do that.
ALLCOTT: Do you remember when you ran your first regression?
ROSE: I do. It was in Boon Turchi’s class at the University of North Carolina, and I struggled through Stata for hours and hours and hours until I figured out how to do reg y x, which is so funny to me now because it’s like I could do that in my sleep at this point. But yeah, I mean, that was a pretty crazy moment when I first started taking metrics classes, because that was when I think I really started to learn the power of the methods that economists use to try to tackle these really big, meaty questions that seem intractable, and yet we are able to apply these tools of economic modeling and econometric techniques to be able to distill an answer that’s actually useful and teaches us something about how the world works. And definitely, the first Stata regression I ran was the beginning of that journey.
ALLCOTT: That is great. A lot of your work has touched on issues in the criminal justice system. You’ve looked at long-run impacts of incarceration, the effects of fees and fines, barriers to employment for ex-offenders, racial disparities, and incarceration. How did you first get interested in these topics?
ROSE: When I got to Berkeley, where I did my PhD, I was really interested in inequality. In fact, I really wanted to go to Berkeley because I was really interested in the work that Emanual Saez and others were doing—studying trends in inequality. And in my first year, I actually started to volunteer at a prison in the Bay Area: San Quentin Prison—it’s one of California’s oldest prisons—which, because it’s so close to the Bay Area, it’s only about a 20-minute drive from Berkeley, gets a lot of volunteers from the Bay Area. And there’s a program at San Quentin where the inmates can actually pick up an associate’s degree. And they use a lot of instructors from professionals and graduate students and teachers at Berkeley and other Bay Area colleges to run this university and teach these classes, and that’s something I started doing in my first year of grad school. And that kind of opened my eyes to the importance of the justice system for the fundamental welfare and wellbeing of just a huge chunk of Americans. If you’re interested in inequality, then surely understanding what’s going on in labor markets, thinking about minimum wage, tax policy—all that stuff is very important. But for a lot of Americans, especially people, you know, without a lot of formal education, what’s going on in the justice system is as important, or if not more important. And the more time I spent teaching at San Quentin, the more I learned about how the system works; the more I talked to people who’d spent time in the system; and the more interested I became in trying to understand, well, how does this actually contribute to some of these important questions that economists are trying to tackle elsewhere. At the same time, the other cool thing about studying criminal justice is that over the past 15 years or 20 years, everybody’s modernized and digitized. So police departments, court systems, correction systems—they’ve digitized all of their data in their systems. And that means you can get access to incredible administrative data about what’s happening in the criminal justice system. And this makes it a uniquely rich area to be researching these questions because the data’s there to actually answer a lot of the questions that we want. And, in many cases, some of this data is actually a matter of public record.
ALLCOTT: So, you said that working at San Quentin was eye-opening in various senses. I’d just love to hear a little bit more about what you learned working there.
ROSE: I think what I learned is that people are complicated and that the lives people lead are very nuanced. And most of my students at San Quentin were actually adult learners, so these are guys who are 40 or 50 years old. Many of them have been incarcerated for 25 years for mistakes they made as youths, essentially. And they were incredibly determined to be able to turn things around; to be able to make a meaningful improvement in their lives; to acquire new skills; and to be able to contribute when and if they were ever released. A lot of my students were actually lifers, so were not getting out any anytime in the near future. And that is important because it made me think harder about what our goals should be in the justice system, and what is the purpose of a system that is going to incarcerate somebody for 20 years for a mistake they made when they were 18, and is that actually accomplishing the things we want the justice system to be doing in terms of motivating public safety and achieving other goals that we might have?
ALLCOTT: Is there anybody in particular at San Quentin that comes to mind that has a story that you think particularly affected you?
ROSE: Well, I have a lot of favorite students from my time there. One of my favorite students was a guy named Roche who had an incredible interest in imaginary numbers, and when I was trying to teach him basic algebra, all he wanted to talk about was the square root of -1, which was not something that the rest of the class was prepared to dive into. It’s important to know that I wasn’t there to learn my students’ life stories; I was there to teach them math, and so we did that primarily, but of course, I obviously got to know people along the way. And what struck me is how many people had been experiencing interaction with the state—Roche called himself a ward of the state since he was a child from a very, very early age—and in particular, when it came to education, how many of my students had stories about, basically, they felt being abandoned by the educational system at a pretty young age and being told that they were inadequate and that they couldn’t do it, and so there was no point in continuing to try. And then they were having these moments as 50-year-old men coming back; “Wait, actually I can do this. I can learn math. I can actually get my GED, and eventually I can get an associate’s degree.” But in between that, there was many, many years of scarring about their ability to do these things and a lot of doubt about their ability to actually be able to do the things that they only discovered they could do many years later in prison.
ALLCOTT: I can only imagine what it would be like facing a life sentence in prison and what that would do to my motivation to improve myself or learn new things. Where do people draw strength and get motivation for studying math or the imaginary numbers when they have a life sentence at San Quentin?
ROSE: There’s a lot of different motivations. Some of my students who were approaching a potential parole date had goals: They wanted to start a business; they wanted to get into stocks. There was a notorious student at San Quentin who called himself Wall Street and was always giving everybody trading tips. And he was determined, when he got out, to become a day trader. And obviously to do that, you’ve got to be pretty good at math and statistics—so, he was determined to learn that stuff. Other students were there to just expand their mind. People in prison are curious, intellectually motivated people, just like people everywhere else. And they were reading Locke and Shakespeare because it was engaging to them and interesting—and they wanted to learn math for the same reason. And I think for students who are facing a very long sentence, the intellectual growth was the reward. I personally feel like just because somebody made a mistake a long time ago, that doesn’t mean they should be denied those same opportunities. And I’m excited, actually, that getting back to prison education is something that’s happening more and more, and you know, pilot programs started in the Obama administration are now actually being expanded to restore Pell Grants for incarcerated people. And that’s, I think, going to open up even more educational opportunities for people who are currently incarcerated.
ALLCOTT: All these things that we’re talking about are super important from a social perspective. But as you said earlier, they’re not exactly classic supply and demand economics. Why is it that economists are studying these topics? Why isn’t it just sociologists and criminologists, for example?
ROSE: First of all, I think everybody should be studying these topics. Psychologists, sociologists, criminologists—there’s no rivalry here. But I do think that there’s a couple sets of questions within the space that economists are perhaps uniquely positioned to try to take on. Most obviously, a system of punishments like the justice system sets up is about trying to create the incentives for people to be productive members of society. And there’s a rich tradition in economics about modeling the incentives of punishment and deterrent effects; trying to understand whether or not those are at the optimal level and how severe we should be, going back to Becker in the ’60s. That is something that I think economics is uniquely good at doing. And it’s not just about deterrent effects; the justice system is not some sort of monolithic robot out there enforcing rules. It’s comprised of people—with prosecutors, public defenders, judges, police officers—and everybody in the system is responding to different incentives in different ways. And there’s lots of really rich agency issues, for example, across this space that economics, I think, has a lot to contribute on. The other things I would say is that it’s becoming increasingly clear that there’s a really strong interaction between classic economic factors and what’s going on in the justice system. So, things like poverty, income, and ability to pay, and liquidity are critical determinants of people’s success, particularly after an initial conviction when they’re trying to pay back fees and fines owed to the justice system. You can’t necessarily look at the justice system in isolation. You have to be thinking about what’s going on in labor markets, in people’s access to credit, et cetera, and obviously those are factors that economists have been studying for a long time and think we know a lot about. The other two things I would say is that there’s many ways to evaluate a justice system. We should be thinking about fairness; we should be doing moral philosophy; we should be doing all sorts of stuff; but I would argue we should also just be asking ourselves, “Does it pay for itself?” From a dollars and cents perspective, is what we’re doing effective, especially when we’re thinking about really costly things fiscally—like incarceration—which cost a lot of money to support but also are very costly for the offenders themselves. And economists—as you know in your work, Hunt—I think are probably some of the best out there at doing welfare analysis. That’s not the only way to look at the optimality of what we’re doing in justice system, but certainly something we should be paying attention to.
[[MUSIC BREAK]]
ALLCOTT: So, I want to dive into some of your work now, and let’s start talking about your work on discrimination. How do economists typically think about discrimination, and how do you think about discrimination in your work?
ROSE: I think the way economists think about discrimination is really closely tied to the way economists typically think about causal reasoning. In particular, what we’re trying to do is isolate the treatment effect of race. So we’re trying to ask for a decision-maker, like a judge or a police officer or somebody who’s trying to decide who to hire: If you have two applicants, all else equal, but one happens to be Black and one happens to be White, or one happens to be Asian, one happens to be Hispanic, what have you, what is the effect on that decision-maker’s behavior of observing that racial category? So, you can really think about a causal manipulation and a treatment effect of race. And I think most economists would agree that’s a reasonable definition for what discrimination is; namely, if there is a treatment effect of somebody’s race.
ALLCOTT: So, when you talk about manipulating race, are you talking about manipulating phenotype? Are you talking about manipulating somebody’s name? Are you talking about manipulating somebody’s lived experience for their whole life? What are you talking about?
ROSE: This is exactly one of the objections that I would have to this kind of reductivist way of thinking about what race is and how we could think about discrimination. And I think this is one of the most common objections to something like a correspondence study, which economists do all the time, where you try to randomly manipulate race by changing the name on a resume. So, if you subscribe to more contemporary visions or perspectives on what race is, it’s not necessarily a biologically determined fact—it’s something that’s culturally determined. And in most economics literature, I don’t think people are particularly clear about what race is; is it skin color or is it something about your name, something about your affects, or your upbringing, et cetera? And there are hardcore causal-inference people who would argue that that whole definition I made doesn’t make any sense explicitly because of this idea that there’s no causation without manipulation. So that’s a problem that economics, I think, has to confront. I think there’s ways around it. Instead of the treatment effect of race, we can be thinking about the treatment effect of how the decision-maker would categorize you if forced to put you in a certain bucket. Part of this is driven by the fact that when you do quantitative work, you need to be able to code race in some way. And people do that all the time, the justice system does it all the time, and we tend to just use that blindly in our regressions. I think there’s more work to be done in economics about unpacking that and asking, “Okay, well, if race depends on context and is socially constructed, what are the incentives that affect how race is deployed, and defined, and used in all of these different settings?” But I don’t think it’s something that economics has done a great job of tackling so far, and I would love to try to think more about it in future work.
ALLCOTT: Another aspect of discrimination that economists have thought less about is systemic discrimination. What does systemic discrimination mean to you as an economist?
ROSE: This is another important area where I think economics can do more on discrimination. So, typically, the type of discrimination we were just talking about is very individual-based. It’s about how a decision-maker is going to treat a particular person. And in other disciplines, people have emphasized for a long time that people can be biased, and people can discriminate—but institutions can also discriminate, and rules and policies can discriminate. And that’s not something that people have studied very much in economics, and I think there’s a lot of opportunity to do that, and I’ve tried to do that in some of my work. And to me, what it means when you have an idea like systematic discrimination or institutional discrimination is not that somebody’s behavior would change if you counterfactually changed somebody’s race or all the affects that are typically associated with race or characteristics associated with race, but simply that the tools and policies that institutions are using are not well-calibrated for one group relative to another, either intentionally or unintentionally. And I think we have this debate all the time. Like, if you think about the conversations we’ve had recently about the value of the SAT for college admissions or even the GRE for admissions into graduate programs in economics and other fields, these are all conversations about whether or not that tool—using a test score—is well-calibrated across groups—or not—in such a way that it ends up systematically hurting one group relative to another. And I think economics is well positioned to answer these kind of questions because we’re good at asking about targeting. This is a question about whether or not the tools we use to target policies, admissions decisions, welfare programs, et cetera, are targeted in a way that makes sense across groups.
ALLCOTT: What do you think we know about discrimination and the justice system today, broadly? Where does the literature stand on how much discrimination there is?
ROSE: That’s a great question. There is a huge body of work in economics and elsewhere documenting that discrimination exists and is pervasive in a lot of different aspects of the justice system. And if you think about the value chain, so to speak, of somebody’s first interaction with police to whether or not they’re actually arrested, to whether or not a prosecutor brings charges, to whether or not they’re convicted, to how they’re sentenced, to how they’re paroled later—there’s evidence that even across that entire margin, potentially, decision-makers are using their discretion to harm particularly historically disadvantaged groups and particularly Black men. In economics in particular, there’s been a lot of debate trying to understand where that discrimination comes from: Is it about taste-based motivations? Is it about fiscal discriminations? And there is some argument about whether or not we’ve done a really good job of identifying discrimination where it is. But I would say by and large, there’s pretty weighty evidence from a lot of different fields that discrimination is pervasive throughout that entire process. What I can’t tell you, and what I think is an interesting area to research, is trying to quantify discrimination—like how much would aggregate outcomes change if we were able to just eliminate all bias decision-making from the face of the Earth. And that’s something that is a really important question because it will tell you something about how much we should think about trying to debias judges, for example, relative to changing other factors, like how we deploy police resources or how we assign sentences to begin with.
ALLCOTT: There’s a specific aspect of discrimination that has received a lot of attention in the last few months, which is police use of force. What’s your read of the evidence on racial bias and police use of force?
ROSE: Putting aside police shootings, there’s ample evidence that the rest of encounters with police—so lower-level encounters where a police officer is frisking you, for example, or even using some more minor levels of force—there’s huge racial disparities in the rate at which individuals are subject to those types of behaviors. Now, whether or not that’s biased depends on what your definition is of discrimination. I think for some economists, they might say “no” because it’s not necessarily clear that police officers’ individual behavior would change if you manipulated someone’s race. But what’s clear is those disparities exist, and they’re going to be driven at least partly by bias and partly by differences in how we allocate police resources: where police spend their time, the type of crime they’re focusing on, and how they’re prioritizing law enforcement throughout a given city. On the issue of police shootings in particular, I think there was one particularly high-profile paper in economics by Roland Fryer arguing that it didn’t look like there was evidence of racial discrimination in police shootings, specifically. I’m not very convinced by that paper because what the author has shown is that conditional on getting into a situation where a police officer might use force, it looks like they’re equally likely to use force or specifically, a shooting—whether or not the individual is Black or White. But there’s nothing done in that paper to address selection into those encounters. And if we think that police officers are much more likely just to get into an altercation with somebody who’s Black relative to White, then you might think that the pool of people involved in those interactions is much less riskier among Black defendants relative to White defendants. In that case, it should actually be the opposite effect you’re finding: that, if you were able to compare Black and White individuals who posed the exact same risk of violence to a police officer, then you would see significant evidence of racial bias.
ALLCOTT: I think the Fryer paper is doing the best he can with the data he has. The police shootings data is just from one police department, and so even setting aside the methodological issues and the selection issues that you raise, the standard errors are really noisy. After finding significant racial disparities in lower level uses of force—whether you’re handcuffed, pushed to the ground et cetera—Fryer recommends that police departments think about increasing the scrutiny and the costs to police officers of inappropriate use of lower-level force. I’m curious how police departments might implement that and what your reaction was to that idea.
ROSE: Roland’s paper is great and there’s nothing wrong with his approach. The facts stand for themselves as he’s presented them, but those results have been interpreted in a particular way that I think takes what they’re finding too far. This other point he makes that’s sort of lost in that overall conversation about police shootings is that what we should really be focused on is the bulk of police civilian interactions, which are much less violent than an actual shooting. And certainly, trying to increase the costs for police officers who are doing that would probably be effective. I’m skeptical about, in current regimes, our ability to hold police officers directly accountable for more minor-level interactions that aren’t going to be particularly documented very well and where essentially, you’re asking cops to hold other cops accountable. There are other interventions that seem to do similar things, more focused on police training. One very promising thing is these procedural justice style of interventions where, for example, Chicago’s rolled out something that looks like this, where police officers are getting training on more appropriate practices in terms of how they should be interacting with civilians. And these are like very basic trainings. In fact, Chicago’s, I think, only lasted for a day and they emphasize ideas about respect, neutrality and transparency, communication with civilians, and try to emphasize the process part of what they’re doing on the street. And in Chicago, that’s been actually pretty effective at both reducing complaints and use of force, including lower-level uses of force against civilians. So, I would probably be more excited about interventions like that rather than trying to use sharper sticks to try to corral police behavior and try to reduce the incidence of that kind of behavior.
ALLCOTT: So, I want to dive now into some of your work on technical violations in the probation system. Can you tell us a little bit more about that?
ROSE: My paper studies the probation system, which I think flies a little bit under the radar relative to how important it is. There’s a lot of focus on incarceration, which is right and appropriate because incarceration is a massive social problem. But there’s maybe two million people in prison and probably four million–plus people who are on probation and parole at any given time. So, for the modal individual interacting with the justice system, probation is what ends up happening to you. And probation—just to make sure we’re all on the same page—the way it works is that instead of going to prison, you get to go home, which is a blessing of course, but you’re going to be asked to respect all these set of technical rules over the next year or two, or more, and those rules are going to make you have to pay back fees and fines; they’re going to make you abstain from drugs and alcohol; they’re going to require you to stay within the jurisdiction of the court. And if you break those rules, you can end up going to prison, which ironically makes probation actually one of the biggest drivers of incarceration overall. Probationers are as likely to go to prison for breaking one of these technical rules as they are for actually committing a new crime, recidivating. And in my work, what I’ve tried to ask is whether or not that system makes sense: What’s the value of enforcing these technical rules and using these technical rules to send a whole bunch of people to prison, and how does it actually contribute to the big racial disparities we see in the prison system? Because when you look at the data, it turns out that Black probationers are much more likely to be subject to technical violations for failing a drug test, or breaking curfew, or not finding a job, or not paying fees and fines. And what I’m finding in that work is that from a general level, these rules seem, in principle, to be relatively well motivated in the sense that they’re useful for monitoring. So, the people who are failing a drug test are also the same people who are very likely to get arrested for actual criminal finding in the future. So, it makes sense for probation systems to use these technical rules to try to identify people you should be paying closer attention to. Where it’s critical though is the signal value of a rule violation—for somebody’s future risk of committing crime, it’s very different between Black and White offenders. It’s a really strong signal for White offenders, and it’s a really weak signal for Black offenders. So, when a Black offender, for example, gets a technical violation for not paying a fee or fine, that’s really giving you no information about whether or not that person is committed to rehabilitation. So, if you start incarcerating people for not paying fees and fines and failing drug tests, you end up actually exacerbating the same racial disparities in incarceration exposure that we see in aggregate. And there’s actually a ton of scope for reform here, I think, because a lot of jurisdictions are still using these types of technical rules very aggressively all across the US.
ALLCOTT: This to me is one of the most interesting aspects of this paper. Why is it, do you think, that the technical violations are a worse signal for Black people than they are for White people?
ROSE: I think there’s two reasons why you might see this kind of pattern. The first could be that the people who enforce the rules are discriminating. So, the probation officers, the case workers who are charged with doing the drug testing, et cetera, they might treat Black offenders more harshly; they might be more likely to write up that person or drug test them more frequently or enforce curfew more strictly. But the other reason could just be that these rules are enforced in an ostensibly race-neutral way, but the behaviors are simply different across populations, and there’s a lot of good reasons why you might think that’d be the case. For example, poverty is particularly concentrated among Black probationers in North Carolina, and a lot of Black offenders are living in neighborhoods where there’s a lot of poverty all around them. So, you might think that access to informal credit to be able to pay down these actually quite large fees and fines is pretty different across the two populations. So, a Black offender might not be paying those fees—on average, it might be more likely that they simply don’t have the money—whereas when a White offender is not paying those fees, it might be that they’re actually electing not to do that because they’ve decided that it’s not worth it. So, thinking about the correlation between the underlying behavior that’s targeted by these rules and what we actually care about—which is people’s propensity to commit crime or conversely, to get rehabilitated—there’s tons of reasons why you might find that those relationships differ across populations. And it comes back to this question of how we design these rules to begin with. It’s not necessarily that these rules were created with the intention of hurting one group relative to another, but somebody who’s probably thinking about a representative offender maybe looks like them and thinking about, “Well, what would be the usefulness of enforcing a drug test or fees and fines” and not necessarily considering the impacts of these types of rules on the diverse set of offenders who are actually going to come through the probation system in practice.
ALLCOTT: Another topic that you’ve studied that interacts with racial discrimination is the ban-the-box laws. And I’m curious if you could walk us through what ban the box is and what are the potential unintended consequences so far.
ROSE: Ban the box is a really well-motivated idea: It’s the idea that nobody should be denied an opportunity to get a job because of a mistake on their criminal record. Now close to 200 cities and jurisdictions across the US have passed rules that said, “Employers, you’re not allowed to ask somebody on your job application if they have a criminal record; you can only do that later in the job application.” The idea was to let people get a foot in the door and try to actually get into an interview where they can impress somebody and get a job. But almost immediately, a bunch of economists raised their hands and said, “Hey, wait a second, I’m not sure this is actually going to work the way you think it will.” And the big concern that people had was around statistical discrimination. If you take away employers’ ability to observe criminal records, they might start using other information to try to guess who has a criminal record because they think a criminal record is relevant for the decision they’re trying to make, namely whether or not to hire somebody. So, if you come from a group who might have a disproportionate share of criminal records—like if you’re a Black man without a high school degree, for example—then employers might be less willing to call you back because they’re just guessing that you’re more likely to have a record to begin with. And there’s an audit study showing that that seems to be exactly what happened in New York and New Jersey, at least at retail establishments, after these jurisdictions enacted ban-the-box laws. Places that used to ask about criminal records stopped calling back Black applicants on average much more than they reduced callback rates for White applicants. There’s been a whole slew of research showing that unintended consequence might be playing out in actual employment as well as in callbacks. You might be willing to tolerate some spillovers into the general population if the law did what it was supposed to do, which was namely help ex-offenders get a foot in the door. But in my own research, what I’ve found is that when you look at the employment outcomes of people with criminal records after ban the box comes into effect, it doesn’t look like they actually succeed in getting more jobs and earning more money.
ALLCOTT: What do you think are ways in which employers should and should not be allowed to use characteristics to decide who should get a job? For example, in the job that we have, you have to have a PhD, you look at publication records, et cetera. There are a set of characteristics or observable factors about people that we agree are useful for deciding who should be interviewed even in the first place. On the other end of things, there are a set of factors that we agree should not be used, such as race. And so, how do you think about what factors should and should not be allowed as part of initial screening for jobs?
ROSE: That’s a great question. And you know, there’s pretty well-developed legal theory about what you can and can’t do in terms of collecting information about potential job applicants, which is motivated by these ideas of disparate impacts. If you’re using something like collecting criminal records or using some sort of employment test and that tends to disadvantage one group relative to another, the value in doing that is going to depend on how related that tool is to your actual business need, namely correlated with productivity. And when it comes to criminal records, specifically what we’ve learned is that even though they might be correlated with actual productivity on the job, employers are not very good at using that information reliably or responsibly in the sense that they seem to really massively overestimate the share of individuals who are applying to their jobs with criminal records—and that’s one of the reasons why you see such big negative consequences of ban-the-box style policy. And that’s a motivation for trying to restrict the information that an employer could use potentially in that sense, because it’s not clear that the costs of doing that to one group relative to another are going to outweigh the actual private benefits to the employer in terms of making better decisions in who they end up hiring.
ALLCOTT: Now, of course, in the Agan and Starr paper, when you do ban-the-box and prohibit employers from using the box as part of their decision, they make another form of statistical error, which is that they over discriminate against Black people compared to what a Bayesian person would do. And so, it seems like there are these multiple different margins of potential employer mistakes, and I guess that an omniscient policymaker would somehow try to trade these off or maybe provide information to employers to help eliminate these mistakes.
ROSE: Yeah, I think that’s right. Generally, if employers have correct information, if they’re making the correct statistical inferences, then in most models, the decisions they’re making are going to be efficient—or at least in the sense that they’re profit maximizing. But you can think about a model where employers are making profit-maximizing decisions; as a result, you get horizontal inequities between, for example, equally productive Black and White individuals because one is subject to statistical discrimination and one is not, or one is subject to a lesser degree. And that’s some sort of public cost, and you could think about trying to correct or incentivize employers to offset that particular difference. But where we are in terms of the levels of what kind of incentives you’d actually want to put in place to do that is really hard because it seems like employers are not making accurate statistical decisions in any way, shape, or form when they’re deciding who to hire.
[MUSIC BREAK]
ALLCOTT: So Evan, I want to switch gears to another one of your many important papers. You have this work with Yotam Shem-Tov on the causal effects of incarceration, and I wanted to have you talk through that. I mean, of course, this is a big debate between people who say we should have longer sentences versus people who say being in jail is not helping people at all. And so how do you think about balancing those things, and what were the data that you used to help answer that question?
ROSE: The question we’re after in this paper is exactly what you said: What does spending time in prison do to people in terms of their long-run outcomes? And we’re focused in particular on their likelihood of committing more crime. This is a really tough question because incarceration sentences are not meted out randomly, thank God, but you need a research design to look at this. And people have used random assignment to judges in the past. What we use is actually sentencing guidelines in North Carolina, which give recommended sentences as a function of people’s criminal history and the type of crime they’ve committed. And there’s points in those guidelines where when you go from having one extra misdemeanor relative to somebody else in the data—you see big jumps in both whether or not somebody goes to prison and the length of time they go to prison when they do. So this is a pretty good setting for trying to get quasi-experimental variation in exposure to prison and look at the effects. And what’s particularly cool about it is that there’s a lot of those discontinuities—in fact, over 20 in the data. Instead of just looking at a single late, what we can do is actually look at a lot of different margins. Looking at the effects of zero prison versus a couple of months, we can look at the effects of four prison versus five years, and in the paper, we do a lot of work trying to unpack those different margins from what those different effects are. And there’s two broad theories about what you might expect. People have talked about prison as a sort of school for crime: you get disconnected from the labor market; it’s a really depressing place; there’s some substance abuse going on; and you’re hanging out with other people who’ve also committed crimes; so maybe when you get out, you’re actually a much better criminal than you would’ve been beforehand, and you commit more crime. On the other hand, there’s really good reasons to expect that prison is not a pleasant place, and people will do anything to avoid going back. So, there’s an important specific deterrent effect in addition to whatever rehabilitative services that the prison’s actually providing. What we find is that there’s no evidence for this sort of school-for-crime idea. If anything, people commit a lot less crime when they’re in prison relative to what they would have been doing if they’re out of prison. And once they get out, it doesn’t look like they catch up in any meaningful way. If you compare two people, one who went to prison and one of who didn’t, eight years later, the guy who spent time in prison is going to have committed significantly fewer offenses over that pretty long time horizon. What’s interesting though is that most of the gains from doing that in terms of reduced crime come from pretty short sentences. It comes from exposing somebody to a couple of months, six months of prison. Exposing somebody to much longer sentences—so shifting somebody from four to five years—seems to have really limited impact on the quantity of crime they commit in the future. And this is interesting for exactly the reason you mentioned, Hunt, that there’s a lot of debate about how long sentences should be. In particular, there’s a question about whether or not prison sentences are too long in the US relative to what they are in other countries. Our evidence certainly supports that marginal increase in sentences over and above one or two years are at least not effective for encouraging people to desist from crime over the long-run horizon.
ALLCOTT: So why is it that a one-year increase on top of a one-year sentence is more effective at reducing crime than a one-year increase on top of a four-year sentence?
ROSE: Part of it is that, if you think about that specific deterrent model—just understanding that prison is a really unpleasant place to be and that if you commit another crime, it’s very likely you’ll go back there—I would wager that most of that learning happens within a pretty short period of arrival in prison. So, getting an extra year of prison on top of your four-year sentence is not necessarily going to increase your expected disutility further, conditional on reoffending. To learn that lesson, you might not actually need to expose offenders to very much prison. The other thing you’ve got to think about is the counterfactual. So, when you send somebody to prison for a year versus no prison, if you hadn’t sent them to prison, they would’ve been on probation. And it turns out that spending a year in prison and then getting released is actually probably much better for people over the long run than actually having to go through this whole probation regime with all these technical rules that we were talking about earlier. So, another way to interpret our results is it would be better for somebody to go to prison for a short spell rather than having to deal with this whole community supervision regime, and all these technical rules, and the many ways that those end up hampering people’s long-run rehabilitation.
ALLCOTT: When you talk about better for people, you mean specifically as measured by rearrest?
ROSE: Yeah, talking about rearrest exclusively.
ALLCOTT: And just to be clear, when you’re talking about an increase in sentence on top of a short sentence versus a long sentence, are you comparing the same populations or are there different or more-hardened criminals who are having the year added to their longer sentence in your design?
ROSE: In our design, there’s both. There’s variation in the margin people are exposed to, and there’s variation in the people who are exposed to those margins. Because we have so many instruments, we’re able to try and unpack those two things. The paper estimates not only nonlinear effects of exposure to prison, but heterogeneity in those nonlinear effects across different people. And what we find is that, if anything, the treatment effects of prison in terms of reduced crime are largest for the people who are the most hardened in the sense that they’re the people who any judge would be willing to send to prison because it seems clear that they should go there. Those are the people who seem to benefit the most. And that’s good news if you think about how judges are making decisions about who should be going to prison today. And it also makes intuitive sense because these are the people who, in a sense, stand the most to gain from exposure to prison because they’re the people who are most likely to reoffend. So, in quantities, the reduction in reoffending for them is larger; it’s not too surprising because they have such a high base to come down from.
ALLCOTT: I guess I’m still struck by part of the language here, which is around whether it’s good for somebody to be in prison, because there’s a whole series of costs and the toll that this takes on people’s families. Is there any way to do a more holistic analysis of the benefits and costs of sending somebody to prison?
ROSE: Yeah, there is. I was probably being a little too cavalier when I was talking about good or bad. Prison has many effects, not only on offenders themselves—but there’s obviously the victims and the benefits they might get from seeing justice served—but also offenders’ families, earnings, et cetera. And we’re trying to get a better measure of all of those different types of costs in ongoing work. What we do in this paper is just a very simple partial cost-benefit analysis where we ask, ”Okay, we know how much it costs to incarcerate somebody for a year, and we know what the all-in reduction in reoffending, rearrest, gets us is. So, what is the breakeven valuation we’d have to put on reducing rearrests in order to justify how much we’re spending on incarceration to begin with?” And when you do that, your average rearrest should probably cost, socially, between something about 70 or $100,000, which is a pretty big quantity if you think about what the composition of arrests actually looks like. And if you were to fold in other crimes—if it turns out that forgone earnings are a really important channel—that would obviously reduce the net cost of incarceration and make those breakeven valuations go down further. So, there’s many ways to try to incorporate those different potential costs of incarceration, but the first step is definitely just measuring them. And this is something that this literature is still trying to get a grip on doing—not only on direct effects on offenders themselves, but also all those important spillover effects on families, on children, on communities, et cetera. We simply don’t have very reliable point estimates yet for how big those effects might be and even which direction they might go in.
ALLCOTT: This is reminding me of some work that I’ve been doing recently in payday lending with a great group of collaborators where there’s a series of papers that say, ”When you have access to payday loans, how does this impact a series of outcomes: whether or not you go bankrupt, whether or not you access social programs, et cetera.” And broadly, this is trying to inform a debate as to whether allowing payday loans is overall good for societal wellbeing. And I think there’s another way to try to get at that question, which is: Instead of trying to measure the causal impacts on every possible thing—which seems to me to be a very difficult task—instead to take an approach where you assume people are optimizing, and you measure any mistakes that people might be making on any other externalities, and you use individuals’ envelope conditions—the assumption that they’re optimizing—to do welfare analysis from there. And I’m curious if you have a sense of whether there’s any promise for doing that. The problem in thinking about rational crime here or the mistakes made over and above a rational optimizing decision, you’d have to get a good sense of what are the actual mistakes that people are making. Do you think that’s another approach that one could take?
ROSE: Yeah, I mean certainly if you’re trying to think about internalities and how people are approaching this would be useful, especially for thinking about the value of incarceration as a deterrent. It seems pretty clear that most potential offenders are making pretty big mistakes about underestimating the probability of apprehension and then underestimating the quantity of time that they would be incarcerated if they did do that. A lot of myopia in terms of discounting the future. And so, thinking about the impact of those things and how we should be setting incarceration policy is pretty useful. What we were talking about earlier was sort of the externalities of an incarceration sentence on other individuals. It’s hard to think about how you would use that approach to try to get an all-in cost of incarceration if the individuals who are optimizing are not accounting for any of the spillover effects when they make those choices. But if you had good measures or you at least had some way to sort of think about how you want to fold in those kind of costs as well, yeah, I think that would be a pretty interesting approach.
ALLCOTT: Speaking of future research, I want to conclude by asking about what you think are the most socially impactful possible research directions.
ROSE: That’s a big question. So I, to some extent, feel like a lot of the stuff we’ve been talking about today is addressing symptoms of deeper issues. Ban the box in particular is a good example of this. These are really well-motivated laws; but when you look at people’s labor market outcomes who have yet to pick up their first conviction but are headed for one in the next couple of years, they’re already extremely disadvantaged relative to the average American, earning less than $1,000 on average per quarter employed in the formal sector, at maybe 30 or 35% of the time at a quarterly rate. There’s clearly a whole set of factors that are pushing people towards interaction with the justice system before they ever get there. And a lot of these issues we’ve been talking about and been thinking about are, “What’s the best we can do to incentivize the prosocial behavior and then to ultimately promote rehabilitation?” But I’m interested in also thinking about what we’re doing earlier in people’s development and human capital development that would change trajectories before we get there. And I have some ongoing work now looking at the impacts, for example, of teachers and whether or not getting exposed to a really high-quality teacher can dramatically change your likelihood of getting involved in the justice system later in life. There’s other really exciting work: I think about school quality, I think about exposure to pre-K stuff, and even potentially very early childhood interventions and long-run impacts there. And I think knowing more about how we can invest in schools, in resources for parents, in opportunities that people have when they’re young, might be, actually, the highest return place for us to be thinking about how we can make meaningful differences in the justice system. It’s not to say that we should stop thinking about incarceration policy; we should stop thinking about ban the box; we should stop trying to help people who are already there today now—but the point is that the issues start much sooner in people’s lives. And that’s where my opinion is we should be focusing our research on identifying the most effective interventions. In our teachers work, we’re certainly finding that teachers matter, and interestingly, we’re also finding that teachers seem to matter the most early in people’s lives. In some sense, your fourth-grade teacher might matter a lot more than your eighth-grade teacher, which is all consistent with this idea that you accumulate a lot of human capital over the course of your life, social skills, et cetera, and that early interventions can be really, really impactful for some of these long-run outcomes.
ALLCOTT: I was smiling at the end of your answer there because I had an excellent fourth-grade teacher, and I feel like I have her to thank for lots of things. I can only hope that everyone else in our society can have the fourth-grade teacher that I had. Thanks a lot, Evan, for coming on the podcast. One of the things I’ve been thinking a lot about recently is: What is the social impact of different research that we could do? When I look back over at my own papers, I can point to some papers that I thought might have been impactful; I can point to other papers that were vaguely about a policy-relevant topic but maybe didn’t have a direct path or a theory of change for how it would actually impact the world or change people’s lives. And so, one of the things that I’ve been thinking about a lot from my own project selection and when I talk to grad students and other colleagues is: When you think about a paper, what is the theory of change? Who exactly is going to read your paper, use your paper, do something different—and how is that going to impact people’s lives? And in particular, how will that impact disadvantaged people’s lives? And that’s a tall order for any paper. I don’t mean to apply that criterion in a way that sounds scary, but it’s something that I’ve been thinking about a lot. And in that context, the reason I was so excited to talk with you is that I think that your work has a clear path towards social impact. It is carefully done work where I’m confident in the results, and I’m confident that people will read these results and change policies in a way that will make people better off. I really appreciate you for doing that, and I think for other folks in this area there’s a lot more work to do, and so I’d encourage you to keep working in this space.
ROSE: Thanks, Hunt. This was a really fascinating discussion, and I’m looking forward to continuing the conversation in the lab.
ALLCOTT: To learn more about Evan Rose and Microsoft Research, check out Microsoft.com/research. Thanks for joining us today on the Microsoft Research podcast. We’ll see you next time.
Economics 
Social sciences