More Guns, Less Crime - Rutgers University



Note: this contains excerpts from an unpublished paper. To view the graphics, make sure that Word is set on "page layout" on the "view" menu.

Selected Graphics from Research on

Murder and Multiple Regression:

By Ted Goertzel

ABSTRACT

Multiple regression has consistently failed to provide definitive answers to policy controversies in criminal justice, yet researchers continue to attempt to use regression techniques for this purpose. This review of multiple regression analyses of trends in homicide rates suggests that the method fails because researchers overfit their models to one data set, then fail to test them with fresh data. The lack of progress in regression modeling of homicide trends over several decades suggests that the trends may be chaotic. Studies that disaggregate trends and combine qualitative with quantitative data have been much more successful.

Note: a summary of this research is available as "Myths of Murder and Multiple Regression," The Skeptical Inquirer, January/February 2002.

Isaac Ehrlich's failed econometric model of the effect of the death penalty on homicide rates contrasts with studies that simply compare trends in states with or without the death penalty, as in the following graph from a recent New York Times story.

Ehrlich's problem was that he did his work too well with one data set, then failed to test it with others. He took a very limited amount of data, of inconsistent quality, and worked very hard at fitting his equations to it. He stopped when he got a model with good statistical properties that made sense to him. The resulting model was overfitted to a specific set of historical data. In effect, Ehrlich took the accidents of history and described them with equations. His mathematical model worked when applied exactly as he did it to the data that were used to construct it. But as soon as

Informed laymen and policy makers are likely to continue to be persuaded by graphs such as one the Times published recently (Bonner and Fendressen, 2000) which showed that homicide trends between 1982 and 1996 were remarkably similar in Texas with 231 executions and in New York and Michigan with no executions and in California with 8 executions. This led the authors to the conclusion that the death penalty had little effect on homicide trends. To debunk such an interpretation, critics would have to show how differences in other factors in these states explained why so many more executions did not seem to make any difference. This might in fact be the case, and readers might be persuaded by a coherent and well-documented argument. Texas, after all, has a different culture and history than New York or Michigan. The lack of differences between very similar neighboring states, such as Ohio and Michigan, would be much more difficult to explain. What seems certain is that a convincing argument would depend on specific arguments and information about the states in question. The track record of multiple regression research in this area has not been good enough to convince any but the most stalwart of true believers.

2. Imprisonment and Homicide. Similar problems account for difficulties Marvell and Moody (1997) had with their study of the impact of prison growth on homicide (as quoted above). Their paper begins with a graph showing trends in prisoners per 100,000 people and homicides per 1,000,000 people in the United States. This very interesting and useful graph is reproduced below from their data. It shows that homicide rates increased sharply from the mid 1960s to the early 1970s, then leveled off. The number of prisoners shot up markedly beginning in the 1970s, as the United States built more prisons in response to the increasing crime rate. The homicide rate leveled off in the 1980s and remained stable thereafter.

These data raise many interesting and important questions. Who was killing and being killed in increasing numbers? Who was being incarcerated in such great numbers, and for what offenses? Getting answers to these questions would certainly help in making useful policy recommendations. But Marvell and Moody, caught up in the futile quest for one big, definitive equation, did not approach the problem in that way. Instead of exploring the complexity of historical change, they used multiple regression techniques to "control" for it. They introduced controls for every measurable variable they could think of, including (Marvell and Moody, 1997: 209) "age structure, economic factors, public relief, race, and variables marking World War II and the crack epidemic." They used four dummy variables to control for historical discontinuities. One of these was scored 0 before 1977 and 1 thereafter to control for the fact that there was a change in the way prison statistics were kept in that year.

The problem with this approach is that all the interesting historical changes are eliminated through statistical controls. This allowed Marvell and Moody to focus one and only one question: the impact of imprisonment on homicide rates, regardless of any other factors. Their analysis led them to the conclusion that a 10% increase in prison populations leads to roughly 13% fewer homicides. If this were true, it would have very important policy implications. Such thinking undoubtedly contributed to the great imprisonment boom since the mid 1970s. But a simple inspection of their own time series graph shows that the promised 13% decline in the homicide rate for each 10% increase in imprisonment since 1975 simply did not occur.

Regression, of course, aspires to go beyond the simple observation of trends. The goal of multiple regression is to take all the other relevant factors into account, telling us what would have happened had the imprisonment boom not taken place. In their paper, Marvell and Moody (1997: 216) estimate that without the prison expansion after 1974, "homicide would have grown to approximately 361 per million population in 1994, up more than 250% from 101 in 1974." They go on, however, to state that "as a practical matter, homicide probably could never reach such levels," because the government would have taken other actions to counteract it.

But what is the value of an analysis that leads to implications that could never actually take place? How valid can the theory underling the multiple regression analysis be if it leaves out key variables, such as political constraints? How dependent are the results on arbitrary methodological decisions? Marvell and Moody (1997: 206) acknowledge that they "are aware of six studies that have estimated the impact of prison population on homicide over time. Three found little or no impact." Indeed, one of these was one of their own previous studies (Marvell and Moody, 1994). Some of these studies used national trend data, others compared states, others examined trends in specific states. Marvell and Moody thought that the inconsistency between their own two studies was due to the fact that an increase in imprisonment in one state may cut the homicide rate in neighboring states. Despite their laudable efforts to document the number of migrating murderers (Marvell and Moody, 1998), it seems implausible that this phenomenon is large enough to account for the strongly inconsistent results between studies, especially with regard to large states such as California (Zimring and Hawkins, 1994).

3. The Right to Carry Concealed Weapons and Homicide Rates. In 1997 John Lott and David Mustard (1997: 1) released a study which purported to show that “allowing citizens to carry concealed weapons deters violent crimes, without increasing accidental deaths.” Tens of thousands of copies of their paper were downloaded on the World Wide Web. It was the subject of policy forums (Cato, 1996 and 2000), newspaper columns (Boldt, 1999), and often quite sophisticated debates on the World Wide Web (, 2000; Friedman, 2000; Gunfree, 2000; , 2000).

In general, the debate followed predictable ideological lines, with one prominent critic denouncing the study as methodologically flawed before she had even received a copy (Lott, 1998: 122-156). Debunking the study was difficult, however, because Lott and Mustard had compiled an enormous data set, with data from every county in the United States, and had done a highly sophisticated multiple regression analysis.

Finding what went wrong with Lott and Mustard's statistical analysis took a good deal of reanalysis of their data. In 1999, Ayres and Donohue (1999) published a paper that disaggregated the data, plotting time series graphs for Florida, Maine, Virginia, West Virginia, Georgia, Pennsylvania, Mississippi, Idaho, Oregon, and Montana. The graphs alone make a fairly persuasive argument that the passing of "shall issue" laws made no appreciable impact on homicide rates in any of these states. But why did the more sophisticated multiple regression analysis say otherwise?

Lott and Mustard argued that previous analyses were defective because the state is not a good unit of analysis. There is more variation between counties within states than between states. It would not be possible to graph the trends in each of America's counties separately, so I decided to begin my own analysis of Lott's data by looking at the counties that contain the largest cities in the country. This seemed justifiable since much of America's homicide problem is found in large cities. In 1992, the last year of John Lott’s data set, eight American cities had over 1,000,000 population: New York, Los Angeles, Chicago, Houston, Philadelphia, San Diego, Dallas, Phoenix, and Detroit. I found that none of these cities was covered by a "shall issue" law during the period covered of Lott’s study. The largest city with a "shall issue" law in effect was Indianapolis, Indiana, with 746,538 people in 1992.

One of the principles of regression analysis is that the results cannot be applied to cases outside of the range of the data. The fact that Lott “controlled” for population size does not compensate for the fact that he simply had no variation in his key variable in America’s largest cities. When I questioned him about this, he did not see it as a problem since one could still compute a coefficient for county size. The fact that one can still estimate a parameter, despite the lack of adequate data, is a problem with regression analysis, not a benefit.

I did find a good opportunity for a natural experiment with Lott's data, however. In the state of Pennsylvania, a "shall issue" law was passed in 1989, but the city of Philadelphia was exempted from it. The graph that follows compares trends in Philadelphia, which is a city and a county, with those in Allegheny County, which includes Pittsburgh. The graph shows that murder rates are generally higher in Philadelphia than in Allegheny County, but the passage of a law giving citizens the right to get permits to carry concealed weapons did not have the positive effect posited by John Lott. In fact, the Allegheny County murder rate was declining prior to the passage of the law, then increased slightly. In Philadelphia, the murder rate had been increasing, then it leveled off despite the fact that the new law did not apply in that city. The violent crime statistics for the same two counties show the same pattern.

Of course, these are only two counties, and there could be other factors that account for these trends. They are, however, substantial population centers with 1,336,832 in Allegheny County and 1,572,096 in Philadelphia in 1992. So these findings are not unimportant. Since I have named the counties, readers can take their knowledge of events and social conditions in them into account in interpreting the trends. Disaggregating the data in this way allows us to draw on our qualitative, historical knowledge in interpreting the statistical trends. To discredit this kind of finding, concealed weapons advocates would have to show how other factors somehow compensated for the failure of the shall issue law to have any apparent effect.

I also plotted trends for a number of other counties where shall issue laws had gone into effect during the period covered by Lott's data set. The next graph shows trends in murder rates for the largest counties in several states that adopted "shall issue" laws between 1977 and 1992. The date at which the laws went into effect varies from state to state. Before reading further, the reader may find it interesting to examine the table and see if he or she can infer when the law took effect.

The pattern in Missoula County, Montana, appears to be quite erratic, with very sharp declines in the murder rate in 1979 and 1991. This, however, is an artifact of the use of natural logarithms instead of the actual rates, not a real phenomenon. The number in John Lott’s data set for Missoula County in those years is -2.30. This seems odd, since a County’s murder rate cannot go below zero. To get the actual murder rate, however, one has to invert the logarithm with the formula true rate = logarithmic rate , where e = 2.71828. This can be done easily with the ex button on a scientific calculator. Entering -2.03 in such a calculator and pushing the ex button yields .100, or one tenth of a murder per 100,000 population. Actually, the true figure for murders in Missoula County in 1979 and 1991 was zero. Lott used .1 instead of zero because the natural logarithm of zero is mathematically undefined, so leaving it at zero would have created missing data. There are a great many -2.3’s in his data files on murder, because many of the counties are quite small, much smaller than Missoula with 81,904 people in 1992.

Natural logarithms are used in multiple regression analyses to reduce the effect of extreme cases. Mathematically, multiple regression analysis assumes a multivariate normal population in which each variable is distributed normally about all the others. This means that most cases should be clustered around the mean, with few at the extremes. No real data fit this assumption perfectly, but the method works reasonably well if the fit is only approximate. Extreme cases that do not fit the normal distribution, however, can cause distorted results.

The distribution of murder rates in American counties is far from "normal" in this statistical sense. There are many small counties with few or no murders, and a few quite large ones with a great many. Converting the data to natural logarithms is one way of minimizing the statistical effects of extreme cases, but it can introduce other distortions as we see in this case. A better solution might be to use methods that do not assume a normal distribution when the data depart significantly from the assumption of normality. However Plassman and Tideman's (2000) reanalysis of Lott's data using a generalized Poisson process did not uncover the fundamental flaws in the data set.

Leaving aside the distortions in Missoula County caused by the conversion of the data to natural logarithms, the trends in these counties are quite smooth. There is no apparent effect from the introduction of "shall issue" laws in Missoula County in 1991, in Fulton County (Atlanta, Georgia) in 1990, in Hinds County (Jackson, Mississippi) in 1990, in Fairfax County (Fairfax, Virginia) in 1988 and Kanawha County (Charleston, West Virginia) in 1989.

One might ask, why are we dealing with these medium sized counties instead of major population centers? Because that's where the "shall issue" laws were put into effect. Although it took two years before Ayres and Donohue (1999) verified the problem in a reanalysis of Lott's data, the problem was anticipated in an essay by Zimring and Hawkins in 1997. They began their discussion with a history of "shall issue" legislation, noting that it was instituted in states where the National Rifle Association was powerful, largely in the South, the West and in rural regions. These were states that already had few restrictions on guns. They noted that this legislative history frustrates (Zimring and Hawkins 1997: 50) "our capacity to compare trends in 'shall issue' states with trends in other states. Because the states that changed legislation are different in location and constitution from the states that did not, comparisons across legislative categories will always risk confusing demographic and regional influences with the behavioral impact of different legal regimes." Zimring and Hawkins (1977: 51) further observe that:

Lott and Mustard are, of course, aware of this problem. Their solution, a standard econometric technique, is to build a statistical model that will control for all the differences between Idaho and New York City that influence homicide and crime rates, other than the "shall issue" laws. If one can "specify" the major influences on homicide, rape, burglary, and auto theft in our model, then we can eliminate the influence of these factors on the different trends. Lott and Mustard build models that estimate the effects of demographic data, economic data, and criminal punishment on various offenses. These models are the ultimate in statistical home cooking in that they are created for this data set by these authors and only tested on the data that will be used in the evaluation of the right-to-carry impacts.

What Lott and Mustard were doing was comparing trends in Idaho and West Virginia and Mississippi with trends in Washington, D.C. and New York City. What actually happened was that there was an explosion of crack-related homicides in major eastern cities in the 1980s and early 1990s, most of them among people who were quite well armed despite the lack of gun permits. In keeping with the usual practice in regression analysis, Lott and Mustard "controlled" for time differences by including a dummy variable for each year. Murders were high in the early years, for reasons having to do with the crack epidemic, and having nothing to do with "shall issue" laws. This was the critical flaw in Lott and Mustard's analysis. In their 1999 paper, Ayres and Donohue (1999: 456) reported that their reanalysis of Lott's data showed that "the whole effect of Lott's regression comes from the fact that the national time dummies are high for the late 1980s and early 1990s and the shall issue dummy reveals that violent crime did not rise as substantially in the shall issue states." Lott's whole argument came down to a claim that the largely rural and western "shall issue" states were spared the crack-related homicide epidemic because of their "shall issue" laws. Just as Ehrlich did, Lott converted the accidents of history into mathematical verities.

In the second edition of his book, Lott (2000: 213) conceded that Ayres and Donohue had raised a legitimate concern, and defended his analysis by noting that he controlled for the price of cocaine in his regressions. He also examined spillover effects and looked at the different dates when shall issue laws were used. None of this, however, compensates for the fact that there simply were no "shall issue" laws in any of America's largest cities where the homicide problem was most acute. As is so often the case, the available data simply do not conform to the requirements of the method.

References

Ayres, I. and Donohue, J. (1999). "Nondiscretionary Concealed Weapons Laws: A Case Study of Statistics, Standards of Proof, and Public Policy." Am. Law and Ec. Revv 1: 436-470.

Black, D. and Nagin, D. (1998). "Do Right-to-Carry Laws Deter Violent Crime?" J. Legal Studies 27: 209-219.

Blumstein, A., Cohen, J. and Nagin, D., eds. ( 1978). Deterrence and Incapacitation: Estimating the Effects of Criminal Sanctions on Crime Rates. Washington: National Academy of Sciences.

Blumstein, A. (2000). "Disaggregating the Violence Trends," In Blumstein, A. and Wallman, J. (eds.), The Crime Drop in America, Cambridge University Press, New York, pp. 13-44.

Boldt, J. (1999). "Study Evidence on Guns," Philadelphia Inquirer, December 14. Downloaded on May17, 2000 from .

Bonner, R. and Fessendren, F. (2000). "States with No Death Penalty Share Lower Homicide Rates," New York Times, September 22. Downloaded from: .

Bork, R.H. (Solicitor General) et al. (1974). "Fowler vs. North Carolina," U.S. Supreme Court case no. 73-7031. Brief for U.S. as amicus curiae: 32-39.

Bowers, W. and Pierce, G. (1975). "The Illusion of Deterrence in Isaac Ehrlich's Research on Capital Punishment." Yale Law Review 85: 187-208.

Brock, W., Hsich, D. and LeBaron, B. (1991). Nonlinear Dynamics, Chaos, and Instability: Statistical Theory and Economic Evidence. Cambridge: MIT Press.

Brock, L. and de Lima, P. (1995). Nonlinear time series, complexity theory, and finance. Santa Fe Institute Working Paper 95-09-077.

Cato Institute. (1996). "Concealed Carry Gun Laws and the Crime Rate." A Policy Forum Featuring John Lott, Jenn Ludwig and Michael Block. Downloaded in streaming audio on from:

Cato Institute. (2000). "Book Forum: More Guns, Less Crime," June 16, 2000. Speakers included John Lott, Carl Moody and William Vogt. Downloaded in streaming video from: .

Donohue, J. and Levitt, S. (1999). "Legalized Abortion and Crime." Stanford University Law School. Downloaded in August, 2000 from: .

Ehrlich, I. (1975). "The Deterrent Effect of Capital Punishment: A Question of Life and Death," Am. Ec. R. 65: 397-417.

Etheridge, H. and Siram, R. (1993). "Chaos Theory and Nonlinear dynamics: An Emerging Theory with Implications for Accounting Research." Journal of Accounting Literature 12: 67-100.

Fox, J. (2000). "Demographics and U.S. Homicide," In Blumstein, A. and Wallman, J. (eds.), The Crime Drop in America, Cambridge University Press, New York, pp. 288-317.

Forst, B. (1983). "Capital Punishment and Deterrence: Conflicting Evidence," J. Cr. Law and Crim. 74: 927-42.

Friedman, D. (2000). "Summary and Critique of "Do Right to Carry Laws Deter Violent Crime," Downloaded on August 9, 2000 from: .

Gainsborough, J. and Mauer, M. (2000). "Diminishing Returns: Crime and Incarceration in the 1990s. Washington, D.C.: The Sentencing Project. Downloaded in August, 2000 from: .

Goode, J. (1999). "Roe v. Wade Resulted in Unborn Criminals, Economists Theorize," New York Times, August 20. Downloaded from: .

. (2000). "John Lott Uncovered." Downloaded on August 9, 2000 from: .

. (2000). "John Lott's More Guns, Less Crime: An Alternate Q&A." Downloaded on August 9, 2000 from: .

Klein, L., Forst, B., and Filatov, V. "The Deterrent Effect of Capital Punishment: An Assessment of the Estimates," In Blumstein, C, et. al, eds, Deterrence and Incapacitation, Washngton: National Academy of Sciences, pp. 336-360.

Lott, J and Mustard, D. (1997). "Crime, Deterrence and the Right to Carry Concealed Handguns," J. Legal St. 26: 1-68. Downloaded on August 10, 2000 from:

Lott, J. (1998). More Guns, Less Crime: Understanding Crime and Gun Control Laws. University of Chicago Press.

Lott, J. (2000). More Guns, Less Crime: Understanding Crime and Gun Control Laws. University of Chicago Press, second edition with additional analyses.

Manski, C. (1978). "Prospects for Inference on Deterrence through Empirical Analysis of Individual Criminal Behavior." In Blumstein, C, et. al, eds, Deterrence and Incapacitation, Washngton: National Academy of Sciences, pp. 400-424.

Marvell, T. and Moody, C. (1997). "The Impact of Prison Growth on Homicide." Homicide St. 1: 205-233.

Mouck, T. (1998). Capital Markets Research and Real World Complexity: the Emerging Challenge

of Chaos Theory. Accounting, Organizations and Society 23(2): 189-215.

Passell, P. (1975). "The Deterrent Effect of the Death Penalty: A Statistical Test," Stanford Law Review 28: 28: 61-80.

Plassman, F. and Tideman, T. N.. (2000). "Does the Right to Carry Concealed Firearms Deter Countable Crimes? Only a Count Analysis Can Say." Downloaded from on August 9, 2000 from: .

(2000). "More Guns Less Crime," Downloaded from the World Wide Web on August 9, 2000 at: .

Rosenfeld, R. (2000). "Patterns in Adult Homicide: 1980-1995." In Blumstein, A. and Wallman, J. (eds.), The Crime Drop in America, Cambridge University Press, New York, pp 130-163.

Schlesinger, A. M., Jr. (1986). The Cycles of American History. Houghton Mifflin, Boston.

Sellin, T. (1959). The Death Penalty. American Law Institute, Philadelphia.

Spelman, W. (2000). "The Limited Importance of Prison Expansion," In Blumstein, A. and Wallman, J. (eds.), The Crime Drop in America, Cambridge University Press, New York, pp. 97-129

Strauss, W. and Howe, N. (1991). Generations: The History of America's Future, 1584 to 2069. William Morrow, New York.

U.S. Supreme Court (1975). "Opinion on Fowler v. North Carolina. Case no. 73-7031.

Walker, S. (2001). Sense and Nonsense about Crime and Drugs. Belmont, CA: Wadsworth.

Zimring, F. and Hawkins, G. (1994). "The Growth of Imprisonment in California." British Journal of Criminology 34: 83-96.

Zimring, F. and Hawkins, G. (1997). "Concealed Handguns: The Counterfeit Deterrent," The Responsive Community 7: 46-60.

-----------------------

[pic]

[pic]

Graphic clipped from The New York Times, September 22, 2000.

[pic]

[pic]

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download