This post was adapted from a research paper I wrote for UCL's School of Public Policy.

There is a statistic going around that more white people are killed by police than blacks along with some associated analysis. While this may seem insightful considering the current climate around police homicides, there are a number of issues with the data used that make them basically useless for any rigorous analysis or country wide arguments or representations. Klinger (2012) echoed this in his scathing analysis of these databases. He argued there is “no sound, empirically grounded, idea of how many people across the nation are shot at by police officers, how many are struck by police bullets, or how many of the individuals who are shot die from the wounds they suffer” (p. 79).  It is interesting to look at the actual data and see the massive holes, so don’t just take my word for it if you are skeptical. I have given an overview of what is going on below.

Where Does the Data Come from?

The Uniform Crime Reporting (UCR) program began in 1930 and is the means by which the FBI receives voluntary information on crime from police agencies, but also information on homicides by police. The supplementary homicide report (SHR) is the section of the UCR that covers all forms of homicide, both justified and unjustified. This form allows police departments to fill out a short description of the circumstances leading to the killing. The UCR handbook (2004, 17) defines justifiable homicides, as they relate to police, as “the killing of a felon by a peace officer in the line of duty.” Homicides by police deemed as unjustified are aggregated with other homicides.

In addition to SHR, there are two other major databases tracking killings by police. They are the National Center for Health Statistics’ National Vital Statistics System (NVSS) and the U.S. Bureau of Justice Statistics’(BJS) Death in Custody Reporting Program (DCRP) started in 2003. The primary sources for their data are death certificates, police department records, and the media. The NVSS specifically relies on death certificates for their data. This database is widely considered to be unreliable (Sherman and Langworthy, 1979). Since 2000, it has been required by the federal government that deaths during arrests must be reported to the attorney general as dictated in Public Law 106-297. The BJS was tasked with rolling out this new database (DCRP). Non-compliance by the states would result in the loss of corrections related grants. The DCRP has the most comprehensive measurements (but not necessarily data) of types of deaths caused by police of any national database (Mumola, 2007).

What is the State of the Data?

It is important to keep in mind that the SHR, DCRP, and NVSS databases have been the general foundations for the analysis of police violence since their respective inceptions. It has been known, at least for the past 35 years, there were serious problems with these databases. Sherman and Langworthy (1979) found that the NVSS data underreported national data by as much as 51%. Much of this is due to the inconsistency of reporting by coroners and medical examiners who seemingly disregard the need for accurate coding on the reports or may be influenced by local politics in determining the cause of death. They also explained that the FBI did not think their SHR database was reliable, which has made it difficult for researchers to access it. The data from each database are inconsistent with each other.

Dobrin et al (2003) discovered a discrepancy between the SHR and NVSS, with the SHR having 29% more recorded homicides by police than the NVSS. It is likely that the FBI statistics are the most complete, but as explained earlier, the FBI does not consider its data reliable. For both databases, failure to file the necessary forms and fill them out properly are key problems with their reliability. In the case of the SHR, some police departments choose not to participate. Maxfield (1989) explained that many of the SHR forms submitted by police departments are incomplete. This results in much of the coding being listed as unknown. These reports provide little information about the nature of the murders or any other relevant information pertaining to the backgrounds of the involved parties. For the NVSS, the coroners often do not indicate on their submitted forms that the killing was by a police officer.

In a Bureau of Justice Statistics report on DCRP, Mumola (2007) explained that whole states did not report any data over multiple years. For the states that did supply data, thirty relied on media searches to collect their data. Only California and Texas required law enforcement offices to report homicides by police to a state agency. Nine states actually used the SHR data to report to the BJS. Needless to say, there was no consistency in how states collected and reported their data. 

To address these shortcomings, some researchers have worked with both city level surveys of police departments and the NVSS data (see Liska & Yu, 1992), but these are few and far between. Jacobs (1998) called into question the validity of any analysis that seeks to generalize from such data, arguing that the results are too inconsistent to be of any value. Ironically, Jacobs used the SHR data in this same article.

Other Issues with Official Databases

Besides the poor quality of the data, the data is aggregated in such a way that analyses tend to lack nuance. For example, the SHR database explicitly lists only justified homicides by police. Homicides by police that would be considered unjustified are lumped in with all other homicides. This actually is not a huge issue considering most homicides by police are judged as justified, whether or not that is actually the case, but this does get at a fundamental measurement issue for the SHR database.

Fyfe (1979) explained that even if we have good data on the number of homicides by police, it would still not paint the full picture. The vast majority of people shot by the police survive. The current metrics would not accurately reflect police violence regardless of any successful efforts to find data for the current measurements. 

Is There Any Hope?

Over the past few years, unofficial databases have been multiplying in response to the poor state of official data. Since the unofficial databases base much of their data on news reports, the information they are able to retrieve is limited by what information is provided in the news article and whether the police departments are willing to share information with the media. The following graph highlights this problem. The data comes from the Fatal Encounters database which is considered the most reliable non-official database and arguably better than official databases such as the SHR. Fatal Encounters is used by leading news agencies and researchers for their work (see The Guardian, 2015).  Consider, if you want to analyze the issue of race as many researchers have, that 34% of the victim’s races listed in the database are coded as unknown. What those numbers actually represent can have major implications on any research done on the subject, not to mention all the data that is likely missing. Other variables, such as location of the homicide, are much more complete than race and are likely to result in more reliable outputs.

The long and short of it: we need a better way of measuring homicides by police.

Sources

The Counted. The Guardian. Available at:  http://www.theguardian.com/us-news/series/counted-us-police-killings, (Accessed: 03 January 2016)

Fyfe, J.  (1979). Shots Fired: An Examination of New York City Police Firearms Discharges. Dissertation.

Jacobs, D. (1998). The Determinants of Deadly Force: A Structural Analysis of Police Violence. The American Journal of Sociology. 103(4), pp. 837-862.

Klinger, D. (2012). On the Problems and Promise of Research on Lethal Police Violence: A Research Note. Homicide Studies. 16(1), pp. 78 - 96.

Maxfield, M. (1989). Circumstances in Supplementary Homicide Reports: Variety and Validity*. Criminology. 27(4), pp. 671-696.

Mumola, C. (2007). Arrest-related Deaths in the United States, 2003-2005. [Washington, D.C.], U.S. Dept. of Justice, Office of Justice Programs. http://purl.fdlp.gov/GPO/gpo2476.

Sherman, L. & Langworthy, R. (1980). Measuring Homicide by Police Officers. Journal of Criminal Law and Criminology. 70(4), pp. 546-560.

 (1984). Uniform Crime Reporting Handbook. United States Federal Bureau of Investigation.

(2015). Investigation of the Ferguson Police Department. United States Department of Justice Civil Rights Division.

Comment