Operational Evaluations in Policing: Units

Operational Evaluations in Policing: Units of Analysis

Units of Analysis Used by the Rutgers Study:

There were three specific units of analysis used by the Rutgers "Do Police Matter? An Analysis of the Impact of New York City Police Reforms" study (Kelling & Sousa, 2001). They were: (1) changes in crime rates in the entire city of New York, (2) detailed analysis of six of the seventy-six New York City Police Department (NYPD) precincts, and (3) individual precincts by virtue of aggregated crime report analyzed at the precinct level.

Conceptual Criticisms of the Units of Analysis Used by the Rutgers Study:

In principle, the units of analysis used in the Rutgers study are logically relevant and appropriate to the analysis of police operations. However, several unique aspects of the New York City community in particular may undermine the analytical value of those units of measure. Specifically, the five boroughs of New York City (Bronx, Brooklyn, Manhattan, Queens, and Staten Island) comprise a tremendously varied population with large sections of each county reflecting racial, ethnic, age, income, and other socioeconomic variables. Likewise crime rate and the types of crimes in each borough and in different neighborhoods in all five NYC boroughs are substantially different.

As a result, changes in crime rates and patterns of specific crimes in the entire New York City area including all five city boroughs is somewhat unlikely to yield meaningful analyses. At best, the first unit of analysis used by the Rutgers study provides little more than a reliable average of changes over much too great a geographic areas and demographic range to be especially helpful in understanding anything much more specific or meaningful about the contribution of policing operational effectiveness.

The second unit of analysis, the detailed review of crime statistics in six specific precincts likely provides a more accurate view of relevant patterns in the immediate geographic areas covered by those precincts but is less likely to provide accurate information of the New York City neighborhoods not immediately adjacent to the service areas served by those precincts. Since this unit of analysis provides little (if any) corresponding information of the city's remaining seventy precincts, representing nearly ninety-two percent of the city's population by geographic breakdown.

The third unit of analysis, the aggregate crime reports representing cumulative individual precinct-level analyses is likely the most meaningful of the three units of analysis used by the Rutgers study. It is the only unit of analysis used that provides information about the entire metropolitan community served by the NYPD. The weakness of this unit of analysis is mainly attributable to the isolated source of analysis in each of seventy-six precincts.

That is because in many respects, today's precincts still correspond to lines drawn arbitrarily by city administrators before the turn of the 20th century (Conlon, 2004). Many of those lines have changed in the last century and numerous new precinct headquarters and station houses have been added while others have been eliminated or absorbed or consolidated by others. Still, individual analyses of random or geographically-distant precinct-specific crime statistics are substantially less meaningful without some component of such analysis that also considers larger areas of overlapping precincts. Given the scale of potential changes in as large and varied community as New York City, the inability of the third unit of analysis to account for larger patterns is a considerable weakness.

Alternative Units of Analysis for Future Study:

Possible alternative units of analysis for future study would include those capable of providing information of larger geographic and demographic areas instead of information about the entire New York City area; a more targeted review (such as by specific types of crime) of more precincts instead of a more detailed review of a much smaller number of precincts; and a centralized analysis of information from each individual precinct instead of reliance on independent self-analysis provided by each precinct.

More specifically, one alternative unit of analysis could be the selection of certain particular types of crime based on apparent city-wide need and relative seriousness, in conjunction with a much narrower analysis of that type of information from all seventy-six precincts instead of a wider or comprehensive review of only a small fraction of all of the city's precincts. Unlike the units of analysis used by the Rutgers study, the suggested alternative unit of analysis would not omit more than ninety percent of the corresponding information from the city's other precincts.