As you may have seen in the news, The New York City Police Department is conducting a comprehensive review of its crime stats. Over the past months, reports have emerged that Precinct Commanders felt pressured to downgrade serious crimes to less serious crimes to both look good at their CompStat sessions and ensure that the overall crime rates did not climb upward.
This case brings to mind an oft forgotten idea in public policy called Campell’s Law. Cambell’s Law is an idea posited by American social scientist Donald Campbell stating that, “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” As you can imagine, Campbell’s Law has been cited by many others in conversations about high stakes test scores, but it is important to remember that we see singular performance indicators driving bad behavior in just about ALL sectors. Think no further than quarterly profit statements for Enron and WorldCom or loan sales at your favorite mortgage brokers (if they are still around).
So did CompStat and the drive to keep crime low in New York City “distort the social processes it was intended to monitor”? I don’t think we’ll know the answer for a while, but as we’ve begun developing a statewide Stat process for the Race to the Top work in Rhode Island, we’ve been reminded of what the Stat process does in a new environment. Whether it be CompStat in New York when it began under Bill Bratton or in any Stat process we develop with a client, the purpose is two fold. First, it is to place the attainment of specific results at the forefront of a managers thinking as they make decisions about tactics, strategies, and resource deployment. Second, it is to use the data itself in many disaggregated forms to inform and enrich the quality of our decisions and to objectively learn from past hypothesis on what works. No one would argue that using this data in this way is bad management and “distorts the process it is intended to monitor”. But at the end of the day, the use of data in management does not cure an organization of unsavory behavior, it simply changes the leverage points of where it can happen.
We’ve also been reminded of the importance of multiple measures. Whether it be value added in teacher evaluation, test scores in AYP decisions for schools, or “crime” in CompStat, one measure never tells the whole story. A good Stat process marries outcome metrics with survey, financial, and observational information to ensure that what gets measures not only gets done, but is what you want