In the mid 90s, I got a chance to work on NYPD‘s Narcotic Investigation and Tracking of Recidivist Offenders (NITRO). This was during the early days of CompStat, when NYPD started using data-based policing strategies. Back then, it was still mainly mainframes and pin maps as I did my regular visits to Police Plaza and the Crime Lab when it was still housed in just 2 floors of the Police Academy on 21st street in Downtown Manhattan.
NITRO, as the name implies, was a database that tracked career drug offenders.
I worked for the LIMS (Laboratory Information Management System) company that won the bid for NITRO, and was tasked as the “PC/onsite guy” – I built the PC GUI that did HLLAPI calls to the green screen; paid regular site visits to install patches; helped with onsite training; and compiled bug reports and feature requests.
I was in my early 20s then, and this was my first major onsite assignment. I looked forward to my visits as I got a behind-the-scenes look of how things worked in NYPD – the largest police force in the US, if not the world.
This was before the age of CSI, DNA matching, and the Internet. When people thought of cops – the popular media portrayal back then in NYPD Blue and Law & Order (both set in NYC), was that of detectives going about their work using “traditional” sleuthing and reasoning skills. “Gumshoes ” in the true sense of the word, who pounded the pavement chasing down leads.
Most episodes in the early days of those shows would often highlight old-school detectives Andy Sipowicz or Lennie Briscoe making the intuitive leap, connecting the dots, primarily using raw brainpower – “wetware“, to crack the latest case. And this was a fair, though dramatized reflection of what actually happened in real-life.
It wasn’t as if NYPD didn’t have computers back then. I can still remember the huge mainframes in Police Plaza on the 8th floor that hosted all kinds of crime databases from which we imported data. Its just that the arrest data was computerized in the digital equivalent of musty filing cabinets.
But by linking the data as was done in NITRO, it was amazing how a relatively simple cross-referencing algorithm allowed NYPD to do a more effective job of gleaning hidden drug distribution networks and allow drug units to prioritize their investigations.
And NITRO was just a small part of the whole CompStat charge in NYC. By 2001, crime has fallen by more than 60% and NYC was heralded as the safest large city in America, resulting in police departments around the world following the CompStat example.
Its interesting to note that around the same time, a new TV show – CSI:Crime Scene Investigation, became a major hit. It showed how forensic science backed by computers and the latest gadgets, allowed lab geeks AND tech-savvy detectives to collaborate and use “software” to solve a whole new class of crime puzzlers that would have been impossible to crack using “traditional” sleuthing techniques.
NITRO has long been retired and supplanted by a more modern system. I’ve lost track of it since I left the LIMS company in the late 90s when I joined the consulting rat race. But I still look back at those days with fond memories as my small contribution to vastly improved quality of life in NYC.
That’s all very nice and good, but what does this have to do with NYCFacets, you may ask?
I believe we’re in the same stage NYPD was right before CompStat was widely adopted. As NYC embarks on its ambitious Digital Roadmap, it will unleash a flood of information that will dwarf the kind of data we dealt with in the 90s.
Sure, all the Open Data innovators will be more than happy to slice and dice the data for the City. If anything, the record-setting number of submissions for this year’s NYCBigApps is ample proof of that.
But much like CompStat, these modern-day data detectives, will need clues, signals and actionable information to crack the Big Open Data puzzle – “Which dataset, API, or external datasource has the best information for my application?” “How do I ‘connect the dots‘ and make federated queries across datasets through a common vocabulary(ontology)?” “Is there an ontology that allows me to map data from one source, to related data in another datasource?”
It’s a bit of a stretch to say that NYCFacets is the modern-day open data equivalent of CompStat.
However, much like CompStat started with Jack Maple’s “Charts of the Future” – pin maps created by a transit police officer that cut subway crime 27 percent, we’d like to think NYCFacets is the progenitor to an NYC Open Data version of CompStat.
