Arizona's missing crime numbers leave the state in the dark
Arizona and the rest of the country painted a picture of how much crime there was in their communities for decades using a system from the Federal Bureau of Investigation. Now that the system is being replaced, it is getting harder, if not impossible, to pin down crime profiles of our cities and states.
And it could be like that for the foreseeable future.
Recently, the Arizona Department of Public Safety, considered the state's best source for crime data, switched over from a manual review process to an automated one.
The Department of Public Safety also stopped publishing its year-end report on the state's crime trends, something it had done for decades.
One researcher said these changes created a “Dark Age of data.”
Now criminologists and data analysts from across the state echoed the concern that without accurate data, there can’t be reliable explanations for crime spikes or even being certain they exist.
Without those explanations, Arizonans are left in the dark about how safe their communities are or what they can do to protect their families and neighborhoods.
Crime: Arizona sees historic spike in violent crime after COVID-19 pandemic
The old way
Since 1930, the FBI began to collect data on crime across the country using a method called Uniform Crime Reporting, or UCR.
It grouped crime into violent crime and property crime. Violent crimes include rape, murder, aggravated assault and robbery.
Arizona and other states collected data from multiple police, sheriff and other law enforcement agencies using the same method. Once the state collected that data, it would send it to the FBI.
It took many years to get the thousands of individual police and other agencies across the country to participate.
In Arizona, DPS takes on the role of collecting crime data. And each year, the state agency would publish a report going over the crime numbers for each department in the state that participated.
But the old method had its limits.
It would sometimes not count lesser crimes that happened as more severe crimes were being committed.
For example, a robbery or aggravated assault would not get tallied if, during the same case, a murder happened. The old method would only count the murder.
The whole thing was voluntary, but by 2019, more than 88% of the law enforcement agencies participated.
When the FBI switched to a new method, about 53% of departments reported.
Some large departments haven’t made the switch. Those include Phoenix and Tucson.
More on crime: Phoenix murder and violent crime rates break records in 2022
The new way
The FBI created a more detailed and more accurate way of counting these crimes called the National Incident-Based Reporting System (NIBRS).
But it also means that law enforcement agencies have that much more to go through when they collect and review the data they submit.
The new system includes information about the following:
weapon used
relationship to victim
time and location
demographics of all involved
date the crime was solved or cleared
description of any property damaged or the cost of the damage
This extra lift bogged down agencies, and many didn’t submit data to the system in 2020 and 2021.
Phoenix and Tucson are absent from the national numbers.
If this is a problem for larger agencies, it is even more of a burden for smaller agencies who had a hard time reporting under the old system, according to Arizona State University criminologist Jesenia Pizarro.
The associate director at the Center for Violence Prevention & Community Safety, David Choate, agrees.
Just as small agencies were catching up and ready to participate in UCR, the game changed, and they will suffer the most, he said.
But as the reporting system changed, so did the way it gets reviewed, leading to more concerns.
Survey on crime: Tucson residents want broad approach to community safety
No human review
In the past, police and other law enforcement departments would collect data, review it and submit it to DPS.
If there were any fixes or changes that needed to be made, such as an aggravated assault turned into a homicide because a victim died, the department would submit a report explaining the need to change the numbers.
The review of data also happened at the DPS level. DPS employees would manually go over data, and if they found any errors, they would send the data back to the agency.
The process is something of a mystery.
As of 2021, all of the people who did that work have left the department, and DPS can’t explain what the manual review process was or how it worked.
According to DPS, the change happened because of the need for a more detailed reporting system. The software the state used to record data could not keep up, DPS explained in an email to The Arizona Republic.
The new program would be automated, DPS said, without clarifying why the manual review was scrapped.
The department also said that it would no longer be publishing end-of-the-year reports that would summarize crime trends and detail which agencies participated and which agencies did not.
These reports signaled an unofficial deadline for departments and helped mark a time when the UCR data could be used more reliably.
The new system is published live on a website, but it sometimes comes without context.
For instance, data from 2023 appears as if complete and is compared to 2022. As of September, it showed a 44% drop in violent crime but that drop comes from comparing six months of 2023 data to the whole of 2022.
The site shows that Tucson’s 2022 homicides totaled 15, though they were closer to 70, according to Choate.
The site doesn’t explain that Tucson only reported three months of data in 2022, so the number DPS shows is not accurate.
These are important distinctions that should be made to the general public, many researchers explained.
Missing murders
All of these changes leave researchers worried. Some say they don’t trust an automated system, and the gaps that the changeover has created could go on for decades.
Choate was skeptical about relying 100% on an automated system to catch errors, see nuances or make commonsense calls about what data to include and not include.
He cited his experience with automated systems recording suicides in the state.
A system did not categorize the suicide of a 10-year-old as a suicide because of the person’s age; the system was programmed to register minors' deaths differently from suicide, but a person looking over the data confirmed that it was a suicide and made the correction.
Another problem with the changeover is that researchers won’t be able to compare the numbers of violent crimes collected using the new method with the decades of numbers that used the old way.
Researchers agree that it's not going to be a one-to-one comparison, which makes it harder to know what caused some of the shifts in data between 2019 and 2020 or since.
Calling this period a "Dark Age of data is not far off," Choate said. With numbers missing and the inability to compare others, it's creating "black holes" in the data.
Making matters worse, small agencies don't have the resources, time or money to log their data, Choate explained. It's just not feasible for them.
If the technology they already use to register crimes internally could be used to form reports, then maybe they will be able to participate, he said.
Choate worries that this type of department will go missing from the conversation on crime.
That means that large agencies will overshadow smaller ones by influencing state trends. Without the ability to dial into smaller areas, researchers will only be able to explain crime in the state more broadly.
But the causes of crime depend on where they happen and what kind of place they happen in, Choate said.
For example, Choate explained that veterans' suicides in Cochise County happen for different reasons than those in Mohave County because one county has a strong military culture while the other does not.
And if we are only looking at rates from one county to understand the same problem in another, it won’t translate, he said.
These places that can’t report into the new system won’t be able to get ahead of long-term crime trends.
In the meantime
For now, some researchers look toward gathering data themselves, while state researchers hope to create a new repository of crime data.
The Arizona Criminal Justice Commission, which tries to improve public safety across the state, uses data from DPS to compile trends.
But the commission says it is working to become another hub for crime data, independently.
The commission supported a bill in 2023 to give it the authority to do an “audit of all criminal justice resources in the state,” commission spokesperson Molly Edwards said.
Commission staff want to be able to pull data from departments and agencies from across the state, review it, and compile it for researchers and the public.
“If we're collecting from all agencies, all data that they're capturing in their management system or some electronic system, we could eliminate that concern about how many months the (agencies) are reporting, and any type of misreporting data to that DPS level,” said Matt Bileski, the commission's data program manager.
Though the bill was vetoed, the commission will help write a new bill in hopes it will make it past the governor's desk. They believe getting this bill passed will help create a long-term hub for Arizona crime data.
Though there are issues with the system, the DPS data still remains the best source, according to all of the researchers whom The Arizona Republic spoke to.
The department still puts out data using the old method of UCR along with the new NIBRS method though some of the numbers are missing.
To make up for missing data, researchers explain that they request data from police and other law enforcement departments directly or have standing requests with them.
But it takes time, especially if they have to submit a public records request to the more than 100 departments across the state.
“You put in a public records request and some police departments take like two months to respond to you and send you what you need. So we're in an interesting time and a difficult time for research,” Pizarro said.
It's not uncommon for police departments to take many months, even years, to comply with even routine public records requests.
Choate hopes that they can come up with a solution to help bridge the departments and the data during this changeover.
“We have to find a way, else we will go from UCR to nothing,” he said.
And it could be that way for generations.
This article originally appeared on Arizona Republic: ASU, criminologists say Arizona crime data is murky, incomplete
