‘We are attorneys, not statisticians.’ Travis County DA reviewing conviction data after KXAN reporting
Investigative Summary:
The Travis County district attorney publicized a sharp increase in the conviction rate for violent, sexual and weapons-related offenses, but KXAN investigators found the formula used to calculate these statistics left some experts with questions. After KXAN’s reporting, the district attorney said his office is re-evaluating the method and trying to find the best way to present its data.
AUSTIN (KXAN) — Flanked by supporters and other top county leaders, José Garza announced his re-election campaign for Travis County District Attorney with a strong message: “Across the street, they can play politics with our public safety, but here in this community, together, we choose to act.”
As Garza called out the Texas Legislature for targeting prosecutors’ offices, he highlighted data on the successes of his office – particularly noting an increase in wins for cases of violence and family violence.
“Over the last four years, we have increased the conviction rate for violent crimes from 38% to 91%,” he said.
In the weeks that followed that August re-election announcement, his campaign touted the same increase in emails to supporters, on social media and on his candidate website.
Conviction rate messaging often comes up during campaign season, according to David LaBahn, president of the Association of Prosecuting Attorneys.
“It’s very common that — whether it’s asked by an opponent, whether they put it out themselves — they’ll come out and say, ‘I have a conviction rate of 90% or, you know, we’re doing really great in our violent crimes, or something like that,” he said.
However, the association recommends prosecutors avoid using conviction rates as a measure of success for several reasons.
For one, Labahn emphasized that the role of a prosecutor is to seek a just outcome in a particular incident, not simply to win the case. Beyond that, he pointed to the many variables that could go into calculating a conviction rate – meaning the public would need to know exactly how a rate was calculated to understand what it means.
“The trouble with a conviction rate is — it’s really — it’s a data point. Right? It doesn’t show you the whole of what the situation was,” LaBahn said. “What did you put in it? You can get really thick on this. And what did you get out of it?”
What is the formula?
In May, during an interview about the number of cases dropped by his office, the district attorney told KXAN investigators his office’s highest priority was prosecuting violent, sexual and weapons-related offenses.
“I think that is reflected in the number of convictions that we are securing and the percentage of cases that come through our door that are resulting in convictions,” Garza said at the time.
His team also provided several tables with percentages, showing a conviction rate for different types of felony crimes each year since 2018 – including the rate of 91% in 2022 for violent, sexual and weapons-related crimes.
In the weeks that followed, KXAN requested the raw data underpinning these percentages and asked for the formula used to calculate them.
Is Travis County DA Garza dropping more felonies? Here’s what the data show
Over a series of conversations, his office explained they looked at the total number of convictions secured for a certain class of offenses during each listed year, regardless of the year the case was received. Then, the office divided this number of convictions by the total number of cases received for that same class of offenses in the same year, that were ultimately closed.
For example, the office took the 1,977 convictions in violent, sexual or weapons-related cases obtained by the office in 2022, regardless of what year each case was received by the office. Then, it divided that number by the 2,165 of those same types of cases that the office received in 2022 and later closed – a calculation that comes out to 91%.
This graph shows the cases considered in the Travis County district attorney’s conviction rate calculation compared to the formula suggested by an expert to KXAN. Only felony cases of violent, sexual and weapons-related felony offenses are included in this graph. Source: Travis County District Attorney’s Office. (KXAN Interactive/Christopher Adams)
Garza explained during an October interview with KXAN, his office was obtaining similar raw numbers of convictions for violent offenses each year — despite outside factors such as court closures during the pandemic and a backlog of cases to work through.
“What we were looking for was a formula that captured the reality that we were seeing in the raw data, and what we were seeing in the raw data is it looked to us like our prosecutors were incredibly effective and efficient. Because what we were seeing is that the number of cases coming into our office — pre-pandemic to now — has declined significantly, we’re just receiving fewer felony cases post-pandemic than we were before. But what we are also seeing is that we are securing a comparable number of convictions, even though the number of cases coming in has declined.”
This graph shows the number of felony cases of violent, sexual and weapons-related felony offenses received each year and ultimately closed by the Travis County district attorney’s office. Source: Travis County District Attorney’s Office. (KXAN Interactive/Dalton Huey)
Alex Piquero, a criminologist and professor at the University of Miami, said he had never seen this specific formula used to calculate a conviction rate.
Piquero said he has worked with police chiefs, district attorneys, and public defenders to try and improve data collection and use in the criminal justice system. He also served as director of the Bureau of Justice Statistics for one year, after being appointed by President Biden last year.
FBI report: Violent crime decreases to pre-pandemic levels, but property crime is on the rise
He noted the importance of measuring the outcome of the same batch of cases over time when calculating a conviction rate — which he did at the federal level by taking the number of cases filed in a year and seeing how many of those resulted in a conviction sometime in the future.
“It could be in that year, but it could also be in that next year, but you’re still following the exact same case over time,” Piquero explained.
For example, 755 of the 2,165 cases of violent, sexual, or weapons-related offenses that were received in 2022, and eventually closed, were closed by conviction. That calculation comes out to 35%.
In mathematical terms, Piquero said the numerator should always be a function of the denominator. Otherwise, the equation contains a number of cases that are not comparable. For example, the 1,977 convictions in the district attorney’s method includes more than 1,500 convictions in cases received prior to 2022. Those cases are not included in the 2,165 received cases that made up the denominator in this equation.
“It could still be comparing apples to apples, but one’s red and one’s green,” he said. “It’s still an apple, it’s still a conviction, but it just doesn’t come from the same tree or same orchard.”
Piquero said he believes outside factors, such as workload and incoming cases are important, but separate, data points to consider.
“They’re both equally important, right? You need to understand what the workload is because then you have issues of staffing and resources and those kinds of things — that’s perfectly legitimate, but that is a different question that has nothing to do with the conviction rate,” he said.
LaBahn said these variables and influences are the reason he said a conviction rate is an unreliable way to measure what an office is doing.
“Crime is always either rising or falling. You’ve got a — very much — a human system. Also, there’s so many alternatives as to what happens to a case. Does it get charged? Does it come in as a felony and get charged as a misdemeanor? Does it get diverted?” he posed.
“KXAN brought us new information that we are evaluating, and we just want to make sure that we get that stuff right.”
José Garza, Travis County District Attorney
However, LaBahn agreed that if an office chooses to calculate them, it does need to follow the outcomes of the same group of cases.
“You can’t take a case from 2018 and use it as a conviction in 2023. Unless you’re using other cases from 2018 and following them longitudinally. So, there’s got to be a relationship between the top number and the bottom number, and it has to be a causal relationship,” LaBahn said. “Otherwise, you’re engaged in very much fuzzy math.”
The district attorney’s conviction rate includes convictions obtained through plea agreements and guilty verdicts reached at trial.
LaBahn noted that prosecutors will sometimes tally their trial record, or the number of convictions obtained solely in cases brought to trial in a given year.
“A very small percentage of the cases go to trial,” he explained, making it a simpler calculation.
For example, the Travis County district attorney’s office obtained a conviction in the one murder case it took to trial in 2021 — a rate of 100%. In 2023, the data shows their office obtained seven convictions of the eight murder cases it took to trial that year — a rate of 88%.
This graph shows the types of convictions for cases received by the Travis County district attorney’s office in 2022. Only felony cases of violent, sexual and weapons-related felony offenses are included in this graph. Source: Travis County District Attorney’s Office. (KXAN Interactive/Dalton Huey)
‘The best snapshot’
KXAN brought Garza’s team the formula proposed by Piquero and used by the bureau. In response, Garza expressed concern about how open cases could influence the percentages, particularly when calculating rates in recent years — when more cases are likely to still be making their way through the court system.
“Because we have been in office for about two and a half years, we don’t have the benefit of the long look of how the cases resolve. So it was important to us to be able to give the best snapshot we can for the two years that we’ve been in office,” he told KXAN. “When you look — for example, using this formula — when you look at 2018 data, you should have a pretty full data set and a pretty full understanding of how cases were resolved. On the other hand, when you look at 2022 data, I think the reality is that most — especially violent cases, which are our most complex — those cases take longer.”
However, it should be noted the data his office used to calculate its own percentages only contains closed cases.
KXAN investigators requested conviction rates from other Texas district attorney’s offices, as the statistic was not readily publicized by their offices. Most said we would need to formally request the data through the public information system.
There is also no national standard for calculating conviction rates.
Piquero said he tried to address the disjointed nature of crime and courts data during his time as director at BJS. In May 2023, his agency delivered a report focused on policing to the president, but Piquero said one of the main conclusions was the need to more standardized data throughout the system.
“Standardization isn’t trying to control things. It’s trying to make it easier for journalists and researchers in the American public to say, ‘OK, well, I know now what happened in Seattle, and I can compare it to what happened in Kansas City and in Dallas because it’s the same or it’s collected the same,” he said.
He had another message for any agency putting out this kind of data: “The more transparent agencies can be about how they’re collecting their data and how they’re presenting their calculations, makes it easier for you, for me and the American public to understand what those numbers are about.”
‘Which makes the most sense’
When KXAN interviewed Garza about conviction rates in October, he said his office was still working to understand what data was “most valuable to the community” that it serves.
He went on to say, “I think it’s important for viewers to know that we shared with KXAN the data set and the formula that we used to secure this number. It is important to us to be transparent about how we got there, and we’ll continue to do that.”
By November, KXAN investigators noticed the 91% statistic had been removed from Garza’s campaign website.
The image on the left is a screen capture from Garza’s campaign website in October. KXAN has highlighted the portion of the website featuring the 91% claim. The second image on the right is a screen capture from November of Garza’s updated campaign website. Use the sliding tool to compare the differences in the images. (KXAN Interactive/Christopher Adams)
When we followed up with him about the change, Garza said his team was evaluating how they would calculate their conviction rate, following KXAN’s reporting.
“You know, at the end of the day, we are attorneys, not statisticians, and so… we take the presentation of data very seriously, and, as I said, KXAN presented us a couple of different formulas,” he said. “And so, we just want to work through those and figure out which makes the most sense – or if there is another way to present this data that is clear and transparent and that captures all of the factors that we are seeing.”
Digital Data Reporter Christopher Adams, Investigative Photojournalist Richie Bowes, Director of Investigations & Innovation Josh Hinkle, Investigative Producer Dalton Huey, Lead Editor Eric Lefenfeld, Investigative Photojournalist Chris Nelson, Digital Executive Producer Andrew Schnitker and Graphics Artist Christina Staggs contributed to this report.
For the latest news, weather, sports, and streaming video, head to KXAN Austin.
