The hunt for proficiency is complex and often elusive

Jul. 20—Pop quiz: What number is both a prime number and a factor of 56?

If a store owner purchased 12 times as many small posters as large posters and she ordered 48 large posters, how many small posters did the store owner purchase?

Under what circumstances is it appropriate to use each of the homophones there, they're and their?

And what's the right way to punctuate a line of dialogue in a story?

That's a small sampling of the content contained in released versions of the New Mexico Measures of Student Success and Achievement exam for fourth graders.

The exam is what's known in the education world as a summative assessment: Students' performances on it indicate whether they can be categorized as proficient in math and reading.

Most New Mexico students aren't proficient in either, with just 38% proficient in reading and 24% proficient in math, according to the latest data from the Public Education Department. At some schools — including several in Santa Fe Public Schools — those figures drop even lower.

Proficiency rates are an essential measure by which education officials and members of the public can gauge schools' progress in meeting state academic standards. On Friday, Saturday and Sunday, The New Mexican has printed schools' scores, though only for the most recent year available, 2022-23.

But the numbers don't necessarily tell the whole story, said Santa Fe Public Schools Superintendent Hilario "Larry" Chavez: Discussing student performance in terms of proficiency rates alone tends to separate kids into two buckets — proficient or not proficient — rather than demonstrating whether they're making improvements, he said.

"I don't think anybody in the state of New Mexico is satisfied of where we're at with the number of students who are proficient," Chavez said. "But at the same time, we need to start celebrating the students who are showing growth."

Proficient has a relatively simple definition.

The New Mexico Measures of Student Success and Achievement exam tests whether students can demonstrate knowledge and understanding of the state's education standards for their grade level, said Beata Thorstensen, Santa Fe Public Schools' chief of assessment and accountability.

Essentially, she said, "That means that I know what I'm supposed to know. ... I have learned what I'm supposed to learn."

The more complicated question is: What does it really mean to be proficient? How do we get from a list of standards to a proficiency score?

The federal Every Student Succeeds Act, passed in 2015, requires that public schools test all students in grades three through eight, plus once during high school. Statute requires that those tests be "aligned with the challenging State academic standards, and provide coherent and timely information about student attainment of such standards."

In addition to the SAT — New Mexico's test of choice for determining whether 11th graders are on track for college- and career-readiness — the state tests third through eighth graders using New Mexico Measures of Student Success and Achievement, an exam created by the Public Education Department and the Georgia-based assessment development company Cognia, to meet the federal requirement.

The test is aligned to New Mexico's unique educational standards — the lists of benchmarks, numbering in the hundreds of pages, that the state's students are meant to meet from kindergarten through high school graduation.

The bar for proficiency, then, is determined by a team of educators who complete a process called "standard setting," Thorstensen said.

Those educators analyze the test's questions and student responses to decide where the line of proficiency sits.

"Once they've determined those cut points, then they can proceed to build proficiency numbers," Thorstensen said. "It's a very technically complex process."

New Mexico's proficiency numbers tend to be pretty honest, she added; they tend to mirror proficiency rates determined by data from the federal Department of Education.

There are things that the test results don't include, though, Thorstensen and Chavez agreed.

For instance, one core element of the state's English language arts standards are known as "speaking and listening standards" — which is impossible to test for on an individual, multiple-choice exam.

"The biggest thing that gets left out of the picture is growth," Thorstensen said.

The test separates students into groups based on performance: Novice, Nearing Proficiency, Proficient and Advanced.

Districts can track students as they move from, for example, proficient in fourth grade to proficient in fifth grade — a change Thorstensen said indicates the student has made school year's worth of growth in one school year. In other words, the student is right on track.

She added that what tends to get lost — and what's harder to measure — is whether students who are not yet proficient are making progress toward proficiency. Thorstensen gave the example of a student moving from the bottom of the nearing proficiency band to the top of the same band the next year. In both cases, that student would still be labeled as "nearing proficiency" while advancing by more than a year of academic growth.

That year-over-year improvement is what's missing from proficiency numbers, Chavez said.

"For us, as a district, we're always trying to get students to be proficient, to be at grade level — but we are well aware that many of our students are not only maybe a year behind but maybe two or three years behind," the superintendent said.

He continued, "We're really focused on the group of students who are showing growth because that growth will eventually, in our mind and in our theory, get them to become proficient."