ChatGPT is a handy AI tool, but it can’t grasp York County’s story

Kurt M. Barnhart grew up in York, lives in California and keeps track of his hometown on Facebook.

Often, with the difference in time zones and all, he’ll share usually playful posts in late evening or early morning on the Retro York Facebook group.

But it seemed that he couldn’t wait to post this one — he shared it one afternoon — about Open AI’s controversial artificial intelligence platform, ChatGPT. This computer system allows you to ask anything and you’ll receive an answer in a conversational, often compelling way.

Acting as a kind of digital almanac, this chatbot scraped or ingested millions, if not billions, of stories. That includes, as it turns out, stuff that I have written and the work of other York County historians and journalists.

Kurt asked this chatbot about the most significant event in York County history.

He also asked: “Who is Jim McClure of York, Pa.?”

“It appears that ChatGPT is not all that smart,” he wrote to me on Retro York. “I guess this is your Mark Twain moment!”

He meant that rumors of my death have been exaggerated. ChatGPT said I had died on Nov. 7, 2021. Kurt wanted me to know this — or else ensure that I was still above ground.

That detail aside — I’m actually the one writing this — the chatbot’s accuracy on the significant event question was a mixed bag. Artificial intelligence, the simulation of human intelligence processes by machines, chose four legitimate moments as significant. Its selection was incomplete, and when it came to accuracy, it fell far short.

ChatGPT has been criticized across many fronts, with fears that students will use it to do their homework and papers, particularly because the AI responses are written so well and convincingly that students might be tempted to cut from ChatGPT and paste into assignments.

Here are ChatGPT’s answers about significant county events, followed by my commentary:

Continental Congress

ChatGPT: “In 1777, the Continental Congress met in York, Pennsylvania, making it the capital of the United States for one month during the American Revolution.”

My response: First, the good news. The chatbot did not buy into the “First Capital” myth, a marketing ploy developed in the 1960s that is not recognized by local or national researchers. Meanwhile, York’s status as the place that the Continental Congress adopted the Articles of Confederation, America’s first constitution, is widely acknowledged.

The bad news: Congress met in York for 271 days — in 1777-78, the Valley Forge winter. The chatbot might have conflated the one day Congress met in Lancaster with the nine months America’s founders deliberated in York.

What is ChatGPT? Everything to know about OpenAI's free AI essay writer and how it works

More:How artificial intelligence — AI — will change the future of health care | Opinion

American Civil War

ChatGPT: “… York served as a major center for Confederate activity and was the site of several significant battles and skirmishes.”

My response: That offers an incomplete story. To be sure, more than 11,000 Confederates invaded York County in the days before the Battle of Gettysburg.

But in a big way, it also was a site for Union activity. A major training camp for green Union troops operated at the York Fairgrounds, at King and Queen streets early in the war. And from 1862 to 1865, a large Civil War hospital treated thousands of wounded Union soldiers at York’s Penn Park.

Further, thousands of Union cavalry and infantry troops marched through here on their way to battle in Gettysburg.

There was one major battle, the Battle of Hanover, that delayed the main Confederate cavalry column commanded by Gen. Jeb Stuart from joining Gen. Robert E. Lee’s infantry until the second day of fighting in Gettysburg.

In Wrightsville, the Union Army’s supervised burning of the milelong covered bridge over the Susquehanna stopped the Confederate advance toward the backside of Harrisburg after a battle at the bridgehead. Other encounters on York County soil were skirmishes, and it’s hard to understand how a skirmish can be significant, almost by definition.

Great Depression

 ChatGPT: “… York became a center of social and political activism, with many residents participating in rallies, strikes and other forms of protest.”

My response: AI is trained by humans, and some critics point out that the trainers are in San Francisco and the Silicon Valley, which means that their politics likely would be liberal. That bias might be apparent here because there was much more to the Great Depression in York County than activism.

Clearly, there was social turmoil. Some of the Black newcomers to York in the Great Migration considered returning, or actually traveled back, to their former homes in the South. Jobs, scarce everywhere, were particularly so for Black people.

But the chatbot’s response ignores the fact that Martin Library, Belmont Theatre and the York Symphony were among the iconic organizations that started as diversions amid the depression.

Also, there’s an exaggerated claim in York County that the Depression only pinched local banks and did not punch the county’s economy. The fact is that while many York city banks weathered the Depression, there were bank closures throughout the county.

And the Great Depression was bracketed by the Hex Murder — a horrible case involving superstition and witchcraft — and the county reaching its then-high-tech best in defense work in World War II. Something happened in this complex place we call York County in the Great Depression that we could make that transition.

Industrial Revolution

 ChatGPT: “York was an important center of industry during the Industrial Revolution, with many factories and businesses … .”

My response: It is true that the entire county contributed to and benefited from the Industrial Revolution, an era that generally extended from the end of the Civil War to the start of World War II.

But that could be said of many cities across America. York County was very important, but not unique in this period.

Maybe the chatbot extended the Industrial Revolution through World War II. As we’ve seen, the county’s reputation gained a spike in its support for the World War II’s York Plan, the sharing of inventories of manpower and machinery among companies so that major defense work flowed to local factories. If the companies had acted alone, they would not have captured such huge defense contracts.

Predictable findings

ChatGPT’s selection of significant events was predictable and could be applied to many American cities or counties.

It’s all-too-common to focus on wars and depressions as shaping events when York County — as any community — has a multitude of other significant events.

I have developed such a list of pivotal York County moments with 40 entries. For example, the Great Migration mentioned earlier led to development of Crispus Attucks Community Center and the building of two segregated elementary schools. Many families influential in the county today came here in those days.

A community is defined by many events — wars and those moments that are nuanced and understated. A lot has been written on the Great Migration in York County, so ChatGPT has work to do not only in accuracy but in the completeness of its stories.

Matrix weighs in

Kurt Barnhart must have felt bad about delivering me “mournful news” about my death, so he checked with another AI source for “a second opinion.”

He got a much better result that time.

“According to Microsoft Bing, you are, apparently, STILL alive!” he wrote on Facebook.

When Google releases its new Bard AI, Barnhart wrote, “we’ll ask it to monitor you weekly.”

At some point in this Facebook conversation, Retro York member Gil Tunney chimed in: “Be careful my friend … The Matrix says your dead … . “

This is all good fun, but it does point to shortcomings in this newest of technologies.

“I feel very much alive,” I wrote in this Facebook conversation. “Keep me apprised if anything else changes about that date (of my death).”

Sources: “Good Faith” podcast, “The Dispatch”; James McClure’s “Never to be Forgotten”; YDR.com.

Upcoming presentations

I’m presenting on these topics in OLLI @ Penn State York’s spring term: “A Walk through the Park: York City Parks, That Is,” “Major Crimes and Misdemeanors in York County's Past” and “The York Daily Record and Gathering York County News Since 1796.

Details: Details: https://olli.psu.edu/york/

Jim McClure is a retired editor of the York Daily Record and has authored or co-authored nine books on York County history. Reach him at jimmcclure21@outlook.com.

This article originally appeared on York Daily Record: ChatGPT is handy AI tool, but it can’t grasp York County’s story