As the debate over the addition of a citizenship question for the 2020 census has continued to rage in U.S. Congress, experts and laymen in the public sphere have reckoned with the potential fallout of such a development.

The greatest fear among policy analysts and average Americans alike is that the question will cause millions of people, including many legal immigrants, not to respond to the census at all. This, in turn, could cause an inaccurate distribution of federal resources and congressional seats, leading to inadequate representation for an estimated 4 million people.

“This census has become this administration’s new front in its war on immigration and immigrants. That context has created a real climate of fear,” Teri Ann Lowenthal, former staff director of the House census oversight subcommittee, told Fortune.

But the contemporary fight over the citizenship question is hardly the first time that the census has been used to partisan ends, often in discriminatory ways.

From its inception, the U.S. census has done far more than just take population counts—it has served repeatedly as a tool to name, disenfranchise, and even root out ethnic “undesirables” as recently as the 1940s.

Lowenthal and other census experts were quick to point out that it’s worth noting a difference between the census being used to discriminatory ends and the census results themselves being manipulated by racism. The question of who should be counted and how has always been a highly subjective enterprise.

“I think the Trump administration’s push to add a citizenship question to the 2020 Census essentially uses the census process itself to achieve similar discriminatory goals, in this case a likely disproportionate undercount of people of color that will reduce political representation for these communities just as the three-fifths compromise did 230 years ago,” said Lowenthal.

The roots of the U.S. census go all the way back to the American Revolution and its quest to ensure fair and representative government. The census is even enshrined in the U.S. constitution, with article one, section two, calling for it to be conducted once every 10 years. And from the start, the census was defined by much of the moral ambiguity and racial tensions that have continued to plague it today.

The first census was far simpler than current versions, and it counted only the number of white men under and over 16 years old, the number of white women under and over 16, the number of other free persons, and the number of slaves. Native Americans did not count, according to the founding fathers, because they were not tax-payers. After the notorious three-fifths compromise, each slave would only be counted as three-fifths of a person.

As the nation grew exponentially in the next century, the census was in many ways a vital tool for ensuring that each state was apportioned fair representation. When eugenic racial pseudoscience began to expand in popularity in the middle of the 19th century, terms like “mulatto,” “quadroon,” and “octoroon” begin to appear in language around the census. The 1850 census instructed enumerators:

Be particularly careful to distinguish between blacks, mulattoes, quadroons, and octoroons. The word “black” should be used to describe those persons who have three-fourths or more black blood; “mulatto,” those persons who have from three-eighths to five-eighths black blood; “quadroon,” those persons who have one-fourth black blood; and “octoroon,” those persons who have one-eighth or any trace of black blood.

Even after the end of the Civil War, such categories lingered on the census. As Jennifer L. Hochschild and Brenna M. Powell pointed out in research for Harvard, an Alabama Congressman in 1888 called on the census to uncover the birthing rates of “negroes, Chinamen, Indians, and half-breeds or hybrids of any description.” Carroll Wright, then commissioner of labor and acting census superintendent in the late 1880s responded: “the sooner the statistics are collected the better.”

In 1850, the census also started asking respondents about their country of origin and naturalization status. Ostensibly the questions, as well as data about English fluency and residence of the respondents, were meant to measure how well people were assimilating into American society. But the results of that census data would end up being used for an altogether different purpose: immigration quotas. The Dillingham Commission, which investigated U.S. immigration patterns at the start of the 20th century, pointed specifically to census data in making its recommendation that immigration be restricted. It was Dillingham and his assistant, relying on data provided by the census, that proposed the quotas that would become law just a few years later.

The history of the census and ethnicity, and even where this question is concerned, is still murky. Particularly in the 1990s, some activists from communities of color wanted questions about ethnicity to be asked and advocated for multi-racial categories to appear on the census.

Meanwhile, the 20th century continued to witness interactions between the census bureau and other branches of the government. The census bureau had also long denied any involvement in the incarceration of more than 100,000 people of Japanese descent—most of whom were American citizens—during World War II. But in 2000, historian Margo J. Anderson of the University of Wisconsin at Milwaukee and statistician William Seltzer of Fordham University found proof that census officials provided block-level information on Japanese-Americans living in California, Arizona, Wyoming, Colorado, Utah, Idaho, and Arkansas. And in 2007 they discovered that census officials had later shared names and addresses of people of Japanese origin in Washington, D.C.

The census bureau issued an apology on this several decades after the end of the war, but many census officials and the researchers themselves are still quick to point out that what the census bureau did was not illegal at the time, thanks in part to wartime legislation.

The legality tightened around sharing census data starting in the 1950s, and the World War II-era legislation that allowed census data to be used in rounding up people would no longer be legal. It’s also worth noting that the census began to be used as a tool for integration in the 1960s following Civil Rights legislation, with the data being employed to measure and root out inequality.

“By and large, the census has been a tool for democracy,” Margo Anderson said in a phone interview. The census has always been a product of its time, she argued. “Is the census any more discriminatory than Congress? Than the courts? It’s just another piece of the enterprise.”

More must-read stories from Fortune:

—2020 Democratic primary debates: Everything you need to know

—The campaign finance power behind Trump impeachment efforts

—Not every state is restricting abortion rights—some are expanding them

—Richard Nixon‘s “Western White House” is back on the market—at a discount

—Trump administration to use former Japanese internment camp to house migrant children

Get up to speed on your morning commute with Fortune’s CEO Daily newsletter.