Red Hat paywall?! How the Raleigh giant divided the open source community.

On June 21, Red Hat Vice President Mike McGrath made an announcement that split the open source software community.

In a 318-word blog post titled “Furthering the evolution of CentOS Stream,” McGrath revealed that Raleigh-based Red Hat, the world’s largest provider of open source software, would begin to reserve the source code of its flagship product, Red Hat Enterprise Linux, to paying customers only.

In most industries, such a decision would not be controversial. Only those who pay are generally entitled to a good or service. Yet within the distinct world of open source, where free access to information is paramount, many saw Red Hat’s decision to essentially paywall Red Hat Enterprise Linux, or RHEL, as sacrilegious.

Word of the new policy sparked a furor online. “Within an hour or two of (publishing) the blog post, they were all aware of it,” McGrath later told The News & Observer.

Initial reactions were a mix of surprise and anger. “RHEL paywall jeopardizes long-term health of Red Hat,” one tweet read.

In a YouTube video viewed more than 100,000 times, tech commenter Chris Titus wondered if the moment marked a sea change for the open source movement, which in 40 years had morphed from a hobbyist-driven subculture to a corporate-backed behemoth.

“I never thought I’d see the day Red Hat went closed source on us,” he said.

Essentially everyone today encounters open source software in some manner. It enables countless daily tasks: booking airline tickets, requesting ride shares, engaging with health care portals, toggling through smart TVs and much more. It has also pervaded big businesses, as Red Hat touts that more than 90% of Fortune 500 companies rely on its services.

The linchpin of this software is the free exchange of information. Open source is strengthened by a huge community of paid and volunteer programmers who reject siloing code behind proprietary ownership. They are fiercely protective of the notion that software performance and security are enhanced when more eyes are watching.

So it was little surprise that Red Hat’s move struck a nerve.

“Red Hat is the only (company) I know who’s done the paywalling,” said Chris Tozzi, whose 2017 book “For Fun and Profit” chronicled the rise of open source software. “It’s a divisive issue in the open source community, which to me is the story. The community is no longer reaching consensus on how open source should work.”

Tozzi said changes to open source licensing threaten to complicate how the public uses the software, and there are fears that others will follow Red Hat’s lead. If it becomes a trend, the number of developers who add to each code would diminish and providers might begin to feel pressure to charge for their code.

Red Hat employees were also conflicted about the new policy, McGrath acknowledged. “I think a lot of even internal associates didn’t fully understand what we had announced and why,” he said.

He recalled some of the backlash being personal: people online alleging he was sent by IBM, Red Hat’s parent company, to undermine the iconoclastic 30-year-old Triangle software firm that employs more than 2,000 people in the area.

“No one from IBM came to talk to me about this decision before, during or after it,” McGrath, who has been at Red Hat for 16 years, said. “I’m always amused by people that think IBM execs have meetings about CentOS and rebuilders. I don’t think they really know what it is.”

Five days after the blog post, McGrath published another, longer message on the company website. He didn’t back down, instead reiterating Red Hat’s decision to stop granting non-customers unfettered access to the RHEL code.

At issue, he wrote, were emerging competitors who copied Red Hat Enterprise Linux, down to even the code’s mistakes, and then offered these Red Hat-replicas to customers for free. These weren’t community members adding value, he contended, but undercutting rivals. And in a year when Red Hat laid off 4% of its total workforce, McGrath said, the company could not justify allowing this to continue.

“I feel that while this was a difficult decision between community and business, we’re still on the right side of it,” he told the N&O.

Not everyone agrees.

Free as in speech, not beer

To understand the Red Hat controversy, it is important to understand how open source started — and what it exactly means.

“It goes back decades,” Tozzi said. “There’s this really deep-seated feeling that control is important. And Red Hat has taken away a level of control.”

In 1984, a researcher named Richard Stallman released a software project called GNU. Stallman licensed GNU for free, with his only stipulation that users sign an agreement called the GNU General Public License. It contained four founding principles. Referred to now as the GPL, this agreement emphasized that people had freedom to use, change, share and even repackage and sell the software.

To Stallman, freedom meant no restrictions — not necessarily no costs. “Think free as in free speech, not free beer,” he is quoted as saying.

GNU stood in ideological contrast to what would back then become the world’s largest software operating system: Microsoft Windows. Windows was closed and proprietary. In 1985, Microsoft marketed its first Windows release for $99.

At that time, GNU had a significant limitation. It lacked what is known in software as a kernel, a central piece of an operating system. Then in 1991, a Finnish student named Linus Torvalds created a PC operating system kernel called Linux. Inspired by Stallman, Torvalds granted the right to anyone to commercially repackage Linux so long as they agreed to the GPL.

“Linux was the right product at the perfect time,” said Michael Tiemann, a software engineer who in 1989 helped start the world’s first commercial open source company, Cygnus Solutions, which Red Hat later acquired.

Red Hat takes over the (open source) world

One of Linux’s key advantages, Tiemann says, is that its code is visible. If Linux were a car and there was a problem under the hood, its owner wouldn’t be reliant on a specific automaker to fix it using protected instructions. Instead, a contributor community, thousands of mechanics strong, could make fixes that everyone could then check.

In this free software, Red Hat saw a business opportunity. Launched in 1993 from the Durham apartment of cofounder Marc Ewing, the company spent its first few years releasing bespoke versions of Linux operating systems on CDs which it marketed out of bookstores to computer science enthusiasts.

By decade’s end, Red Hat had made inroads into the corporate world. In 2001, Red Hat released the first version of Red Hat Enterprise Linux, a package geared toward businesses. Rather than CDs, the company started to sell RHEL subscriptions for consulting and security services.

Red Hat continued to grow — both in the industry and the Triangle. In 2013, the company transferred more than 700 employees to its new tower headquarters in downtown Raleigh. Six years later, IBM acquired the software firm for $34 billion.

“Red Hat is, by most measures, the most successful open source company in history,” Tozzi said. “I can’t think of another who comes close both in terms of financial success and in terms of being a leader in the open source community.”

No longer an underdog, Red Hat has continued to flourish with corporations. The company has recorded double-digit revenue growth every fiscal quarter except one since the IBM acquisition. And in its 2022 survey of nearly 1,300 technology leaders, more respondents said they used open source than proprietary software.

“I think long gone are the times of that sort of romantic view of hobbyists working in their spare time to build open source,” McGrath said. “I think there’s still room for that — we still have that — but quite a lot of open source is now built from people that are paid full time.”

Downing the downstream clones

The Linux kernel is one of several operating systems built over time by a coalition of thousands. Some are unpaid hobbyists who contribute code in their spare time. Others are employed by big tech firms like Google, Meta, Red Hat and even Microsoft to write code and send it “upstream” to the Linux kernel.

To make Red Hat Enterprise Linux, Red Hat repackages this free code from upstream sources, including the kernel and the Red Hat-owned Fedora Project. In a similar way, downstream distributions were, until recently, able to take RHEL’s code and repackage it themselves in a process known as forking. They could also contribute code back to RHEL’s upstream sources.

This year, Red Hat decided the downstream distributions weren’t sending enough upstream to overlook the level of copying they were doing.

“There are many, many ways in which freeloading in open source is a feature, not a bug,” said Tiemann, who left Red Hat last year after more than two decades with the Raleigh company. “In this particular circumstance, Red Hat basically decided, ‘You know what? It’s not a feature anymore.’”

This was not the first step Red Hat has taken in this direction; in late 2020, the company announced it would terminate CentOS Linux, which had been a sanctioned free version of Red Hat Enterprise Linux. To fill this gap, a pair of prominent downstream RHEL distributions, RockyLinux and AlmaLinux, emerged the following year, each advertising themselves as bug-for-bug replicas of RHEL. “For all intents and purposes, they were exactly the same,” said Jonathan Wright, the infrastructure team lead at AlmaLinux.

“We would rip out the Red Hat name, repackage it and release it,” he said. “Basically make sure it did not infringe on their trademark.”

While acknowledging his company’s aim was to clone, Wright still believes Red Hat’s move to restrict RHEL code violated “the spirit of the GPL.”

He said no-cost downstream replicas like AlmaLinux attracted a larger, more diverse group of contributors to the open source community — a community that professes to thrive on strength-in-numbers.

“The foundation of (open source) is to make your work consumable and available to everyone,” Wright said. “Be it someone who can barely get on the internet on the other side of the world or a Fortune 500 here in the US. That’s really what I think Richard Stallman was going for back in the day.”

Wright and other critics of Red Hat’s policy shift also point out that RHEL itself is based on the accessible coding contributions of others, though he recognized Red Hat employees do extensive work to make RHEL into a useful, stable and widely popular consumer product. He also said AlmaLinux has, since its founding, contributed code back upstream to the wider open source ecosystem.

Open source community moves forward

McGrath offered little consolation to customers who were relying on one-for-one versions of RHEL. They could stay with the downstream distributions, find another provider, or pay for Red Hat.

“I think (people) were just so used to the way things work,” he said. “There’s a vocal group of people that probably need Red Hat’s level of support, but simply don’t want to pay for it. And I don’t really have… there’s not much we can tell them.”

Since its RHEL decision, Red Hat has secured several prominent partnerships. In September, the cloud-based software company Salesforce moved 200,000 of its systems from the free CentOS Linux to Red Hat Enterprise Linux. The same month, Red Hat announced RHEL would begin to support Oracle’s cloud infrastructure. Oracle was one of the few major companies this summer to publicly criticize Red Hat for essentially paywalling its most popular code.

On Oct. 24, Red Hat notched another win when the data security firm Cohesity said it would also ditch CentOS Linux for RHEL.

McGrath pushes back on the notion that RHEL was paywalled, saying people could still freely access the code upstream through a distribution called CentOS Stream, which is a beta version of RHEL.

Critics note that CentOS Stream is not the same as the RHEL source code as it is earlier in the development process. Wright described it using a cooking analogy: CentOS Stream may offer the ingredients but not the exact proportions.

The source codes of other Red Hat programs, like the automation platform Ansible, are still accessible to non-customers.

In mid-October, Wright traveled from Birmingham, Alabama, to attend the All Things Open conference, an open source event held every year in Raleigh. Red Hat is among the conference’s biggest sponsors.

“Every other question (people asked me) was about how the change is affecting us,” he said.

In July, AlmaLinux officially pivoted its aim away from being an exact copy of RHEL, with the company instead stating it will offer an operating system that is compatible with RHEL. Three months later, Wright said Red Hat’s decision has made AlmaLinux better.

“It’s almost like we’re off our crutches now,” he said. “We’re kind of making our own path.“

So does he now agree with Red Hat’s decision?

Not at all.

Wright believes Raleigh’s software giant played “fast and loose” with the GPL this summer. For many open source believers, such a threat to its hallowed text isn’t forgivable.

Open Source

Do you enjoy Triangle tech news? Subscribe to Open Source, The News & Observer's weekly technology newsletter and look for it in your inbox every Friday morning. Sign up here.