House Majority Leader Kevin McCarthy pulled out of the race for House speaker Thursday, throwing uncertainty on congressional leadership ahead of crucial budget negotiations.

House Republicans said the party's leadership election would be delayed from Thursday to a later date that has not been determined. McCarthy had been considered the frontrunner to replace Rep. John Boehner after the Ohio congressman surprisingly announced he would leave at the end of October. Boehner said Thursday he will remain in the position until a successor is elected.

"If we are going to unite and be strong, we need a new face to help do that," McCarthy told reporters. "I feel good about the decision."

Read More McCarthy drops out of speaker race; who's left?

The House speaker will hold a crucial role in negotiations to fund the government later this year, as a recently passed stopgap spending measure expires in December. Some lawmakers have threatened a government shutdown over federal funding of Planned Parenthood.

At a party meeting, McCarthy told fellow Republicans: "I am not the one at this time," expressing doubt that he could garner the 218 votes needed to win the position. Some within the party have floated South Carolina Rep. Trey Gowdy, Utah Rep. Jason Chaffetz and Florida Rep. Daniel Webster as possible successors to Boehner. Chaffetz said Thursday that McCarthy's departure creates an opening for his candidacy.

Read More Boehner: House leadership turmoil would do harm

California Rep. Darrell Issa said he did not believe any current candidate for the position could have won a majority vote in the full House chamber.

In a statement, Wisconsin Rep. Paul Ryan stressed he would not pursue the position. The 2012 GOP vice presidential candidate said he was "disappointed" in McCarthy's decision, calling him "the best person to the lead the House."

Read More Obama: 'Still a path' to budget compromise

McCarthy, the House majority leader from California, recently received criticism for implying the special panel investigating the 2012 attack on a diplomatic compound in Benghazi, Libya, helped to reduce support for Hillary Clinton's presidential bid. On Thursday, he said he "should not be a distraction from" the invesigation.

McCarthy acknowledged that his comments did not help his case for speaker.

Because his decision to leave the race could lead to Boehner holding the position longer than anticipated, the situation could bode well for a debt deal, said Edward Mills, a financial policy analyst at FBR Capital Markets. Boehner is seen as more willing to compromise with Democrats, especially on his way out of office.

Read More So...what happens when the government shuts down?

Republican presidential frontrunner Donald Trump chimed in on the House leadership, saying on Twitter, "we need a really smart and really tough person to take over this very important job."

Read More House conservatives back Webster for speaker

Click here for the latest on the markets.

— Reuters contributed to this report

Correction: An earlier version misstated Webster's state. He is from Florida.

More From CNBC

• Top News and Analysis
  

• Latest News Video
  

• Personal Finance