"And so yeah, I'm sleeping a little better," he told "World News" Anchor Diane Sawyer in an exclusive interview. "But again, I think it's really important not to be complacent. We have a long way to go, a lot of work to do, and we're going to keep doing that."
[Watch "World News With Diane Sawyer" tonight for the interview with Ben Bernanke, Federal Reserve chairman, at 6:30 pm EDT]
Bernanke, chairman of the Board of Governors of the Federal Reserve since Feb. 2006, acknowledged that gas prices are a "major problem" but he called them a "moderate risk" in threatening the economic recovery. The national average price for a gallon of regular has risen to $3.92, according to the Energy Department.
"They're obviously a hardship for lots of people. It must be awfully frustrating to get a small raise at work and then have it all eaten by a higher cost of commuting," he said.
Bernanke said higher gas prices may cause inflation to be "a little bit higher" in the next few months and take a bite out of consumer spending, leading to "a hit on growth."
"But at this level we don't think yet that - particularly given the other good news we've seen in labor markets and so on - we don't think it's going to be anything that's going to stall the recovery," he said.
He said it's "far too early to declare victory" on a strong economic recovery and the national unemployment rate of 8.3 percent is still "too high."
The head of the U.S. central bank forecasts the unemployment rate will be close to 8 percent by year's end, "but that depends very much on how fast the economy grows."
Although the U.S. has seen moderate job growth for the past six months, when Sawyer asked, "What's the one thing that keeps you up at night?" Bernanke pointed to 40 percent of the unemployed who have been out of work for more than six months.
"Those people are obviously facing a lot of hardship," he said.
Bernanke, who was chairman during the financial crisis that began in 2008, described what it felt like to be the only principle policymaker who bridged the administrations of former President George W. Bush and President Barack Obama.
"I was there for the transition, and I was the continuity over that period. And, yes, this has been a very serious crisis followed by a tough recession and recovery," he said. "It's been a long haul certainly."
For 2009, Time Magazine named him the Person of the Year and said he was "the most powerful nerd on the planet."
"I am very proud of my nerd-dom," he said after he was asked if he took offense at the title. "In fact, the world needs more nerds. Nerds, you know, create more jobs and advance science and I hope make good economic policy but that remains to be seen."
With his central role in the U.S. economy, Bernanke has been a target of criticism for the Fed's monetary policies, most recently on the campaign trail. In September, GOP contender Newt Gingrich said he would fire Bernanke if he became president, calling him "the most inflationary, dangerous and power-centered chairman of the Fed in the history of the Fed."
Bernanke said the job of the Federal Reserve "is to do the right thing for the economy irrespective of politics."
"We're not paying any attention to election calendars or political debates," he said. "We're looking at the economy. We want to make the right decision. We want to do it without political pressure, and that's what we're going to do."
Investors have been deconstructing Bernanke's every word, trying to get a sense of whether the Federal Reserve will engage in additional monetary policy stimulus. The central bank's "Operation Twist," which aims to bring interest rates lower through a swap of long-term debt for short-term debt, expires in June. The Federal Reserve previously implemented two rounds of quantitative easing, or bond buying to push long-term interest rates lower, the second of which was introduced in November 2010.
"Well, we don't take any options off the table," he said, when asked if the Federal Reserve could initiate additional economic stimulus. "We don't know what's going to happen in the future, and we have to be prepared to respond to however the economy evolves."
This month Bernanke began delivering a four-part lecture series at the George Washington University School of Business about the Federal Reserve and the financial crisis that emerged in 2008.
"I was a student of the Great Depression as an academic and I think some of the lessons - the mistakes - that were made during the Great Depression are very helpful in thinking about our response to the recent crisis," he said. "And I think we avoided some of the most important mistakes."
"Have we hit rock bottom on housing?" Sawyer asked.
Bernanke said the housing market, with home prices near decade lows, remains "a big concern," though there have been a "few signs of progress" including extra building permits and more construction in multi-family housing."So there's a bit of a green shoot there if you will," he said. "But you know, we're not really yet in a full-fledged housing recovery. And you know, that will be part of the full recovery of the economy."