He was friends with Prince Andrew, hosted Bill Clinton at a private Caribbean island called Little St James, and flew Kevin Spacey and Chris Tucker to Africa as part of a humanitarian project. He was an associate, too, of Donald Trump, and a regular for a time at his Mar-a-Lago club.
“I’ve known Jeff for fifteen years. Terrific guy,” the now president, told New York magazine in 2002. “He's a lot of fun to be with. It is even said that he likes beautiful women as much as I do, and many of them are on the younger side. No doubt about it - Jeffrey enjoys his social life.”
Most of those high profile figures dropped contact with the man who first appeared on the New York social scene in the 1990s, as he became the subject of criminal proceedings that resulted in a 2008 plea deal in which he pleaded guilty to a single charge of soliciting prostitution with an underage girl.
But as prosecutors in New York revealed new charges against the 66-year-old, a one time options trader at Bear Stearns who made regular donations to Harvard University, some of those earlier associations may come back with a sting.
The new charges against Epstein - he was charged with one count of sex trafficking and one count of sex trafficking conspiracy - relate to alleged offences said to have taken place at his 45,000-sq-foot mansion on East 71st Street in New York City, between between 2002 to 2005. Prosecutors said it was important that the young woman and girls Epstein is said to have have abused, saw justice be done.
“The alleged behaviour shocks the conscience, and while the charged conduct is from a number of years ago, it is still profoundly important to the many alleged victims, now young women,” said Geoffrey Berman, US attorney for the Southern District of New York. “They deserve their day in court.”
The new charges against Epstein, who was born in the New York borough of Brooklyn, where his father worked for the city’s parks department, appear to have at least partly been the result of a series of investigative reports by the Miami Herald.
Last year, the newspaper revealed prosecutors in Florida had originally drawn up a 53-page charge sheet against him, but that it was dropped at the last minute when a deal was reached between prosecutors in Miami and Epstein’s lawyers.
“We were assisted with some excellent investigative journalism,” Mr Berman told a press conference in New York on Monday.
The deal was overseen by Alexander Acosta, the former United States attorney in the Southern District of Florida, who is now the US labour secretary in Mr Trump’s cabinet. It provided immunity to Epstein from federal prosecution, something that was kept secret from the women who had levelled accusations at him.
Earlier this spring, a federal judge declared that prosecutors had broken the law in the way they negotiated the plea deal and the department of justice opened its own investigation.
Democrats on Capitol Hill have already quizzed Mr Acosta about his role, during a sub-committee hearing.
“I understand the frustration,“ Mr Acosta said, according to Reuters. “It's important to understand that he was going to get off with no jail time or restitution. It was the work of our office that resulted in him going to jail.”
Epstein, a former hedge fund manager who had once taught maths teacher at the exclusive Dalton Schoo on New York’s Upper East Side, was accused of arranging for girls to perform nude “massages” and other sex acts, and paying some girls to recruit others.
He was arrested on Saturday evening at Teterboro Airport in New Jersey, where he had returned on his private plane from France. His lawyers have said he plans to plead not guilty when he appears in court.
The New York accusations against Epstein, which carry a penalty up of up to 45 years in jail, are the latest twist in a journey that has seen the billionaire’s profile dramatically shift from that of being the casually dressed friend of celebrities, to a convicted sex offender, now facing new allegations.
Four years ago, fresh accusations against Epstein arose when two women filed lawsuits against him, claiming Ghislaine Maxwell, the socialite daughter of newspaper tycoon Robert Maxwell, introduced Prince Andrew to Epstein and acted “a madame” to him.
The 57-year Ms Maxwell had always denied the claims, and Prince Andrew, who is known to have gone on vacations with Epstein up until around 2010, also felt obliged to issue a statement “categorically denying” having sex with an underage girl who was a “slave” of friend.
In the spring of 2015, speaking at Davos, Switzerland, the prince said: “I just wish to reiterate and reaffirm the statements that have been made on my behalf. My focus is on my work.”
Prosecutors have asked that Epstein he held in custody until his trial.
“He has enormous wealth,” said Mr Berman. “The charges are very serious and carry with them a maximum sentence of 45 years, which to someone of Epstein's age is basically a life sentence. So we think he has every incentive to try and flee the jurisdiction.”