Victor Gonzalez and eleven other leaders of Imperial Valley Ministries (IVM) were charged on Tuesday with subjecting dozens of homeless people to forced labour.
They also face charges of conspiracy, document servitude and benefits fraud.
IVM also operates five group homes in southern California.
“The indictment alleges an appalling abuse of power by church officials who preyed on vulnerable homeless people with promises of a warm bed and meals,” said Robert Brewer, US attorney for the southern district of California.
“These victims were held captive, stripped of their humble financial means, their identification, their freedom and their dignity.”
IVM allegedly induced people to enter the homes with promises of free food and shelter.
Participants were assured they would be given resources to eventually return home.
Officials suspect IVM leaders of forcing homeless people to beg for nine hours a day, six days a week.
The leaders would later take the money gathered.
Pastors also allegedly forced residents to surrender their belongings and documentation and locked them in their rooms.
They also stole residents’ welfare benefits and food stamps, prosecutors said.
In some cases people were told that if they left, the ministry would take away their children.
“Human trafficking robs victims of their most basic human rights,” said Scott Brunner, an FBI Special Agent.
“Victims of human trafficking are often unseen by society, left pleading in silence.
A list of IVM rules includes a ban on leaving the house alone, a ban on communication with family members and a ban on making phone calls.
“You are not to discuss things of the world,” the list states.
“If receiving any income, 10 per cent will go to tithes, 30 per cent to the home...and the rest will be put away for you by the director.”
Residents were told to have five minute showers and “at no time” to lock the bathroom doors.
They were limited to two meals a day and banned from keeping food in their rooms.
“None of the above rules apply to the pastor or his family,” the list adds.
A diabetic resident was refused medicine and food for low blood sugar, prosecutors said.
The indictment alleges that a second woman was refused treatment for a prolapsed uterus.
Prosecutors said that windows at some of the homes were nailed shut, leading one 17-year-old to break a window to escape.
The teenager ran to a neighbouring property and called the police.
“This is the most significant labour trafficking prosecution in this district in many years,” Mr Brewer said.
“These cases are few and far between because many victims live in captivity and fear, powerless to report the crimes against them.”
Imperial Valley Ministries has declined to comment on the indictment.
Additional reporting by agencies