Authorities set up concrete barriers and shut down streets and bridges to try to contain and separate the rival groups.
Flag-waving members of the Proud Boys and Three Percenters militia group began gathering late in the morning, some wearing body armour and helmets. Meanwhile anti-fascist protesters known as Antifa were among the several hundred people on the streets.
A viral video of conservative blogger Andy Ngo being attacked at a June 29 rally led to a group called the Proud Boys to organise Saturday's event.
Police asked residents not to call 911 unless it's a life-threatening emergency and to stay away from the heart of the downtown area.
Donald Trump escalated the tensions on Saturday morning but tweeting that the city is being “watched very closely” and that federal authorities are considering designating Antifa a terrorist organisation.
But Portland mayor Ted Wheeler hit back, saying: "This is a potentially dangerous and volatile situation, and adding to that noise doesn't do anything to support or help the efforts that are going on here in Portland."
Portland Police Bureau spokeswoman Tina Jones said officers were focused solely on maintaining order, and not about favouring left-wing counter-protesters, as some right-wing figures have suggested.
"We receive criticism no matter what. It's either too much, not enough, you arrested this group and not this group," Jones said. "It seems like some people are keeping score, and that's not what it's about on the law enforcement side. One hundred percent, it's about public safety."
In addition to the Proud Boys, the white nationalist American Guard and the Three Percenters, a far-right militia, have said they will have members in Portland. Hate group watchdogs say the Daily Stormers, a neo-Nazi group, are also expected.
The Oath Keepers, another far-right militia group, said in a statement they were pulling out of the rally because organisers have not done enough to keep white supremacist groups away.
Agencies contributed to this report