Demonstrators clashed at Nathan Phillips Square Saturday afternoon.
One group protesting a bill they say goes against free speech, the other accusing them of racism.
Though there was a lot of yelling, things never really got out of hand. Police were on hand with the mounted unit.
One protest was titled the March for Freedom against M103, a motion tabled by Mississauga Liberal backbencher Iqra Khalid.
It would give representatives the right to condemn Islamophobia and track incidents of hate crimes against Muslims.
The group said Cananda already has hate speech laws and we don't need a seperate law for just one religion.
A large group showed up to protest the marchers messages.
A woman in that group told CP24 "I think that we need to draw a very serious distinction between the ability to critize and the ability to send messages of hate."
Tempers did flare at some points and police stepped in, but no major incidents were reported.
Toronto police say three people were arrested: two charged with obstructing a police officer, one with assault.
With files from CP24 & CTV