Just hanging out and having a good time not hurting anybody,
then there's a confrontation with a stranger, based simply on what you look like and where you come from?
Welcome to how racism works.
This is the authentic experience for racism, with a layer of abstraction in that you are roleplaying / acting, and not experiencing it as a direct attack on you as a person.
Now, roleplaying is a leisure-activity, so no, I don't think people should always express themselves in this sort of confrontational way, even if their characters "would".
But if you have issues with your character being treated badly based on where they're from,
I honestly think you should just stick with your FC / group.
As an example, If someone's playing a Garlean, and others come up to them who have lost things to the Garlean war of conquest and get in their face: This should not be something that causes you major distress, and if it is, you probably shouldn't roleplay as a character with ties to those controversial things that people in the setting would see as friction-points.
Same for Voidsent, Sin Eater, and Auspice RPers:
These sorts of things have very clear connotations along with them.
The common experience with a Voidsent often ends with fighting for your life and death.
The common experience with a Buduga band in the steppes is violence, sometimes resulting in death, or other vile things.
This is clearly not representative of ALL Buduga, but the reputation travels far, and that prejudice is real in the world.
So when somebody has prejudice toward your character upon learning of their tribe or upon seeing their onyx scales and making myriad assumptions on which tribe they're in, know that it comes from a place of belivability:
And when you are playing a character from that background, region, or group,
then you should know that can happen sometimes.