I think you're underestimating potential prejudices about that nomadic lifestyle.
What the Domans get to see isn't "ah, they're just like the Raen but with different coloured scales". They see "ah, they're like Raen with different scales, and the bulk of them are unpredictable, potentially violent, and not amenable to trade".
So you get the Xaela being viewed as this group of generally violent people, who can't be negotiated with, and are better avoided especially by non-combatants...
What the Domans get to see isn't "ah, they're just like the Raen but with different coloured scales". They see "ah, they're like Raen with different scales, and the bulk of them are unpredictable, potentially violent, and not amenable to trade".
So you get the Xaela being viewed as this group of generally violent people, who can't be negotiated with, and are better avoided especially by non-combatants...