
It is absolutely OK. Do what you need to do to keep yourself happy and safe. It might be easier said than done, and I say this as someone who has a very difficult time saying no to people for fear of letting them down and guilt and all that, but... honestly, if you aren't comfortable then whatever situation you have has a good chance of getting far, far worse.