
With that information in hand, I don't roleplay it as all or nothing, I roleplay it as varying degrees.
My mage character teleports like riding a bike, without even breaking a sweat. A perk of being a mage and manipulating aether for years. My more martial character can do it, only once in a while... maybe once a week, but it takes a huge toll, and leaves him barfing all over the place. He hates it. And worse still, my tribal Miqo'te has no idea what that fancy technology is, and simply can't do it.
I'm like you in that I think easy use of Aetherytes cheapens the world, but outright cutting it out of your headcanon is a bit extreme. Try making it much harder in your headcanon first, and see how that works... Like a skill, that must be practiced and most people just don't know how, and can't do it effectively. Make it take a huge toll on your body to the point where its preferable to just take a ferry or ride that chocobo across the beautiful world.

You have options before straight up slicing it right out. I think there's something interesting to Aetherytes, and they really remind me of Mass Relays from Mass Effect.
Talks the talk, and walks the walk.
Serious, lore-abiding, mature roleplayer.
Serious, lore-abiding, mature roleplayer.