There's french in the very naming of locations in the Elven kingdom.
Actually more to the point... We don't speak English there either. English is simply the most widely accepted 'translation' of whatever language is being used in Vanadiel because we're (mostly) Americans. In Germany it's a different case and in Japan it's a different case. If somebody uses French words it's no more absurd that pretending, in game, everyone is speaking English.
Well of course, but that's just my example. When you play a game that has established languages, and someone suddenly starts going "OUI OUI BAGGUETE DU FROMAGE?!" don't you want to slap them?
Depends how it's handled. It can be very condescending.