Also, alongside everything that has been mentioned about the Allagan Empire, it was mentioned that house Darnus and it's bloodline was pretty much the keepers of Allagan lore and knowledge on Hydaelyn. I think SE even went so far as to say that house Darnus is the only confirmed bloodline to still exist that has roots with the Allagans.
Which leads me to believe the Allagans may have been Elezen, seeing as we can assume Darnus was elezen based on his body structure. It was mentioned that Elezen were the original inhabitants of Eorzea (and they were not happy with the Miqo'te and Hyur creeping up on their territory~) if my mind serves me correctly, and it seems there are lots of Allagan Ruins in Eorzea. Pure speculation, however.
Which leads me to believe the Allagans may have been Elezen, seeing as we can assume Darnus was elezen based on his body structure. It was mentioned that Elezen were the original inhabitants of Eorzea (and they were not happy with the Miqo'te and Hyur creeping up on their territory~) if my mind serves me correctly, and it seems there are lots of Allagan Ruins in Eorzea. Pure speculation, however.