3.3 & 3.4 Housing Updates - Printable Version

3.3 & 3.4 Housing Updates - Printable Version

+- Hydaelyn Role-Players (https://ffxiv-roleplayers.com/mybb18)
+-- Forum: Final Fantasy 14 (https://ffxiv-roleplayers.com/mybb18/forumdisplay.php?fid=41)
+--- Forum: FFXIV News (https://ffxiv-roleplayers.com/mybb18/forumdisplay.php?fid=9)
+--- Thread: 3.3 & 3.4 Housing Updates (/showthread.php?tid=16114)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

RE: 3.3 Housing Updates - PhantasticPanda - 05-15-2016

Jeez whizz.. I wonder if their employees get paid for all the work they do for keeping the servers stable?

People talk about storage size of SSD's or HDD's, but the main problem is bandwidth or not even that, more like how much data is being built then transferred from one machine to the next. In each Housing zone, you have a lot of people growing crops, personally-owned retainers that people use to sell and buy, chocobo training, and maybe some other things I may have missed. There's a constant stream of data being transferred that not just housing or furniture items, but things that are read, configured and transferred 24/7. I hope someone has a source for this but I think I read somewhere that more Wards would be possibly if they got rid of the gardening and chocobo training system out from the Housing zones.

Maybe it worked for LOTRO since they didn't have all of these minor features taking up more data than housing would by itself. Maybe it worked for Wildstar because every house is "housed" in its own instance, thats not loaded unless someone is there, rather than connected to a bunch of other neighboring homes. Its difficult to compare housing systems from each game when they all have a different infrastructure or foundation.

Back to a more related note, I can't wait to press the log in button for two hours straight while maintenance is happening.

RE: 3.3 Housing Updates - McBeefâ„¢ - 05-15-2016

(05-15-2016, 05:22 PM)Setoh Aliapoh Wrote:
(05-15-2016, 04:59 PM)McBeefÂ© Wrote:
(05-15-2016, 04:55 PM)Setoh Aliapoh Wrote:
(05-15-2016, 02:32 PM)O Wrote: The simplest solution would be to just set the server to automatically spawn a new empty ward if all houses of any one size were already bought in all other wards. Low-pop servers would end up with a small number of wards, high pop servers would have a high number of wards.Â

I don't buy the excuse that this can't be done due to PS3, because a PS3 user is still only loading a given ward that they happen to personally be inside of at that time, so having a larger list of wards which exist shouldn't impact anything.

This solution would also gut the flipping market, because unless a person is really dead-set on getting a specific plot, there would always be at least one small, one medium, and one large available at all times. This solution also accommodates multiple houses per account, and even accommodates neighborhooding if players are so inclined (a cluster of friends could wait until a new ward spawns, and then all buy houses that are next to each other).Â

I also don't buy the excuse of it costing too much in server infrastructure/database/etc, because the higher pop servers are where all the income is coming from anyway, so... why are we per-capita getting less services than a smaller server?

This, exactly! LOTRO had a similar system, where they'd spawn new wards as the old ones filled up.

There are those who defend the current housing system by arguing that the housing servers can't support more wards (which is, honestly, pretty ridiculous in this day and age). Assuming this is 100% true, you could still get nearly the same benefit by simply pooling *all* the wards in a cross-server pool, and then allocating them to servers as they fill up. Low population servers would end up with a few wards, while high population servers would have quite a bit more. It still wouldn't solve the problem entirely, but it would help mitigate it.

Housing in an MMO isn't rocket science, though. It's a solved problem, in the sense that other games have been able to enable letting anyone who can afford it buy housing.
Honestly, I'm guessing it's a PS3 issue.

Something like some table somewhere can only hold so much data, and they can't rebuild it. It's why they have weird workarounds like adding sub-divisions instead of just increasing the number of wards.Â

I get the feeling they have to like reverse engineer it every time they want to increase the number.

That makes no sense. There are plenty of PS3 games that provide access to all sorts of information. I worked on the PS3 version of HBO GO, and there was no problem at all with the console being able to handle large amounts of data. There was no problem at all with the ability to visualize that data, to scroll through it on the screen, to page around it, etc.

I think we're coming up with excuses for them, but realistically housing is in bad shape because SE don't care enough to make it better, and because we're willing (as a whole) to live with it the way it is. Like I said before, housing in MMOs is a solved problem.

I did a little math, in case you like math:

Show Content

CS/Math follows
I've gotten the "It's too much for their servers" objection whenever I talk about housing before, so let's do some math on this.

Every housing ward is the same. This means that the overall map to the place is something the client can store. The only thing the server needs to handle is the specifics of what's unique about each house. What you really end up talking about then is bandwidth and storage costs. So let's look at those. Really, since we already know their network can handle transmitting the information, we're only looking at storage costs. So let's look at how much storage we're talking about.

A house has an owner, some permissions, and the like. We'll stretch it and say it takes 1k to store all that. 2k when you include the message on the placard.

A house also has some 200-odd items. Each item has an X and Y coordinate, as well as a rotation value. It has an id (because the individual asset is already in the client, so it only has to say "This is an X"), and a dye color. Say it's 20 bytes for all this. Let's actually bump that up to 50, to include all the extra stuff you might want to have on items, to cover for things like garden crops, etc.

A house also can have rooms inside, if it's an FC house. This gets pretty variable. Lots of FC houses have only a few rooms. Some have many, many rooms. The max is 512, but I'd guess the average is somewhere around 10 (including all the personal houses, which have none). So the average per house might be 10, but the max per house - assuming every house is an FC house - is 512. A FC room can have 50 items in it, plus we'll give it 1k for the message, permissions, etc.

So a FC room ends up having a max storage cost of 3.5k. A house, with 200 items and 10 rooms, has a storage cost of 47k. Assuming it's filled to the brim with 512 FC rooms, then it's about 1.8 MB.

A ward of 60 houses, then, has a storage cost of just over 2.8 MB. For our worst-case ward, where every house is a FC house, every FC house has 512 rooms, and everything is filled to capacity, it's 108 MB.

They're adding wards in 3.3, so each server gets 3 housing areas with 12 wards each. 36 wards. Our math above, then, says each server that fills up all its wards will have about 100 MB of housing data. The worst-case, where every house on the server is a FC house, every FC house has 512 rooms, and everything is filled to capacity is 3.9 GB.

I think there are currently 62 servers for FF XIV. Assuming every server fills all its wards, we're talking about 2,232 wards. If every server fills all it's wards, with the average sizes we calculated, then SE needs to store 6.2 GB of housing data. If we keep extending our fun worst-case scenario, this goes up to a maximum of 242 GB.

Let's pad that out a bit, because it's hard to account for everything. All this data will be stored in some sort of database, so there's some overhead there, too. So. Let's double it. Our best-guess calculation then becomes about 12.5 GB of data for all the housing data, to a max of about 484 GB. By this point, it's pretty clear that the worst-case number is ridiculous, so let's look at the best-guess number, instead.

12.5 GB.

It's easy to say "Hey, you can buy an SSD that stores that much on Amazon for $5", but that's naive. What you really want is something scalable. Something reliable. Something like a Thecus N8810U-G 8-Bay Rackmount NAS Server loaded with 8 SSD drives. That's about $1500, plus another $1200 or so for 8 500GB SSD drives. With a RAID 10 configuration, you get extreme reliability and 1 TB of storage out of there for $2700.

With our best-guess data, that would store all the existing wards, plus enough for every server to have 2,880 more wards. With our worst-case data (which is getting more and more ludicrous as we keep piling on those assumptions), it'll still provide double the wards across the whole game. But really, we're talking about our best-guess data.

2,880 more wards. Less than a ward per dollar.

Considering SE gets at least $156 per year per dedicated player, that seems pretty damn reasonable to me.

I really think that they've made some sort of mistake on the underlying architecture when they made the wards, that makes it difficult to change after the fact.

I think the subdivision thing is a good example of this. Why on earth would they make subdivisions (that are annoying to get to, and have no benefit over just more wards). I think they made subdivisions because they couldn't just make more wards for some reason. I think they've been working on a fix since then, and finally have something (hence more wards).Â

Clearly they just didn't expect the desire for housing (else they would have done it differently) and I think whatever solution they originally put in place just doesn't scale up well. I mean, if it's not an architecture issue, then why wouldn't they just add more wards? Because they're lazy? Because they don't feel like it?Â

I think your example assumes everything done in an optimal way, which if you look at SE's history with this sort of stuff, is rather unlikely. Things like this can been seen all over the housing, from the subdivisions, to the orchestron's not playing high quality versions of songs (which is apparently also due to performance issues).Â

I agree it's silly they can't just add more wards, but I think it's a little harsh to hold them up to perfect standards. I do believe it's due to performance issues, and I do think it's just something they need to figure out on their end. Whether or not those performance issues are due to incompetence is another question.

RE: 3.3 Housing Updates - McBeefâ„¢ - 05-15-2016

Also for the record, something like a 50k of data for a room seems like a low number, until you consider that it has to be accessed several times a second.Â

Then you zoom out and you have 1000 groups of people all accessing their own 50k of data several times per second.Â I don't think it's as trivial of an engineering problem as you're suggesting.

There is a reason that when the server is overcrowded, the first thing that happens is that you're unable to enter your FC house. The database can only service so many requests (small as they are) and it cuts it off when it hits its limit. Again, you can argue that they've designed it poorly, but it's not as much about storing the data, it's about accessing it. When you change an item in a room, it's immediately updated for everyone in it, it's not just a a static database.

I think you're being a little harsh on them. LOTRO's housing was pretty basic by comparison.

(Also 50 bytes for an item seems far too low, considering it also has to store things like whether or not it is bound to a player or an FC, and what that player or FC's name is (which can be up to 20 bytes long, which probably has some sort of key, but its still gonna be more data) I'm sure we're forgetting about other minor stuff as well.)

(Ah and linked items, an object has to know about what objects are on top of it. For example if you put flowers on a table, and you move the table, the flowers move too. So then you get into object references, andddd in any case I commiserate with the SE team)

RE: 3.3 Housing Updates - C'kayah Polaali - 05-16-2016

(05-15-2016, 08:35 PM)McBeefÂ© Wrote: Also for the record, something like a 50k of data for a room seems like a low number, until you consider that it has to be accessed several times a second.Â

Then you zoom out and you have 1000 groups of people all accessing their own 50k of data several times per second.Â I don't think it's as trivial of an engineering problem as you're suggesting.

I was estimating 50k for a *house*, not a room.

In any case, this data wouldn't have to be sent several times a second. It can be sent once per person in the room house, and then the players local client can take care of rendering everything. Remember: that data doesn't change unless someone is actively redecorating the house.

Honestly, bandwidth isn't the issue at all. FF XIV does fine for raids with highly complex environments. It does fine with populated areas filled with people who all have their own gear sets (with dyes and glamours and the like). The problem is pretty much solely storage space and technical debt. I think you're right that they made some bad architectural decisions, and it's limiting things right now. Personally? I don't care. It's not my problem. I'm a paying customer and I want better housing.

EverQuest 2 had *far* more complex housing back in 2004. This is *not* rocket science.

RE: 3.3 Housing Updates - Unnamed Mercenary - 05-16-2016

(05-16-2016, 03:57 PM)Setoh Aliapoh Wrote:
(05-15-2016, 08:35 PM)McBeefÂ© Wrote: Also for the record, something like a 50k of data for a room seems like a low number, until you consider that it has to be accessed several times a second.Â

Then you zoom out and you have 1000 groups of people all accessing their own 50k of data several times per second.Â I don't think it's as trivial of an engineering problem as you're suggesting.

I was estimating 50k for a *house*, not a room.

In any case, this data wouldn't have to be sent several times a second. It can be sent once per person in the room house, and then the players local client can take care of rendering everything. Remember: that data doesn't change unless someone is actively redecorating the house.

Honestly, bandwidth isn't the issue at all. FF XIV does fine for raids with highly complex environments. It does fine with populated areas filled with people who all have their own gear sets (with dyes and glamours and the like). The problem is pretty much solely storage space and technical debt. I think you're right that they made some bad architectural decisions, and it's limiting things right now. Personally? I don't care. It's not my problem. I'm a paying customer and I want better housing.

EverQuest 2 had *far* more complex housing back in 2004. This is *not* rocket science.

You know player position is polled every 100-300 milliseconds, right? That has to be sent to the server and broadcasted to the entire zone. For every player. When there's a server issue, it's usually bandwidth-related or something done to prevent a bandwidth issue. The whole reason instances are well, fast and instanced is for that exact purpose.

Player housing was intended to be a gil sink in this game. It also is likely limited by SE's trend to not do big long scrolling lists for everything, which is how the buggy 1.0 client worked, and from what I've seen, how FFXI worked. If it can't be displayed on a flat window, it's probably going against their user interface specifications, hence why we get subdivisions and not just a bigger list of normal ones.

Should they stop treating all servers the same in regards to the amount of houses? Yeah, probably. But they probably also use the same database template across them all and there's no system in place for dynamic allocation. Destroying old houses that aren't in use could have been a step in that direction though. The development team is pretty small and that's not going to change no matter how much people ask or whine or beg. If resources are put to housing, they'll be taken from other areas which keep the game running. That's just how software engineering works in a product like this.

If you need sources on the packet system, there's loads of research done on 1.0, courtesy of the Seventh Umbral and FFXIV Classic server development projects. While the architecture has surely changed, the database likely didn't as many of the 1.0 tables were directly copied over into 2.0. They probably still use a similar packet broadcasting mechanism.

Source on shitty resource management: the Comcast-owned company I work for, which goes through the same bullshit.

RE: 3.3 Housing Updates - C'kayah Polaali - 05-16-2016

(05-16-2016, 04:58 PM)Unnamed Mercenary Wrote: You know player position is polled every 100-300 milliseconds, right? That has to be sent to the server and broadcasted to the entire zone. For every player. When there's a server issue, it's usually bandwidth-related or something done to prevent a bandwidth issue. The whole reason instances are well, fast and instanced is for that exact purpose.

Why should the player position code, which has to deal with mobile players moving around an area, have anything to do with the code that sends out information on the immobile things in an area?

It doesn't really matter, I suppose. The game is what it is. Though I have to admit, I would love it if a limitation like this addressed crafting or the endgame. "Unfortunately your HQ synthesis failed because the server cannot contain any more HQ Thavnairian bustiers..."

RE: 3.3 Housing Updates - McBeefâ„¢ - 05-16-2016

(05-16-2016, 05:41 PM)Setoh Aliapoh Wrote:
(05-16-2016, 04:58 PM)Unnamed Mercenary Wrote: You know player position is polled every 100-300 milliseconds, right? That has to be sent to the server and broadcasted to the entire zone. For every player. When there's a server issue, it's usually bandwidth-related or something done to prevent a bandwidth issue. The whole reason instances are well, fast and instanced is for that exact purpose.

Why should the player position code, which has to deal with mobile players moving around an area, have anything to do with the code that sends out information on the immobile things in an area?

Because they're not immobile.

Any player can move them at any time, and collision boxes need to be updated immediately for all players.

RE: 3.3 Housing Updates - C'kayah Polaali - 05-16-2016

(05-16-2016, 05:55 PM)McBeefÂ© Wrote:
(05-16-2016, 05:41 PM)Setoh Aliapoh Wrote:
(05-16-2016, 04:58 PM)Unnamed Mercenary Wrote: You know player position is polled every 100-300 milliseconds, right? That has to be sent to the server and broadcasted to the entire zone. For every player. When there's a server issue, it's usually bandwidth-related or something done to prevent a bandwidth issue. The whole reason instances are well, fast and instanced is for that exact purpose.

Why should the player position code, which has to deal with mobile players moving around an area, have anything to do with the code that sends out information on the immobile things in an area?
Because they're not immobile.

Any player can move them at any time, and collision boxes need to be updated immediately for all players.

*eyeroll*

Transmitting only a delta of changed data is computer science 101.

I am really amused at the SE apologists, though. I mean, I'm sure they're doing the best they can. Gold star for the housing. Don't change a thing.

RE: 3.3 Housing Updates - McBeefâ„¢ - 05-16-2016

(05-16-2016, 06:29 PM)Setoh Aliapoh Wrote:
(05-16-2016, 05:55 PM)McBeefÂ© Wrote:
(05-16-2016, 05:41 PM)Setoh Aliapoh Wrote:
(05-16-2016, 04:58 PM)Unnamed Mercenary Wrote: You know player position is polled every 100-300 milliseconds, right? That has to be sent to the server and broadcasted to the entire zone. For every player. When there's a server issue, it's usually bandwidth-related or something done to prevent a bandwidth issue. The whole reason instances are well, fast and instanced is for that exact purpose.

Why should the player position code, which has to deal with mobile players moving around an area, have anything to do with the code that sends out information on the immobile things in an area?
Because they're not immobile.

Any player can move them at any time, and collision boxes need to be updated immediately for all players.

*eyeroll*

Transmitting only a delta of changed data is computer science 101.

I am really amused at the SE apologists, though. I mean, I'm sure they're doing the best they can. Gold star for the housing. Don't change a thing.

Coming off a little spicy there, friend.Â

I am sorry their implementation isn't to your liking.

RE: 3.3 Housing Updates - C'kayah Polaali - 05-16-2016

(05-16-2016, 06:37 PM)McBeefÂ© Wrote:
(05-16-2016, 06:29 PM)Setoh Aliapoh Wrote:
(05-16-2016, 05:55 PM)McBeefÂ© Wrote:
(05-16-2016, 05:41 PM)Setoh Aliapoh Wrote:
(05-16-2016, 04:58 PM)Unnamed Mercenary Wrote: You know player position is polled every 100-300 milliseconds, right? That has to be sent to the server and broadcasted to the entire zone. For every player. When there's a server issue, it's usually bandwidth-related or something done to prevent a bandwidth issue. The whole reason instances are well, fast and instanced is for that exact purpose.

Why should the player position code, which has to deal with mobile players moving around an area, have anything to do with the code that sends out information on the immobile things in an area?
Because they're not immobile.

Any player can move them at any time, and collision boxes need to be updated immediately for all players.

*eyeroll*

Transmitting only a delta of changed data is computer science 101.

I am really amused at the SE apologists, though. I mean, I'm sure they're doing the best they can. Gold star for the housing. Don't change a thing.
Coming off a little spicy there, friend.Â

I am sorry their implementation isn't to your liking.

That's not exactly what I'm referring to. Wink

RE: 3.3 Housing Updates - K'nahli - 05-16-2016

I don't think anyone is disagreeing that the housing system leaves a lot to be desired, but that doesn't stop them from reasoning as to why it is proving to be such a problem for the development team. Issues with what would otherwise appear to be technologically simple feats appear all throughout the game; it can often be simply impossible to hit an enemy that's moving while playing as a melee class - no matter how much you're sitting on it's tail.

I don't think anyone has come close to saying that "the important thing is that they tried" or that they shouldn't try to find a way to seriously revamp their system so that minor systems are as easy to work with as they should be for a game in this day and age. The impression I am getting, is that people aren't satisfied with an answer as conveniently pessimistic as 'SE doesn't care' or anything along those lines... and frankly I'm not either. There are a number of things I am dissatisfied with, the biggest one having absolutely nothing to do with technical restrictions but blunt hard-headedness on Yoshi's part, but it's clear that there are multiple issues that prevent the ease of implementation we would all hope for in the case of housing. It's borderline absurd to think otherwise.

RE: 3.3 Housing Updates - Kellach Woods - 05-16-2016

The shitstorm on Balmung will be pretty damn good.

That's really all I know about housing.

In so far as lol SE is concerned, if Housing was the biggest problem we'd be pretty set, really.

RE: 3.3 Housing Updates - Warren Castille - 05-17-2016

ITT: People who didn't play XI or 1.0, apparently.

SE can't make anything efficient. Player inventory storage was 30 items. Total. Equipment still consumed inventory. It could be expanded to 50 items, and then Square said "nope, memory limitations and PS2 limitations." Then they expanded to 60 items and went "Nope, can't do more, ps2 limits." Then they expanded to 70. Then 80. Then added a perk for security token users to get a mog satchel that gave you another 80.

I don't think they were lying. I think they just had years of shitty spaghetti code that made them not able to increase personal inventory because of bad programming that didn't make for a good MMO scenario. In a stand-alone Playstation game, you don't have to worry about years of growth, since your game is done at launch.

They did go on at length about how housing is complicated: The wards themselves also have to process everyone's chocobo and garden data to every person, and inside of the house I was under the impression the game is constantly checking to see if something gets moved/interacted with/whatever. I believe them, because I've seen them handle it all very poorly before. Twice now, if we're counting games and not specific incidents.

It's not the Square Defense Force insisting everything is fine. It's people pointing out that this is SE doing what SE does: Shitting things up and then needing to work extra hard to find a fix to a problem they should have anticipated.

RE: 3.3 Housing Updates - Lydia Lightfoot - 05-17-2016

I wonder why they don't just... I dunno. I mean, here's an idea:

Know how we access the Workshop? Kay, what if we also had a Barn and a Greenhouse which we access the exact same way?

Inside the Barn, we could have a storage chest in which only items specifically relevant to Chocobos and the current "stable" stuff could be stored (krakka roots, color change items, cleaning brooms, etc) - and users who have permission to do so can direct-use those items without needing to withdraw them (so if there's krakka roots in the barn chest, you can feed the birds from the chest without needing to first withdraw them). You'd be able to visually see a few chocobos, and maybe give those who have "Barn Visuals" permission the ability to specifically select certain chocos to be excluded from the cycled list of the ones that could be seen, or to set certain chocos to always be visible if they're in the stable. Include the ability to have a chocobo racing interface NPC if desired, so that players who want to do chocobo racing could do it from the Barn instead of needing to go to Gold Saucer.

Inside the Greenhouse, a house has the maximum number of planting spots depending on its size (so I think 8 for a small, 16 for medium, and 24 for large) and a storage chest in which only seeds, grown plant items that have since been harvested, fertilizer, and other garden related items. Garden items which are relevant to the Barn could also be swiftly transferred from the garden chest into the barn chest without even needing to re-zone.Â

This makes it so that the "updating data" for these only needs to be refreshed when there is a character who enters the Barn or the Greenhouse, thus rescuing the housing wards from the data drain.

As for the building exterior, we obviously no longer need the Stable or Garden items, so that frees up a couple of yard slots (YAY!). For players who do still want a visual representation, SE could add a couple new yard decor items called Barn and Greenhouse, and do a few different visual designs for them, so people could make them match their building. The only rule with it would be that just like the "awning" items, it would have to be placed right up against the exterior of the building. These would have no function, other than visual effect. I'd also say we need a roof decor item that looks representative of an airship dock, which would also serve no purpose other than to let a guild that wants to visually show the outside part of their workshop do so.

Good idea, right?

RE: 3.3 Housing Updates - Warren Castille - 05-17-2016

I tried typing up a reply like five times but I kept banging my head on the fact that I don't actually know anything about programming so I can't come to any conclusions.

* Warren Castille makes sad trombone noises.