Release 5 Post Mortem

I’m normally long winded in my release wrap ups but I’m going to try and keep this one short for a change. So my overall rating for release 5 would probably be around a B- in terms of what got done BUT I think the things that went well did not have much outward visibility to the players. Unfortunately that B- is really an average of a D+ for player facing elements and an A- for all the behind the scenes stuff that got done that wasn’t very visible to players just yet.

Because of the timing many of the most visible elements the players were in a very unpolished state for this release. In fact, even just two days before we went live the game was in a completely unplayable state. Combat didn’t work at all, AI’s weren’t working, switching maps would crash 100% of the time, other players couldn’t see objects placed on lots, all the scenes were either too bright or so dark you couldn’t see anything, and conversations weren’t working at all. So while the release was really rough for this release, I have to admit that internally we were high fiving each other that it was not MUCH worse.

I’ll briefly cover a few player facing items and then talk about some of the behind the scenes elements that were only barely visible to the players but are super exciting and were very needed additions.

So, first the bad stuff. On Thursday and Friday until the afternoon, the frame rate was atrocious. Several bad things snuck in at the last minute due to our rush to fix things. The worst was that almost all character models, lost their visibility distance data. This means that when you entered Owlshead and looked North, towards the volcano end of town, you were rendering about 70-80 skinned, animated models even though you could only see about 3. UGH… Frame rate tests have been added to our “Smoke” and “Sanity” tests so hopefully in the future this won’t sneak by us. Also, due to another really good performance change in how we update changes to shared spaces, we had a large number of memory allocs and frees going on that resulted in some really nasty hitching. The fix for this was simply to pre-allocate space for it and not free it but we didn’t catch it until after we were live.

So, on the positive side of player facing stuff, a LOT of good changes were made to help us improve the visuals going forward. The largest was switching from gamma to linear lighting model. For a longer post on what this means, you can go here:
https://docs.unity3d.com/Documentation/Manual/LinearLighting.html

This is the model we should have been using all along but had put off switching because it was too much work. Well, this was milestone we decided to bite the bullet and make the switch. The pain in switching comes in the form of hundreds of little stings caused by everything working a bit differently and the light values in the whole game needing to be adjusted. Additionally, a few hundred materials and shaders needed adjustment as well.

So, what do we get for all that effort? We are now able to much better handle the crazy dynamic lighting situations of our very sandbox style game where a scene might need to be super dark and have a single torch light things appropriately OR the same scene might need to be daylight and have 5+ dynamic lights all effecting light colored objects and still hold up. What we were finding before was that no matter how hard artists tried, we just weren’t able to get good results without everything blowing out to pure white in the extremes or be overly dark. We didn’t get all the scenes adjusted in time for release but Owlshead, Braemar, Kingsport, the Sewer, and the Clink all received a lighting pass and it made a huge difference.

So, I seem to be failing on my promise to keep it short so I’ll skip ahead to one of the most exciting things that went in that probably no one noticed, the addition of a scene server. So previously, we only had the concept of a “room” that was shared by those in a single instance of a scene. So if we had 3 instances of Owlshead, there were 3 room instances. This worked great BUT meant that nothing really was shared between the scenes without a lot of work and that every instance of Owlshead essentially became an island.

The new scene server gives us a place to share data common to all instances of the map. So what’s the big whoop about that? Well, the obvious thing is that now, a player in one instance of a map will immediately see when a new house goes up in another scene. They will also see decorations immediately appear once placed. In future releases it can mean that if you pay the town cryer to shout something for you, it will immediately start showing up in the other scenes. Also, vendor lists will immediately update with items sold or bought without more work.

Now for the less obvious stuff that is even more exciting! We can flag specific players to store data to the scene server instead of the instance server so they will exist in all instances of the same scene. This means if Lord British and DarkStarr want to make an appearance in Braemar and 5000 people want to see them live, they will appear in all instances of Braemar! This also means we can do things like track player deaths in a dungeon in the Scene server. Then we have a spell that senses danger that shows you where people have been dying in this dungeon and put up small ghostly tombstones at the spot where players have died. With only a bit more work, we could make boss mobs that exist across multiple instances as well. So instead of having creatures that 40 or 50 people fight at once, it could require 400 or 500 people across 5 or 10 instances!

Another very large change that was likely invisible to end users dealt with how bags and chests work. For Release 5 we made changes to how things work so containers don’t transmit what is in them until they are opened. So containers are things like bags, chests, or corpses. This was a very large change that had huge implications in multiple areas.

First, if you walk through a town and can see 40 houses and each of those houses has 5 chests and each of those chests can hold 30 items, that works out to 6000 items! Previously, we would have to send you all 6000 items. Now multiply that by 50 players in the scene and we’re now sending 300k items. Ugh… So part of this was to reduce how much we have to send players in towns but this was also needed for other, hack related reasons.

First, if we send all those items to every client then someone who had hacked their client could essentially get a list of all your most private items. Also, if a player figured out how to hack their clients, they would be able to walk into any dungeon and see what was in the chests anywhere in the dungeon without having to open the chests first. This lets us put a server side lock on things so we can limit what stuff the bad guys see.

So much for not being too wordy! Ok, on to R6 stuff. We have been considering a polish milestone for a while and I think with how rough R5 was, I think it is clear we need it. In addition to the visible polish we’ll be doing, we will also be going back and trying to improve some internal processes.

I usually sneak in a favorite development saying to my posts and this time I saved it for the end… This month’s appropriate developer saying is: “Sometimes you have to go slow to go fast.” We’ve been doing a bit too focusing only on visible results each release. Spending a month to “Sharpen the saw” is much overdue and going slow for a month to improve processes, work on fixing nagging bugs, and improving visuals now so less has to be redone later really will help us go fast in the long run.

Looking forward to R6 and seeing you guys in chat and in the forums till then!