[Fixed 68373] low FPS and hitching / GUI causing huge lag

This FPS/Lag issue started when barding and extended party sizes came on board - make sure you join an ERG party as a player in the party itself, and have 10+ people with someone spamming bard songs.

 

FYI, @Chris confirmed this on Twitter:

 

I noticed as well GUI intensive actions, as well as the chat, are major lag sources.

I couldn't get any clean, replicatable tests so I wasn't sure. I'll try this one later.

As of making a new GUI, if you want to write one quickly from the ground up, using a tile-engine layer, or some other Unity 2D game animation layer for the HUD, should make a nice clean, hyper-responsive UI with far less lag.

 

Are these actually GUI lag, or are they caused by writing to the local log file? There is a command line option to disable the log file (-nolog), which could help determine where the lag is. If its log file related, the new verbose combat logging is going to cause all kinds of issues.

 

I've been noticing GUI interactions triggering Full GC cycles resulting in the lag timer triggering in the Unity debug log. I don't know if that displays in the one displayed on /debug. Populating lists and something in the chat widget seems to do this.

Using a capped scrolling buffer for chat should fix the issue of chat continually allocating more memory.

For lists you scan use serialized arrays of arrays with null padding, or populate them in blocks with null padding.

Additionally keeping items in a separate sparse storage cache of the remote database, and only keeping UUIDs/GUIDs for the items in the actual inventory lists will prevent creation and destruction of nearly as much data.

As of the UI itself, using the Unity 2D game engine to render the UI instead of the clunky Unity UI tools could make a much faster and more efficient UI.

 

The FPS jump was huge, in a party of 12, my fps went from 24 to 58-62 with gui turned off, it immediately jumps up or down with gui being toggled on or off.

 

I know that the tooltips above gui, especially the hotbar were causing some performance issues and small but extremely annoying lag when casting.

As for the scenes it seems to me that most of the time my performance is fine and then for no reason it drops but restarting the game fixes it. Have not tested turning off GUI though I expect I have different issue.

 

As a note, I can't stress enough how much using static objects if at all possible, ESPECIALLY with very long strings like the chat window, can improve performance in Unity.

The chat is a MAJOR source of lag for extended play sessions because it doesn't clear and has a continually growing buffer. Do to the very poor string handling in C# which is even upstream from Unity in the Microsoft created standard for the language, every time you extend a stream, you basically copy it and then delete the old string. Now imagine doing that with a 2MB chat buffer and you will see clearly where a big chunk of lag comes from.

String length extension routines create a new variable and destroy the old one every time you run them. This is a BAD upstream flaw in Unity. While consolidating operations can reduce it, the only 100% effective workaround is to avoid the operation altogether.

This can be done in tooltips by using a fixed object for the tooltip and simply hiding, changing and redisplaying it.
This can be done in the chat window by using a fixed length buffer for the chat text.

As of other object creation-destruction related to item data. If you are still storing item data attached to inventory items, move it to a separate object-base attached to GUID poitners, and use an array of GUIDs in the container lists. For containers with a size cap use a fixed list for this. Using a GUID list in this manner also allows you to centralize the storage of all items, making deduplication of data easier, hence reducing database size and memory burden of storing database cache client side.

Yes, and as I said, you can blame Microsoft for so many object modifications copying and deleting the object. Not Catnip/Portalium, Not Unity, Not even Mono. It was Microsoft. They literally wrote the bug into stone when they made the C# language, and barring a complete revision of the C# language it can only be mitigated not fixed. Unity's refusal to apply industry standard mitigations such as better GC algorithms on their end has not helped, but the blame squarely lies on those who wrote the bug into the language standard.

 

Here's a nice article on it although the writer claims it's not a problem:
https://blog.maartenballiauw.be/post/2016/11/15/exploring-memory-allocation-and-strings.html

I mean it's all a big facepalm for anyone who knows anything about low level computing it's like WTF were you thinking Microsoft, yet most are afraid to point this out and instead saying things like, ah the new 2nd generation GCs are fast enough to handle it.
Except two problems:

1. It shouldn't be necessary in the first place. Java does NOT have this bug, nor does Python, nor Perl, nor most Javascript Interpreters (except Microsoft Edge which uses .Net JScript)
2. Unity 3D is still using a first generation GC that can't handle the load, and they are stuck on it due to code dependency that is putting them in a dependency nightmare any time they try to switch.

Simply put, you can't keep reallocating these lists of objects and massive chat buffers by concatinating them or appending them without massive lag being created.

Strings are great for things that are created once per a scene. However, unlike other languages, a pointer to a string database entry or an array of char is more efficient for more dynamic usages.

On the last release I worked as a tester on KSP, they switched all the strings to pointers to a database of strings. This not only permitted internationalization, it cut what had proven to be one of the major sources of GC lag almost completely, leaving only one (Terrain Heightmap Quad loading.)

(edit: it wasn't a fixed database, it was technically an external database provided by a localization library)

 

I just did ERG -- it was a *little* better maybe 2-3 FPS -- but turning off the GUI completely with F9 - holy cow - 10 FPS to high 50's.

So naturally, I played most of the party without the GUI, remembering my deck layouts pretty well- it was only hard with the unlocked glyphs.

 

Performance was better last night in ERG without a bard in the party by about 10 FPS - compared to a similar party today with a bard - I had to play most of the time with the GUI off. FYI.

 

While enjoying a random single-digit FPS, I pressed F9 and saw a slight different. Not enough to get out of the 15FPS hell that I currently burn in randomly. If I reload the game, it runs at 100+ fps everywhere I go but at random, when I look in a specific direction, the frames will tank and stay tanked until I quit the game.

EDIT #1: I decided to keep messing with it. I forced the game into a fullscreen borderless windows and added the -nolog and -force-d3d11-no-singlethreaded command line arguments. At his moment, I'm getting 100+ fps except when looking at someone's mega-mansion in a very large town. It drops to about 40fps there which is totally playable. It's also nighttime, so I'm staying logged in and waiting around until daylight to see if this is just a coincidence.

EDIT #2: It was just a coincidence. I've attached an example of what I'm dealing with, which from what I gather, isn't unique anymore.



MY SPECS:
Windows 10 Pro
MSI Tomahawk B450
AMD Ryzen 5 3600
Cooler Master MasterLiquid ML240R RGB
Corsair Vengeance RGB Pro 16GB DDR4 RAM
MSI NVIDIA GTX 1660 w/ 6GB RAM
Sabrent 960GB NVMe SSD
EVGA 700 BR PSU

EDIT #3: I'm going to try one other thing. Shot in the dark... I'm going to whitelist SotA with my virus scanner. For reference, I have no other programs running in the background while this is going on.

EDIT #4: Since I whitelisted the game folder with Windows Defender, I have only experienced predictable frame drop when looking at complex scenery. It's still responsive to the controls and it doesn't tank to single-digit unplayable framerates anymore. Will keep testing.

EDIT #5: Okay, after playing for a little longer, it began to sink to 1FPS or so in regular spurts. I guess it's not related to antivirus or graphical settings on the client side. *shrug*

 

Join a large party with a bard at ERG and play the waves for 10 minutes or so -- then compare with/without the GUI.

 

it’s the opposite - you will gain FPS - Chris is aware of it, thankfully.

 

I have noticed this issue as well while doing the Control Point in East Reach Gap. The reason i actually even stumbled upon this, is because i just got an RTX 3070 and was surprised at the low FPS there.

I thought dang, maybe my GPU is overheating?! So i turned on my monitoring software and realized that my GPU is hardly even being pegged while there. CPU seems fine as well.
When i hit F9 - Hide GUI - FPS shoots up by like 47. GPU starts to heat up again like its normal processing as well.

Just wanted to share my experience and what i have noticed. Definitely still seems to be an issue.

 

Merged the "GUI Causing Huge Lag" thread with similar thread already in Critical bug reports and updated title.