Save broken, infinite loading for some players

[mAritz]

Game mode: Online private
Type of issue: Bug
Server type: PvE self-hosted dedicated (no BattleEye)
Region: EU

---

I’m having the same issue for multiple players on my own server. Only solution seems to be to roll back saves.

It started about 2-3 days ago when one player couldn’t load in anymore (stuck on loading forever). After a day (and some testing to try and fix it) he could log in, we’re not sure what fixed it.

Since yesterday another player and me can’t load in anymore. Even after complete reboots of the clients and the server, it didn’t change.

The other guys both said that they waited 20 minutes once to see if it eventually loads and said it didn’t.
For me it loads after about 10-20 minutes. However, after that I’m respawned (probably died of hunger/thirst during load) and am bugged: No UI, can’t walk. The only UI element that works is the console. But none of the menus (ESC, Ctrl+M, Inventory, Map, etc) work. Movement doesn’t work, but attacking, stance change, roll, jump work.

I tried disabling all mods, no help.

I also don’t see anything in the server logs that looks like it could cause this. This is what the launcher shows for players though, so login itself is not the problem:

[2020.09.24-13.14.27:228][298]LogNet: Login request: /Game/Maps/Startup?Name=[…snipped out because sensitive data…]
[2020.09.24-13.14.28:080][349]LogFuncomLiveServices:Display: VerifyIdentity Completed: PlayFabId - [1a34fc8cd8fab0c8fbe9881075936286] PlatformId - [e3fa9292cc7cc6bb1991de7b25c87cee].
[2020.09.24-13.15.06:284][632]LogNet: Join succeeded: […my friend id…]

---

Please provide a step-by-step process of how the bug can be reproduced. The more details you provide us with the easier it will be for us to find and fix the bug:

1. Start server
2. Try to connect
3. …
4. Stuck on load or load in after 20 minutes in unusable state

Sorry, I really don’t know how to trigger this in a save game. I could of course send the broken save game or upload it or something, if that helps?!

[mAritz]

I uploaded the broken save here: https://drive.google.com/file/d/1SfGYwVrbcpqqDi6lZJyLx8fcmNisDm18/view?usp=sharing
The working one here: https://drive.google.com/file/d/1LJGcx84AnxHPvosx4oOagsYUtf8YJIWF/view?usp=sharing

[mAritz]

Okay, this is getting weirder. If I go to console during load and suicide from there, the loading screen goes away and I suicide. But there is still no UI and I can’t move.

[mAritz]

I managed to half-way kind of fix it.

Steps to temporarily fix the broken save as an admin that can’t connect:

1. Wait a few seconds in the loading screen to make sure you’re loaded in in the background (just guess, no indication afaict)
2. Open Console in the loading screen via shift+insert
3. Type “Suicide” -> The loading screen should go away and you should be dead with no UI
4. Wait for respawn
5. Make yourself admin via console “MakeMeAdmin YourPassword”
6. Make yourself fly via console “Fly”
7. Destroy your your spawn bed/bedroll via shift+del while kinda looking in the direction of your bed. This is a bit difficult and can lead to accidental removal of much more than intended. (repeat for all your spawn beds)
8. You will now respawn in the default spawn location with still broken UI
9. Restart game -> You spawn with UI and can move and everything works

To fix other players, repeat steps 7-9 with them.

However, upon reconnecting again you are back at step 0 and need to repeat the whole process.

[mAritz]

Okay, we figured out that it apparently has to do with distance to our base. I tried deleting random things from our base in batches and at some point it worked again. But trying to figure out what exactly it was is next to impossible.

[Ignasi]

Hey @mAritz

This long loading issue originated by some assets could be caused by a couple of reasons. First is the use of mods. Some have not been updated to be compatible to the latest version of the game and could cause issues by themselves or in combination with other mods. The other could be the hardware where the server is hosted on. Keep an eye on RAM and CPU usage when hosting a server, if possible, especially when using mods or hosting it in the same computer as the one you’re playing the game with.

[mAritz]

Hey there, @Ignasias.

The easy one first:
Resources are not an issue. This machine has 32GB RAM and a Ryzen 7 2700x. It’s not even close to being utilized fully whenever I look at the usage (I have graphs of that on one monitor). I even run it at 60hz and >90% of the ticks are <16ms with the worst 0.1% still being <50ms - according to the GetServerHealth command.

Mods:
Removing all mods did not fix this issue - but I guess a mod could destroy a savegame in a way that even removing the mod won’t help?! Aside from that I checked all mods for being updated after the recent update, so that shouldn’t be it. Usually the entire game/server doesn’t start when mods aren’t updated to the september patch.

I went to the base as Admin and removed a lot of stuff bit by bit, reconnecting after about every 20-40 assets removed to check if it broke. I removed the recently built mod assets first and it didn’t make any difference. Since we know the change to the loading happened in a time span of about 3 hours (working rollback save and non-working save are uploaded), we kinda knew what items could’ve caused it. But in my testing it turned out to be none of those - it’s however entirely possible I missed some.

At some point it worked again as it should, after I removed a bunch of stuff. So I reloaded the fresh broken save again and tried to remove the same things again, but this time it didn’t help or I missed the thing again.

Our “solution” for now is to have warp gates from/to a far-away safe login zone and from spawn locations. When someone forgets to log out there, they have to suicide via console and hard-restart (Alt+F4) their game.

If there is anything I can do to help debug this issue, feel free to ask. I’m a developer myself and debugging is actually fun to me.