Whitelist corruption when using "extended characters"

Conan Exiles PC Discussion PC Bug Reports
,
1

Basic Info:

Platform: Steam
Issue Type: Other
Game Mode: Online Private
Server Type: PvE
Map: Exiled Lands
Server Name: Lost Horizon [Private Server]
Mods: None. (We use all kinds of mods, but the issue was tested without any mods at server setup.)

Bug Description:

Server writes whitelist.txt as UTF-16 when it comes across “extended characters”, but cannot then read the file back into memory without corruption. A guess would be that the game always reads the file into memory as UTF-8. (Note that the characters don’t need to actually be unavailable in UTF-8. My Funcom ID is Ahnìon#93756, which is entirely covered by UTF-8, but the game reverts to UTF-16 as soon as my ID is added.)

Bug Reproduction:

  • Add the SteamID of a user with extended characters in their Funcom ID to the whitelist (via console, RCON or manually editing whitelist.txt)
  • Let the player log onto the server.
  • Do something that makes the game flush the whitelist to disk (such as adding a new player.)
  • whitelist.txt is now UTF-16.
  • Restart the server.
  • Whitelist in memory is now broken. If written back to file, it will use UTF-8.

Issue has been reported before,
here: Whitelisting Likes to Break - I think I know why
and here: Whitelist failed to load

2

A friend of mine who is a lot more of a coder than I am pointed out that the Windows memory buffer is UTF-16, which makes it sound like there might a conversion (to UTF-8) during write that fails (silently?) when extended characters are encountered.