You can’t… you have to edit the Input.ini file manually to get rid of that one (and some others…)
Input.ini is located on your SteamApps\common\Conan Exiles\ConanSandbox\Saved\Config\WindowsNoEditor folder
Search for the action TogglePlayersList and change the Key=B to Key=None, or remove it from the list… don’t know which is better.