The causes can be numerous.
But there is one thing here NOT causing this: Conan Exiles
I’m going to get this out of the way first as its a very common misconception. No game can make your computer overheat. Your computer parts, (specifically your CPU and GPU) can run at 100% of their capability for the duration of their expected product lifespan (which is about 5-10 years). That’s 100% capacity 24 hours a day, 7 days a week, 365 days a year.
So what causes them to not be able to do this? That’s the question you are posing obviously.
Cooling Failure
Malfunction/Damage
Mistuned Overclock
Cooling failure is the most likely. This can come from dust in the heatsinks, obstructions in the airflow, or poor airflow. You’ve said it wasn’t dust but make sure there is no dust on any heatsink in the machine or on the blades of any fan within the case. You also want to make sure the intake and exhaust of the machine are not obstructed and make sure there is several inches of space in those areas. And finally, you need to make sure the ambient temperature of the area the PC is in is also kept cool. Room temperature should not exceed around 80F (26C). Also keep relative humidity below 85%.
The device in question could actually have a defect or damage. This isn’t easy to test as you would need another machine or another device to test by swapping parts. However your CPU and GPU are both not brand new components. They are good enough to play the game on decent settings. But they are getting close to the end of their expected lifespans. Thermal stress over the years may finally be catching up. Nothing can be done about that except for replacement if this is the issue.
Finally as I stated before, that devices are designed to run at 100% for the duration of their lifetime. This is assuming factory default settings with NO overclocks. Overclocks can not only cause stability issues, but they can also push devices into heat levels that their cooling systems are not designed to handle. If an overclock is being used you do need to use aftermarket cooling devices to keep them cool. Also overclocks cause far more wear and tear on devices and can sometimes significantly shorten their operational lifespans. You did not mention an overclock so I assume there is none, though if there is, this can explain some of the issues too.
As others said already I think its a cooling system as well. My experience with a PSU without enough wattage is that the whole system shuts down when too much power is drawn. But that was nearly 10 years ago. But I wouldn’t put it past a faulty power supply giving dirty power to a GPU and overheating it.
So here’s my recommendation. Check the fans for dust, make sure airflow is clear. Also check to make sure all fans are running. Including the ones on the GPU. It means you’ll need a little flashlight and get your eyes looking at it underneath while it is running with the side panel off.
If any fans aren’t working, they’re cheap to replace. If its fans on the GPU see some tutorials about how to replace those if that’s possible. If not then a new GPU will be in order. Good news is your motherboard and cpu are compatible with all modern GPUs.
While its understandable that all your games don’t require this, that they do not draw as much GPU power as CE does. Understand that newer games will. Addressing the issue for Conan Exiles will fix future issues.
Personally I think you will eventually need to upgrade the GPU. Thankfully your CPU is quite strong for its generation and will serve you until it breaks.