They do. Fireplaces generate +20 heat (in a tiny area in front of them) when turned on and +10 when not, same as a Furnace or Improved Furnace. But the difference is that Fireplaces default to on, whereas Furnaces default to off.
Insulated Wood buildings give at least +10 heat, I think it might even be +15. Black Ice Reinforced Wood buildings only give +5.
Also, you should still get the benefit of armor when logged off, so if you’re logging out up by Temple of Frost, make sure you’re in the best possible cold-resist armor. Cold resist armor is +1 heat per piece, +1 extra if it’s epic, +1 or +2 extra for exceptional or flawless, +1 extra on top of that if it’s Vanir armor.
Right now it’s not possible to make Flawless Vanir armor; the armorer spawn point is bugged. If it were possible, a full set of Flawless Epic Vanir armor would be +25 heat, comparable to the -25 heat that you get from a full Silent Legion set.
(Source: I spent about a week running temperature experiments.)