I have had them spawn at my base, but I did not have a spawn point there.
Was funny to see a pile of corpses underneath the one thrall I had at my base.