I’ve seen a situation where my Golems turn on one of the other Golems for no reason at all. Maybe your Golems killed your human slaves.
Set be with you