For me its not random at all, they return as soon as I leave their render range, I tested multiple times putting them on hold and walking away backwards till their model disappears, instantly they return behind me and give the little grunt noise.