Engagement default is 100% ‘Attack All Enemies’. I usually put down next to Set City and every time I forget to change my new thrall’s behavior when I enter the town, he tries to go on a killing spree.
*Have not verified in more than a week, so if they revised the AI setting to not attack neutral NPC’s, this might not still happen.