Nope, been there now on 6 official servers getting the recipes and didnt lose a single thrall.
Equip your thralls better and yourself.