To prevent this and other things I’ve seen on the forums I simply don’t have thralls outside my walls anymore and I put my treasures in a simple chest. I’ll put them in a clan chest if and when I want to do a purge.