Hi and welcome to the forums 
Sorry to hear this is happening to you (as well) - this seems to be a major ongoing bug with the Living Settlements system. A lot of people are reporting the same sort of experience you are having. Hopefully Funcom is working on a fix (it can be hard to tell, so it’s worth every report that gets made). In the meantime, you may be interested in adding your voice to the unnofficial petiton to get Living Settlements turned off on official servers (again) until the problem is actually fixed. If so you can find it in the thread here.
2 Likes