NPC buildings used to be actual buildings back in the early days, but they were changed to static meshes because of said performance impact.
I remember blowing up Black Hand tents with a trebuchet. Good times. (The pirates may not agree about the goodness of those times.)