This is your main problem. In a single player game, your client is also the server, and it saves resources by only dealing with what is rendered in on your screen. This means there are no AI’s to deal with in areas that you don’t care about (like on the other side of the map). The problem with this is that when you enter an area the server (your game client) has to spawn everything in for you.
If you insist on playing single player, then probably the only way to fix your problem would be to host a server (just for yourself) that runs on your own computer. This will require a little extra work but can be done fairly easily and for free. It will greatly improve your game experience and will also fix other problems that are native to single player.
This is the download for the latest Dedicated Server Launcher (1.6.2) and here are the installation instructions. A long installation guide always makes something sound overly complicated, but I assure you it is fairly simple. Best of luck to you. 