Some very interesting findings, thank you.
I’ll have to keep my eye out for stuttering on testlive - must admit I hadn’t noticed any yet with the current version (also singleplayer, also nvidia (1060 in my case)) - I spend most of my testlive time in admin mode, hadn’t directly noticed a problem, but I did suffer a crash today after about 30-60 mins. Now I’m wondering if being in admin mode may have contributed. (It also sounds like I have less going on in my testlive base than you do, which may also be why I haven’t noticed stuttering - though I also haven’t had the problem in my live branch base, which is much more heavily populated.)
I have a growing suspicion that the current testlive patch may have some performance issues that are causing it to lag much more than the live branch - detailed more in reports on the testlive forum (here). I wonder if you may have uncovered some more clues as to what’s going on - I’ve added a link back from the bug report to this thread as I think it may be useful 
It also makes me wonder if the same points might apply to the current live branch and just not be being noticed because it’s running smoother overall.