I recently started a own server, and its quite heavily modded for the moment - for comparison reasons.
Now playing there gives me heavy FPS breaks most of the time, then suddenly it works for the moment and a bit, then it breaks again.
I was not the only one testing that experienced this.
Also, when testing all those mods single player it was more smooth and not so heavy FPS drops.
Now I notcied that my GPU is not working at full capacity, sometimes only at 12%, and when it ran smooth the CPU usage had a spike upwards.
I know thats not all that much information, but maybe someone has an idea WHY the GPU is not doing its job as it should?
It could be a number of things, potentially a power issue, you can try disable all power savings features in the bios and set power mode under windows to full power.
It is also possible the system is intentionally throttling the GPU due to overheating or something like that, I recommend downloading HWmonitor (it’s free and non invasive), scroll down to GPU temp and utilization. boot up the server and let it run for awhile while monitoring the temp fluctuations.
It could very well be that the spike in CPU that made it more smooth was the CPU compensating for an problem with the GPU, this is not ideal as the CPU will take on additional strain if there is a problem with the GPU that it is trying to compensate for.
Have you had a look through windows event viewer while the server is running? that could also potentially give some clues as to what may be happening.