I know you can just go there and put favorite and stuff.
The thing is: In the background the game loads all server data to find your favorites, and if you dont have an amazing connection, some servers delay a lot to show up.
My suggestion is: Add local stored server metadata that is available for you to set up.
Instead of a online feed the game always load, allow players to make a list of servers they trust and always have the same data to be made, and instead of loading all servers when you join the lobby, you just go there and click the server you want to play.
Like in the case of my local server (just not to advertise specific server, even mine). I am always going to 192.168.13.207:7777 server. Steambootstrapper is crap, it does not work reliably. So get me an option to add a list of servers that I go and put there in fields: 192.168.13.207:7777, Local Server, etc, and every time I start the game, instead of going to the lobby and waiting the game allow me to direct connect and have to write it down (sometimes servers do not show up even if they are GPortal), I just list my servers in my local machine, and connect.
It is much easier for each player individually update their list if needed be, than use the current system of server listing for going to specific servers you often play, EVEN if you favorite them, as you have to wait and sometimes they dont show up even so.