The refresh button becomes available after the list is done loading. Unfortunately, the server browser is kinda fubar at the moment.
Instead of continually restarting the game, you’ll almost certainly have better luck letting it finish loading the list (it takes 5-10 minutes sometimes and yes that blows!), then going to your favorites folder and clicking “refresh”.