There is no evidence to indicate that they were serpent-men. In fact, there is no evidence to indicate that the serpent-men even still existed by the time of the Hyborian Age. They were already near extinction when Kull swore to exterminate them and now that man kind was aware of their existence, and how to spot them hiding in their midst, it is likely he made good on that promise. Where as if you read the descriptions of the Giant-Kings and various other people such as Akivasha you find very distinct commonalities which would lead to the conclusion that the Giant-Kings were in fact the pre-human race living in the region which would become known as Stygia. I have previously posted a link which goes more in depth into that analysis.