On the wiki and in the game it says that to get a water filled glass flask i need ice. is that the only way? Why cant i take my empty glass flask down to the river?
You can. Then put it in the Stove to cook it, creating purified water.
You can. Put an empty flask on the hotbar and use it at any water source - lake, river, waterwell, Mitra fountain. You can even put a hole stack (20 flasks), you just have to push the button 20 times 