This one had a similar thought. Unfortunately, it did not quite work that well.
That said, Golems are a very solid option for the tower defense aspect.