Sometimes it's not the bugs that cause the problems -- sometimes the bugs *follow* the problems. I don't know what these little guys are, but the fact that the problem started from the bottom of the planter and moved up makes me wonder if the soil at the bottom stayed too wet for too long, and the roots started to rot, thus attracting the bugs.

Most small, soft insects that hang out on leaves and stems are susceptible to soap spray, but I'm not sure what to do if they're mostly in the dirt.