Hey bro, caterpillars can be a pain in the neck. The best thing you can do right now is pick them off. If you are only a few weeks away, that may be the best approach. You can use diatomaceous earth, which isn't a pesticide, to sprinkle over the top of you soil. It will kill the ones that haven't made it to your plant yet.
Check your plant every day, especially the under sides of leaves, that is where they are going to hang out at. They will first appear to be very small pieces of black snot, like the size of a pencil tip. If you can catch them early, they won't do much damage. Just pick them off and squeeze them. I had a pretty bad infestation a while back and it took me about two weeks of picking daily, sometimes 30-40 a day to get rid of them all. It just takes diligence. Check the underside of every leaf every day.