How do I deal with a partner/investor that hasn't delivered on his end of things and now wants his money back?

My investor wanted to partner up instead of just investing, he hasn't met any of obligations as a partner and now I'm stuck with just 50% of my company. We didn't write up a contract so I only have a verbal (email) agreement. He wanted me to pay back the investment money if the startup didn't work. What should I do? I can't pay back the investment (although I'd love to).