I've been doing this a few years, and built a mortgage-industry-specific CRM that had about 1500 clients. It was fine.

That said, it was no grand slam by any stretch, but that wasn't because of our technical execution of product, it was more about my mismanagement around product-market-fit. Moving forward I'm still outsourcing, but as one of the others indicated, have a few tips:

1. I never, ever let 1 team dominate the show, and each team doesn't have enough code of the machine to make it all run without me or hold me ransom. Also, I'm very hands on, speaking to them daily, managing tickets via bitbucket, etc... there are dozens of pitfalls in oursourcing such as coder bait-and-switch i.e. they'll use some sharp fluent in english coder to win your business, then the moment it's committed shuffle you to the sloppy rookie with marginal english skills (with a lower hourly rate and thus more profit for them).. I'm pretty wary of companies personally preferring to gel our own culture and hire only individuals, but that has challenges too like more frequent power outages (people working from home in india are more likely to have this happen).

2. I'm technical, and though I'm a sucky coder, I do speak coder and understand it all pretty well including the technology, DB design, etc. so if someone tries to blow smoke at me, I know pretty quick. Frankly, you can often cut this off at the pass early by knowing the lingo and doing some research i.e. if you ask for a ZF2 / doctrine ORM coder, that's not really rookie-level system design and so you'll probably command a bit more respect out of the gate.

3. As we get traction and/or the burn rate can be prudently increased, I'd bring about 60% in-house full time staff coders, as outsourcing does have a hidden cost. I think this transition can be mitigated if you use single coders not companies, and have clean, organized, well documented code (and even then, styles vary among coders).

Hope that helps -
RogerV in Los Angeles