We have our three core team members testing currently, 100 testflight signups, they break down as friends and family, cool early adopter tech people, massive enterprise customers in retail and hospital systems, and physician early adopters. How do we think about rolling out our testflight beta? Over what time frame should we be considering? We are pushing releases essentially weekly at this point with a tiny team.
We just finished a small beta with 200 people on TestFlight, so I'd be happy to share my experience! I've run 5 TestFlights now totalling many hundreds of users so I'm somewhat qualified to help you out.
My main advice here would be to roll out to users as slow as you can afford to, but obviously there's a load of factors to consider, notably how much testing you've done so far.
The issue with large scale rollouts is twofold. Firstly, there's a massive hit to your servers all at once, which could cause availability issues and highlight any bugs. Secondly, the more people you get involved early on the more people will SEE those availability problems and bugs. If you control the rollout you'll be able to see how your systems react to the increased load and scale accordingly.
That goes for platforms too. What's powering your iOS app? Do you have a web app? Secondary services that add on to your main platform? If there's anything that it's possible to hold off on offering, you should think about it. It's best to start out with something that's really great and reliable - you don't get a second chance at making a first impression.
Most of this stems from the rather uncomfortable truth that people don't actually understand what betas consist of. You're probably hoping for 100 people that really want to help you improve your product, but what you may actually have is 100 people that really want early access to something that's already finished.
It's best to manage their expectations up front. Just be clear about the feedback you're expecting, the level of contact you'll have, your update schedule, how you'll be communicating with them, how often, etc.
There's other factors to consider here too like the reliability of TestFlight (Apple are usually great, but TestFlight has its own unique issues with sending out invites and invalidating them). You'll also need to think about the drain that handling feedback will have on your team's ability to iterate quickly. It sounds like you're doing a great job at the moment, so you need to consider how much impact it will have dealing with support requests for non-technical issues.
I'd love to help you more. Sounds like you're working on a really interesting product, and betas are right in my area of expertise. Let me know if you'd like to schedule a call.
Answered 9 years ago
Access 20,000+ Startup Experts, 650+ masterclass videos, 1,000+ in-depth guides, and all the software tools you need to launch and grow quickly.
Already a member? Sign in