Just wondering how to run a closed-beta for an ios app. 1. How many users? 2. What kind of tools? (TestFlight? - recommend me!) 3. Other things to keep in mind

Apple will allow a developer to register 100 UDID devices per 12 month cycle to test via TestFlight or HockeyApp. Having started with TestFlight, I would really encourage you NOT to use it, and go directly to HockeyApp. HockeyApp is a much better product. There is also enterprise distribution which allows you far more UDID's but whether you qualify for enterprise distribution is difficult to say.

As part of your testing, I'd encourage to explicitly ask your testers to only register one device. One of the things we experienced was some testers registering 3 devices but only used one, essentially wasting those UDID's where we could have given to other testers.

Who you invite to be a tester should be selective as well. I think you should have no more than 10 non-user users. These people should be people who have either built successful mobile apps or who are just such huge consumers of similar mobile apps to what you're building, that they can give you great product feedback even though they aren't your user. Specifically, they can help point out non obvious UI problems and better ways to implement particular features.

The rest of your users should be highly qualified as actually wanting what you're building. If they can't articulate why they should be the first to use what you're building, they are likely the wrong tester. The more you can do to make them "beg" to be a tester, the higher the sign that the feedback you're getting from them can be considered "high-signal."

In a limited beta test, you're really looking to understand the biggest UX pain-points. For example, are people not registering and providing you the additional permissions you are requiring? Are they not completing an action that could trigger virality? How far are they getting in their first user session? How much time are they spending per user session?

Obviously, you'll be doing your fair share of bug squashing, but the core of it is around improving the core flows to minimize friction as much as possible.

Lastly, keep in mind that even with highly motivated users, their attention spans and patience for early builds is limited, so make sure that each of your builds really make significant improvements.

Happy to talk through any of this and more about mobile app testing.

Answered 6 years ago

Unlock Startups Unlimited

Access 20,000+ Startup Experts, 650+ masterclass videos, 1,000+ in-depth guides, and all the software tools you need to launch and grow quickly.

Already a member? Sign in

Copyright © 2020 LLC. All rights reserved.