I recently went through a Start Up Weekend event as well as a Venture School program through the University of Iowa. The entire course was on doing exactly what you are asking. They teach the lean start-up and business canvas model of proving a business idea.
The core strategy with proving your business has potential is customer discovery - determining a hypothesis and testing it through interviews (customer discovery). You'll do these interviews with several different hypotheses, concerning every aspect of your business idea - the problem, the potential solution, who are the possible consumers, what will they pay (if they will pay), distribution models, sales tactics, etc. I've provided a few links that present the theory behind the business canvas. It provides a means to be agile in your business development, helping you identify when to pivot or change up your idea to best fit the market needs.
Good luck! Happy to chat further if you have questions about this concept or how to use it.