I tried several solutions myself while building my businesses, and it also depends on the type of business that you are planning to build;
1- If its digital with no need for F2F/virtual contacts with customers - You should maintain your 9-5 job, and work the second shift in your startup, or try a job that gives you some schedule during the day for your startup F2F appointments with customers.
2 - If you are building something that needs a lot of demos, meetings with customers, you will need a lot of flexibility and consulting or freelancing is best for your time management.