Anthony W. Richardson is the author of "Full-Scale: How to Grow Any Startup Without a Plan or a Clue" and General Partner at Agility Collective -- an entirely new model for venture capital and entrepreneurial support.
I believe that the old moniker "If you don't value your time why should anyone else" rings true here. Giving it away for free may indeed give you great experience *if* you can allow people to let you into the fold. Consider startups that *can* take on interns but don't simply because its a chore to see a good example of this in motion. If you're not creating value in yourself you're certainly not going to translate that value to your clients, customers etc..
I think that the answer is dependent on the level of technical talent that is already on-board. If you have an engineering focused squad, you can get away with rudimentary understanding. To your description a "non-technical CEO" is just that. Of course, whenever there is a gap in the team, it is generally the CEO's job to wear different types of hats as the weather changes. One of those hats may be a hacker.