Anyone have advice on developing a testing strategy that covers realistic production scenarios? I find unit tests to be fast but not very realistic, and functional tests to be realistic but slow and brittle. Anyone have a happy medium? I'm working on a systems integration project and we're leaning on BDD with Selenium, but I'd rather know my options. Maybe a mix?
Yes, there's a lot we can discuss.
We internally use BDD tests (Gherkin for user stories), coupled with Selenium via CoyPu wrapped around any mock-up output (in our case, Indigo Studio).
You can take a look at an example from a while ago on our YouTube Channel (Axelisys Wonder), specifically at:
https://www.youtube.com/watch?v=ZWeC56bL3Oc&list=UUdTOmUegcjXIPtZt2sVutRQ&index=11
What we typically do is:
1. Sit together with the business, BA and a Dev to develop user stories representing customer journeys using "As a... I want... So that..." often in conjunction with mock-up web pages which are then changed into static HTML pages.
2. Run through the primary scenario and develop the "Given... When... Then" statements.
3. Commit that to your Version Control system ("What?")
4. At this point, the devs or a DiT (Developer in Test - if your organisation uses that) can pull it out and wrap the mock-up in CoyPu/Selenium with references to the appropriate elements. The behaviour of the mock-up should represent the user journey. Hence the BDD test should use the same data as the mock-up did. However...
The maximum number of tests should be 2 to the power of the number of "Given" "Ands". So if you have
"GIVEN I am 17 AND I have a driving license"
This counts as 2 test variates and as such, creates upto 4 test cases.
5. As the devs work through the process, they fill out the mock-ups and push data back through the layers (perhaps walking the skeleton for the primary scenario) and filling in any data persistence layers etc. as they go. The very same tests are used to regress existing functionality too.
So the test strategy is basically to make sure you have covered as much of the valuable journey as possible (all primary scenarios/happy paths and the most valuable secondary scenarios, as long as TDD level tests can adequately cover the rest).
How do you know what is adequate? I'll direct you to a couple of articles.
Feel free to arrange a call if you need more information.
http://goadingtheitgeek.blogspot.co.uk/2015/02/code-coverage-metrics-cyclocmatic.html
http://goadingtheitgeek.blogspot.co.uk/2014/09/going-from-acceptance-tests-to-code.html
Best of luck!
Answered 7 years ago
Access 20,000+ Startup Experts, 650+ masterclass videos, 1,000+ in-depth guides, and all the software tools you need to launch and grow quickly.
Already a member? Sign in