17. Tip: Think carefully about which outcomes you want your tests to cover
This lesson highlights the importance of thinking carefully about not only your test inputs but also the outputs. Including parameters and values is not sufficient. You should ALWAYS think critically about what different types of outcomes you would like your tests to cover. If you are putting tests together for a System Under Test that has three types of common outcomes, for example, you should probably make sure that each of those outcomes appears in your set of tests a reasonable number of times. Doing so will often require you to modify how you have defined your test inputs and/or require using the Requirements feature.
Equivalence classes aren't just for test inputs!
Depending on your System Under Test, you might make the determination that whether a user of the application selects "Spindrift," "Teal," "Sky," or "Turquoise" will almost certainly not matter. Any one of those colors, because they're all "Blue" would be treated equivalently by the system. That is a very common test design technique. Indeed, when you select test inputs to include in Hexawise, you're strongly advised to use Equivalence Class partitioning. Doing so will allow you avoid long lists of Values in your test inputs, which will help you keep the number of generated tests down to a reasonable number. But don't stop your equivalence class partitioning there (with your test inputs).
Think about equivalence classes of test outputs too!
Imagine that these three types of shipping fees are possible outcomes from a System Under Test you are designing tests for - and that your colleagues are counting on you to test those 3 outcomes. Are you sure that the tests you generate will result in all three of those different outcomes? If not, you should use one or more of these strategies to ensure that each of the outcomes gets covered in your tests at least once:
- Use Hexawise's Requirements feature to force a specific "high priority" combination of values to appear. If Free Shipping was only applied to orders of 100 or more bottles of champagne shipped to Palm Bay, Florida when the order is placed on a Tuesday and paid for using an American Express Platinum card, you would need to specify (a) the quantity, (b) the item, (c) the location, (d) the day the order was placed, and (e) the type of credit card used. Doing so would result in a test being included in your test set that would allow you to test for that specific result.
- "Do Nothing" - If only one or two test inputs is required to trigger an outcome, and you have already included those test inputs in your plan, you're all set. You don't need to do anything else to trigger that specific outcome. Let's say "extra shipping fees" would be triggered whenever a washing machine was ordered in the summer. Provided you've included both item = "washing machine" and time of year = "summer" as test inputs, at least one 2-way set of tests will definitely include both of those values in the same test, so you will be 100% that you'll be testing the "Extra Shipping Fees" business rule even before you click on the "Create Tests" button.
- Add a Needed Value - In some instances, when you ask yourself "What test inputs would need to be included in a scenario for the desired result to occur?" you'll realize you've forgotten to include a Value. Referring to the last example again, if we had forgotten to include values for "time of year," we would need to go back to the Define Inputs screen and enter Time of year as a parameter and include "summer." Otherwise, we wouldn't have any way to trigger the "Extra Shipping Fees" result.