Saturday, February 23, 2008
Million Dollar Question - 20 % Brand new test cases ?
When is the last time you wrote a test case or a scenario which you had never written before?
When is the last time you came across a scenario you had never thought in your testing career?
If % of above scenarios (for a given project) is not more than 20 % then:
- You are not adding value to the customer by rewriting the old stuff.
- You are not realizing your true potential.
- You are reinventing the wheel
While doing test case creation, 80 % of times we write fewer than 20 % brand new test cases or scenarios (not trying to fit in pareto just because it looks cool)
I know by now you must have understood where am i coming from and you have would have possibly concluded that it is something not new what I am going to tell you.
But if I don't tell what I am going to tell you then there is no ways that you can find out what am I going to tell you:) so read on.
I promise you that two words I am not going to use in this post are "reuse" and "library", doesn't how much I will be successful because will be tempted to use them.
Reason: Reuse and Library are two most abused words not just in testing industry but IT as a whole.
Disclaimer: Author is not saying that he doesn't believe in reuse and library, its it just that we want to look at the true meaning of the words without using them.
Technique:
Next time you get an assignment of writing test cases, ensure following things:
First Step: TC Creation
1. Your test case / scenario contains two different logical sections - Generic Vs Application Specific.
2. Generic section should be written in plain English considering what that test case means to someone in real world generic step of execution and expected outcome for those
3. Application Specific section should contain details specific about your application only like exact steps, exact expected result, test data
4. Add a field specifying the "Area" - Web Application (.Net/ Java etc) / Desktop application / Database Application / ERP etc
5. Go through the complete cycle and keep on refining you Generic and Application specific areas.
Effort: (120 %: 100 % (Application specific sections) + 20 % (Generic sections) )
Second Step: Central Repository formulation
1. Pick only logical generic sections from the above set and classify them with different areas.
2. Spread this to other fellow testers and team.
3. Each times someone add any test case / scenario to this repository then quickly look for duplicates:
a) If duplicate, then merge both the test cases to have an enhanced test case.
b) If not duplicate, then make an entry into the repository.
Additional Effort: (20 % - Identifying duplicates and upgrade of repository)
Third Step: Think Hard - Test Case Creation
1. Now for this project (similar to above) ensure the following:
a) Take the generic test cases from the repository based on the "Area" filter.
b) Work on expanding the generic test cases
c) Now ensure that you think of at least 20 % new scenarios which are not present in the repository.
d) Go through the cycle and keep refining step b and c
Effort: ( Iteration 1: 80 %, Iteration 2: 60 %, Iteration 3: 40 %)
Fourth and Final Step: Paying back to the community
1. From Third Step, 1 (b), update the central repository to make the test cases more generic
2. From Third Step, 1 (c), add the test cases to the repository.
Additional Effort: (20 %, identifying duplicate and upgrade of repository)
Saturation Point:
After following Step 2,3 and 4 continuously over a period of time, you will reach a state of high TCM (Test Case Maturity) when it will start becoming identifying new scenarios and the effort will continue to go down and you get ROI of the work.
One man initiative:
It can be started by one man but can't be accomplished without strong management support and contribution from the community.
Million Dollar Question: Is it always possible to write 20 % brand new scenario's for every project which were not covered in past for a similar project?
Answer:
Scenario 1: If you haven't followed the above practice then yes have a very high chances of writing more than 20 % brand new scenarios very easily for a similar project.
Scenario 2:If you have followed the practices described above by author over a period of time (>3 Iterations) then the possibility of finding another set of 20 % brand new scenarios will become difficult and difficult with every iteration. And one day,you even after applying all your imagination / thinking and even after covering all the requirements, you wont be able to find 20 % more brand new scenarios and that is when you have reached saturation point.
Note: The more the difficultly in identifying brand new scenarios, the more (higher) is your requirement coverage and test case maturity.
New Testing term coined: Test Case Maturity
You must have heard of CMM levels which define the maturity of an organization's process. TCM is a term which defines the maturity of your test case.
Each time you write a test case , it has certain maturity associated with it which can be calculated by using the
1. Is the test case covering a most obvious requirement? - Level 0
2. Is test covering the very high level requirement ? - Level 1
3. Is the test case was already written by past and can be used as it is ? Level 2
4. Is it test case written in past can be re-adopted and can be customized for your project? Level 3
5. Is the test case can re-adopted and extended and made generic for other projects too ? Level 4
6. Is your test was a not present in repository and can be added to the central repository for other to adopt ? Level 5
7. Did your test case cover any unique aspect which can impact the existing test cases in central repository ? Level 6
Benefits:
1. Value addition to customer
2. Lesser effort required, lesser cost (> 3 Iterations, not overnight :) )
3. Challenging task for testers, keep them 'thinking'
4. Contributing to the community as a whole
5. Able to get time to think of scenarios and find bugs which never got time for.
6. Standardization of test case across community.
7. Easy for new comers, end users to execute and perform testing.
8. More accurate effort estimation.
Ending note:
Saturation point doesn't mean "stop there", but it means that you got to find our other ways to take this to the new level.
Example: You can dissolve 2 teaspoon of sugar in a cup of water in a matter of 20 secs,
another teaspoon of sugar, may be 30 secs, one more teaspoon , its becoming difficult.
just one more tea spoon, not it doesn't work . Is that is what we call a saturation point?
Thinking: The saturation point is different at different temperatures. The higher the temperature, the more sugar that can be held in solution.
So that's not the end, you can heat the sugar solution and it can hold more amount of sugar so keep trying.
But infinite amount of sugar, may be no.
BTW, did you do a CTRL + F to verify that if I did use the term "reuse" and "library" again :)
Please feel free to write your comments !!!! Contact: raj.kamal13@gmail.com
Sunday, February 17, 2008
Test Automation Change Control Process - Sounds Interesting?
This change control process is to ensure that in a "big" automation project whenre "Modular" Library architecture is followed, libraries are not mistakenly / intentionally changed by the automation testers and every change in the library goes through a well defined "Automation Workflow" depending upon the role and the access rights.
Changing libraries without this process can even cause the other automation scirpts to fail or having a possibility of redundant code being written for same functionality which is often the reason for automation failure.
Thought of depicting this process graphically so that everyone here can understand this process.
Changing libraries without this process can even cause the other automation scirpts to fail or having a possibility of redundant code being written for same functionality which is often the reason for automation failure.
Thought of depicting this process graphically so that everyone here can understand this process.
Library Change Control Process
Subscribe to:
Posts (Atom)