Abstract
Rational Unified Process raises everyone’s hope yet often confuses testers to follow it. IBM Rational Tools are used extensively but there is no documentation available, stating when and how to use Rational Testing tools in different RUP phases i.e. Inception, Elaboration, Construction and Transition. It frustrates and disappoints testers. Although Rational Unified Process provides “Tool Mentors” to explain the usage of all the tools but still there is no direct mapping between usage of the Rational Testing Tools and Testing Life Cycle which explain the responsibility of the Testing team through out the SDLC in different Iterations.. This paper presents the guidelines which will suggest the role and responsibility of testing team in different phases of Rational Unified Process. Follow these guidelines as you staff, tool, or schedule your test automation project, and you will be well on your way to success.
A Fable
I have worked in many testing projects, big and small. I have seen different projects following RUP model and I have personally talked to many testing teams. I am presenting this paper to avoid these problems faced by them. But first we need to understand it. Let me illustrate with a fable.
Once upon a time, we have a big project following RUP for a very important client. Client provides the team with Rational Testing Tools. Sujata is a Test Manager; she pulls the experienced testers. Everyone is very excited as they are going to work with best tools in the market and no more excel sheet to write test cases, track defects and to generate report.
Testers have been trained on Rational Unified Process. Inception Phase, project starts and lot of testing tools available like ClearQuest, RequisitePro, Robot and Test Manager. Testers know how to use tools, thanks to Sujata once again for sending them for Rational Tools Training before putting them into project. They know Rational Unified process too but they don’t know how to use different tools in different phase (Inception, Elaboration, Construction and Transition).
They use RequisitePro for requirement understanding, test manager for writing test cases but still not sure when to use them and in which phase. Elaboration phase comes followed by construction phase and finally transition phase and Sujata realizes that nothing is in place. All the requirements are not documented, changes are not documented, test cases mapped to requirement have become obsolete, automation was pushed too late, and version was not maintained after a certain period.
Sujata couldn’t find the problem, she provided the best training to the best testers and they used the best tools in the market but still they couldn’t manage the project. She quits and hand over to Ashish, new test managers. Project is a failure.
That's my fable. Perhaps parts of the story sound familiar to you. But I hope you haven't seen a similar ending. This paper will suggest some ways to avoid the same fate
The Problem
This fable illustrates several problems that plague testing project following RUP:
What to use when?
This is the major problem which hinders the success of the projects and many times result in total disaster. In the worst case, testers don’t know RUP, usage of automation tools which stops project from doing well. If we consider the best case as mentioned by me in the previous section that even if testers are trained in RUP and usage of testing tools like Robot & TestManager still project might not succeed because they should know when to use Robot, when to use Test Manager and when to use other testing tools like ClearQuest and ClearCase in different phases in RUP.
How to use?
Other big question is the functionalities and features of the tools those should be given more focus in different phases of RUP.
Who should use?
This question can be answered if the tester knows what he is supposed to do and what manager is responsible of doing.
Guidelines- usage of Rational Testing Tools in a RUP Project
This paper will be organized by the normal steps that we all use for our testing projects, making special notes of the considerations and challenges that are particular to RUP.
1. IBM Rational Administrator
2. IBM Rational ClearCase
3. IBM Rational RequisitePro
4. IBM Rational TestManager
5. IBM Rational Robot
6. IBM Rational ClearQuest
All the above mentioned tools are explained with respect to their usage in different phases of RUP. I haven’t covered Rational Rose as I wanted to keep the focus restricted to the tools which are used in software testing.
I am giving a brief introduction about RUP phases for the audiences who are new to RUP.
Inception – Focus of this phase is the understanding the scope of the project.
Elaboration: The architecture as wells as the requirement of the products being built must be understood by the end of the phase.
Construction & Integration: Product must be constructed in this phase.
Transition –The product must be rolled out to customers in this phase.
Step 1: IBM Rational Administrator
a) Phase: Inception
During this phase, a shared repository is created using either MS-Access or SQL Sybase Central. A new project folder is created in the share with UNC (Unified Naming Convention). The project should be UCM (Unified Change Management) Enabled to ensure integrity with Rational ClearCase for Configuration Management. Rational Administrator provided the platform to integrate all the rational tools together to achieve end-to-end mapping.
Activities:
Create a project
Connect to a project
Create a test datastore
Create integration between products
Decide whether to associate the administrator project with UCM project.
Fig.1. Mapping of RUP phases to Rational Testing Tools
b) Phase: Elaboration
During Elaboration phase, Rational Administrator is used to create new users with different access rights depending upon the requirement of the project
c) Phase: Construction & Integration
Like Construction phase, Rational Administrator is used for managing users and access rights. It also maintains the integrity between different tools used.
d) Phase: Transition
Like Elaboration & Construction phase, Rational Administrator is used for managing users and access rights. It also maintains the integrity between different tools used.
Step 2: IBM Rational ClearCase
a) Phase: Inception
ClearCase provides life cycle management and control of software development assets. With integrated version control, automated workspace management, parallel development support, baseline management, and build and release management, Rational ClearCase provides the capabilities needed to create, update, build, deliver, reuse and maintain business-critical assets.
Fig.2.Rational Clear Case Workflow
Activities:
Create the configuration management repositories that represent the subsystem defined by your architecture.
Import existing files and directories into the repositories to create an initial set of development configuration items.
Enable your ClearQuest schema to work with UCM
Enable your clear case project to work with ClearQuest.
Set Policies for new projects
b) Phase: Elaboration
During this phase too new assets created are base lined and updated. When new test cases are created using Test Manager and new scripts are created using Robot are base lined using UCM concept.
Activities:
Updating your project work area using Rational ClearCase.
Checking in and checking out configuration items.
Delivering your work with ClearCase
Promoting project baselines using Rational ClearCase
Comparing baselines using Rational ClearCase.
Fig.3.Rational Clear Case detailed diagram
c) Phase: Construction & Integration
Out-of-the-box integration of Rational Test Manager, RequisitePro and Rational ClearQuest with Rational ClearCase LT delivers full base lining capabilities to accommodate the natural evolution of a project, moving from one build or release to the next.
d) Phase: Transition
In transition phase, all the artifacts are delivered and existing one are moved to baseline to maintain version control.
Step 3: IBM Rational RequisitePro
a) Phase: Inception
During this phase RequisitePro is mainly used for Managing Requirements. Inception phase focuses mainly on managing the requirements collected.
“Manage Requirement” is one of the best practices in RUP. Research from Standish Report shows 31 % of projects gets failed because of poor requirements managements, incorrect definition of the requirements from the start of the project and poor requirement management throughout the project life cycle.
Requirements involve the transitions of stakeholder requests into a set of key stakeholder needs and system features. These are detailed into specifications for functional and non-functional requirements.
Traceability helps us to assess the project impact of a change in requirement, impact of a failure of a test on requirements, manage the scope of project, verify the implementation of requirements and manage changes.
With IBM Rational RequisitePro there is little to no learning curve as requirements can be created and updated directly in Microsoft Word, a familiar environment for most people. Documents contain requirements in addition to contextual text that is vital for understanding requirements.
IBM Rational RequisitePro is integrated with IBM Rational Rose XDE, IBM Rational TestManager, IBM Rational SoDA, IBM Rational Unified Process and IBM Rational ClearQuest, as well as Microsoft Project. Integrated tools provide a smooth workflow and eliminate errors due to requirement discrepancies across tools. Integrations keep the entire project team in sync and improve team productivity. Easy access to requirements from other tools encourages all team members to view the requirements before starting their work.
IBM Rational RequisitePro support for traceability allows you to easily set up and track relationships between requirements to verify whether high-level requirements are associated with detailed software requirements. Querying these relationships provides coverage analysis to ensure completeness and to make sure time is not wasted building irrelevant functionality.
With IBM Rational RequisitePro, when traceability is established between requirements, and one of the requirement changes, suspect links automatically appear. This allows you to identify other requirements that may be affected by a change.
Fig.4.Rational RequisitePro – Shows Coverage Analysis, Features and Vision, Glossary, Impact Analysis, Supplementary Requirements, Use Cases and Requirement Management Plan
Activities:
Setting up RequisitePro for a project.
Adding Templates to your Rational RequisitePro project.
Capturing a common vocabulary.
Developing a vision using Rational RequisitePro project.
Detailing a Use case.
Detailing a Business use case.
Reviewing Requirement.
b) Phase Elaboration
During Elaboration phase, RequisitePro is used to set priorities for the Requirement and the attributes for the requirements to be implemented.
In the Elaboration phase, we identify system imperatives and requirements. The product we were developing is part of a larger offering package whose products must comply with system imperatives. Although these requirements did not always add value to our product or our specific client base, we had to invest the time and resources required to implement, test, and document compliance features. This meant cutting our feature list, which required another round of evaluations. At the end of Elaboration, we enter the remaining requirements into IBM® Rational RequisitePro®, our requirements management tool.
When using IBM Rational Rose XDE, suspect links are displayed between changing requirements and their associated design elements, so that it is clear what parts of the system are affected. Understanding the impact of change allows you to more accurately assess the potential cost in time and money of changing requirements.
With IBM Rational RequisitePro, defining attributes like priority, difficulty, and cost and having the ability to sort and filter requirements based on these attribute values allows project managers to objectively identify the highest value requirements
Having carried out the risk-to-use-case mapping for all of the technical risks, we can remove use cases without arrows as candidates for development during Elaboration; we can safely postpone these risks until the Construction phase.
Then we can look for the risk and identify all the intersecting use cases; implementing any of those use cases will attack that risk. If we continue this exercise for each risk, it soon becomes apparent that we can attack a number of high risks by implementing just a couple of use cases during Elaboration.
However, remember that earlier we said the goal of Elaboration is to implement the smallest amount of functionality that will confirm whether the high-priority risks we identified either
Activities:
To set priorities for requirements and attributes.
Analyzing the impact of changes by showing suspect links.
Assessing the potential cost and time required for requirement changes.
Sorting and filtering requirements.
Attack requirements with technical risk first.
Create and Manage Traceability
Fig.5.Rational RequisitePro shows traceability between Features and Use Cases
c) Phase: Construction & Integration
Out-of-the-box integration with Rational RequisitePro ensures seamless traceability between requirements and test cases, resulting in clear test coverage metrics and sensitivity to requirement modification.
d) Phase: Transition
In transition phase, progress reports can be generated using requisite pro to measure requirement coverage and traceability between the requirements.
Step 4: IBM Rational TestManager
a) Phase: Inception
Use TestManager in this phase to design test cases and analyze test logs to do POC (Proof of Concept) for the very few requirements to be implemented. We will be covering this tool exclusively in Elaboration, Construction & Transition Phases.
Activities:
Creating a test plan
b) Phase: Elaboration
Rational TestManager, which ships with Rational Robot, is the central console for test activity management, execution and reporting. Built for extensibility, it supports everything from pure manual test approaches to various automated paradigms including unit testing, functional regression testing, and performance testing.
Rational TestManager is meant to be accessed by all members of a project team, ensuring the high visibility of test coverage information, defect trends, and application readiness.
Establish and manage traceability: Requirements are linked to test cases, ensuring proper test coverage. In addition, suspicion analysis ensures that when requirements change, test cases traced to the requirement are automatically flagged as possible candidates for modification.
Rational TestManager puts the test team in control of the testing effort, focusing their effort by automating and simplifying crucial tasks. The tool allows teams to start from, keep track of, and test all the required functionality of an application, helping to ensure no critical business requirements go untested.
Activities
Test Folder Creation
Test Case creation
Test Case Design
Associating test inputs and attaching External documents are given more emphasis.
Test implementation both Manual and Automated is done for the requirement in scope for elaboration phase.
Fig.6.Create a Test Plan, Test Case Folder and Test Cases under that in hierarchical fashion
c) Phase: Construction & Transition
Support for all test types: Plan, manage and execute functional, performance, manual, integration, regression, configuration and component testing from the same interface.
Support for local and remote test execution: Run tests on the local machine or on remote machines in the test lab. Parallel test execution is limited only by the number of system resources at your disposal.
Detailed test evaluation: An integrated log viewer constructs a log for each test run, including test status and environmental information.
Out-of-the-box integration with Rational ClearQuest enables in-context defect submission directly from the TestManager Log Viewer, automatically porting test run data into the defect report to ensure accuracy.
d) Phase: Transition
Test manager is also used to generate coverage reports to analyze the testing effort and progress for the current phase or iteration. Test plan can be exported to a parcel file and can be delivered as an artifact.
Step 5: IBM Rational Robot
a) Phase: Inception
User Robot to create scripts for the requirements implemented in Inception phase as a part of POC (Proof of Concept). Even Robot will be explored in the subsequent sections.
Activities:
Setting up the test environment in Rational Robot
POC (Proof of Concept)
b) Phase: Elaboration
Test Scripts are created for the requirements covered in elaboration phase. Test Execution is carried out for the same.
Identifying and developing reusable parts is difficult. It is during the iterations in the elaboration phase that common solutions for common problems are found. This is the phase were many reusable functions or procedures are written in the common libraries to be used effectively in subsequent iterations.
c) Phase: Construction & Integration
In construction and integration phase, Rational Robot is exclusively used to execute regression test scripts (both manual as well as automation scripts) to do regression testing and new test scripts are developed for the new requirements being implemented in this current release so that the same can be used in the subsequent iterations to perform regression testing. As we know majority of the requirements are implemented in this phase so for these requirements automation is only possible in this release.
IBM Rational Robot can be used to automate and perform regression, functional and configuration testing for e-commerce, client/server and ERP applications. It's used to test applications based upon a wide variety of user interface technologies, and is integrated with the IBM Rational TestManager
I am listing down the advantages of using Rational Robot for functional testing:
Simplifies configuration testing.
Tests many types of applications
Ensures testing depth
Tests custom controls and objects
Provides an integrated programming environment
Helps you analyze problems quickly
Enables reuse
Fig.7.Create modular test scripts and include the VP’s and libraries used (depends on the automation framework being used)
Activity: Creating Test Suite
Implementing each Test Suite, and managing all subsequent changes to it
Ensuring that the Test Suite accurately reflects the test idea being realized
Ensuring that the Test Suite is implemented according to defined standards so as to be compatible and maintainable with other Test Suites, and with any Test Scripts it is dependent on.
Ensuring that the Test Suite makes reasonably efficient use of the available resources
d) Phase: Transition
In transition phase, all the scripts developed previously are verified by grouping them into shell scripts or as a suite and are executed in sequence to verify the functionalities before delivering the scripts.
Test data is exported from Test manager and test logs are also taken from test manager to deliver along with the suite, shell scripts and scripts.
Step 6: IBM Rational ClearQuest
a) Phase: Inception
During this phase Schema Repository and Defect Database are created using Rational ClearQuest Maintenance Tool and Rational ClearQuest Designer respectively.
Activities:
Establishing the change request process.
Defining change and review notifications.
Fig.7.Showing defects or change requests in Rational ClearQuest
b) Phase: Elaboration
Defects are raised based on the test logs generated by either executing manual or automation scripts.
c) Phase: Construction & Transition
Rational ClearQuest is also used exclusively in this phase because most of the requirement can only be tested only after implementation. In this release manual and automation scripts are executed to uncover defect in the application and they are logged in ClearQuest for tracking and closure. Once all the defects are raised in ClearQuest, Development team undertakes all valid defects and after fixing delivers a new build or iterations. If the defects rose in ClearQuest is a change request which is enhancement or change in requirement, automation script need to be modified accordingly.
d) Phase: Transition
Reports are generated to find all the change requests still not being resolved.
Depending upon the priority and severity of the change request, appropriate decision is taken to go for release or to go for one more iteration. If team wants to go for release then all the defects should be moved to different buckets to have a track like deferred or waived. ClearQuest changes are also integrated with RequisitePro requirements.
What are the top things you should do?
• Get an expert in RUP & automation consultants work together. A RUP expert can provide guidance in the usage of RUP in different phases of software testing.
• Integrate all the tools to get the maximum benefits: Make end to end implementation of all the rational tools from RequisitePro to ClearQuest with the help of Rational Administrator to derive the maximum benefits and cost cutting in long run.
• Usage of Right tool at the Right time: It’s very important to understand the significance of using the right tool at the right time (RUP phase) to avoid rework and maintain all the information right in place.
• Adopt RUP best practices and be guided by the spirit of RUP. For success, you need to be guided by the spirit of RUP and you want to follow RUP best practices for the best possible outcome.
• Generate different Metrics: Using this approach you can get different reports (statistics) for development and test managers using Rational Test Manager, ClearQuest and Rational RequisitePro, which can help to determine the correct progress of the project and the state of the artifacts at any point of time.
Conclusion
In conclusion, I hope that this article will spur "inclusive thinking" among software testing organizations, encouraging them to practice RUP with respect into Software testing right from the beginning -- rather than treating them as an afterthought late in the development lifecycle. We also hope that tool vendors and third-party suppliers for RUP and other development frameworks will begin embedding an integrated approach for accessibility into their products.
Testing team should be aware of the usage of different rational testing tools in all phases of RUP. If the project is following RUP where requirements are delivered in multiple phases and multiple iterations are expected, using these tools efficiently and effectively can lead to smooth and successful completion of project. This paper can be treated as a guideline document by testing teams working in a RUP project to know what to do and when to do in different phases of RUP.
For all software development organizations, we believe that the approach we describe in this article will ultimately increase productivity and application quality. In making products that are accessible and practical for all potential users, progressive organizations can take advantage of new market opportunities and gain a strong competitive edge.
Reference
Rational Unified Process
Bach, James. 1996. “Test Automation Snake Oil.” Windows Technical Journal, (October): 40-44.
http://www.satisfice.com/articles/test_automation_snake_oil.pdf
Marick, Brian. 1998. “When Should a Test Be Automated?”. http://www.testing.com/writings/automate.pdf
Kaner, Cem. 1997. “Improving the Maintainability of Automated Test Suites. http://www.kaner.com/lawst1.htm
Dustin, Elfriede. 1999. “Lessons in Test Automation.” Software Testing and Quality Engineering (September): 16-22.
http://www.stickyminds.com/sitewide.asp?ObjectId=1802&ObjectType=ART&Function=edetail
Fewster, Mark and Dorothy Graham. 1999. Software Test Automation, Addison-Wesley.
Groder, Chip. “Building Maintainable GUI Tests” in [Fewster 1999].
Hendrickson, Elisabeth. 1999. “Making the Right Choice: The Features you Need in a GUI Test Automation Tool.” Software Testing and Quality Engineering Magazine (May): 21-25.
http://www.qualitytree.com/feature/mtrc.pdf
