Aglie Performance Testing
Abstract
Increase in competition and leaps in technology have forced companies to adopt innovative approaches to have a system which performs well. Today performance testing happens just before product going live with very short timeframe allocated for it which leads to missing performance problems. With small extra efforts, efficiency of performance testing can be increased significantly and these extra efforts usually pay off multi-fold, which can be achieved using Agile Performance Testing.
This paper addresses performance testing using agile methodology, its approach, advantage, risk mitigation and many others.
1.0 Today’s Market need
Increase in competition and leaps in technology have forced companies to adopt innovative approaches to have an application which performs well. Having more features and functionality may not necessarily translate into success for a system - what ultimately count is how stable the consumer finds these features.
Thus performance testing is the testing conducted to evaluate the performance of a system with specified performance requirements.
Today performance testing happens just before product going live with very short timeframe allocated for it. This may lead to missing performance problems and propagate to production and have unhappy customer.
To have customer satisfaction in regards to performance, one can have Agile Performance Testing which states: -
With small extra efforts, making the process more agile, efficiency of performance testing increases significantly – and these extra efforts usually pay off multi-fold even before the end of performance testing.
2.0 Current Performance Testing
2.1 Objective
§ To validate/determine the performance behaviour of the system with respect to the end-user experience.
2.2 Approach
2.3 Risks
§ When major issues are found at the end of test life cycle, it becomes much more expensive to resolve them
§ Major performance issues identified during later stages may hold up important releases and delay the project delivery
2.4 Drawbacks
§ Performance tuning is applied to server-configurable performance parameters (Hardware/Software). Thus, scalability issues are addressed by expensive hardware/software resizing techniques
§ If code optimization is done, then one needs to go through the complete Testing Lifecycle.
3.0 Agile Performance Testing
3.1 Objective
§ To determine the performance bugs/bottlenecks in key business processes as early as possible during the project development cycle
§ Identify, isolate and fix performance bottlenecks at the component/code level
3.2 Approach
Agile Performance Testing is a three phases approach: -
§ Unit Level Tuning – For optimizing
» Execute tests to isolate and fix bottlenecks at code level
§ Component Level Tuning – For testing Components
§ Execute tests to isolate and fix bottlenecks in application components level.
§ Application Load Tuning – For testing Application Flows
§ Test the critical application flows for user-experience under normal and peak loads
Let’s have detail look at each of the phase: -
Unit Level Tuning
Component Level Tuning
Application Load Tuning
3.3 Benefits
§ Iterative performance testing leads to better code, optimized for performance
§ Early detection of performance bugs/bottlenecks reduces the effort & duration for performance tuning and retesting of application
4.0 Effort Metrics for a Release
What will be the efforts required to incorporate Agile Performance Testing?
To evaluate this, effort required is calculated based on following criteria during a release: -
§ With No Change in requirement
§ With Change in requirement
4.1 With No Change in Requirement
Testing Lifecycle
|
Current Approach
|
Agile PT Approach
|
Unit Testing
|
10
|
10
|
Unit Level Performance Tuning
|
0
|
2
|
Integration/Functional Testing
|
3
|
3
|
Component Level Tuning
|
0
|
2
|
UAT Testing
|
4
|
4
|
Application Load Testing
|
2
|
(Script Re-Used)1
|
Tuning Performance bottleneck at Code Level.
|
1
|
0
|
Tuning Performance bottleneck at Component Level.
|
0
|
1
|
Unit Testing after Tuning
|
2
|
Not Required, because Code Level Tuning has been already done in step2
|
Integration/Functional Testing after Tuning
|
1
| |
UAT Testing after Tuning
|
2
| |
Application Load Testing after Tuning
|
(Script needs to re-create due to code tuning)2
|
(Script Re-used)1
|
Total Efforts Required (Man Day)
|
28
|
24
|
Saving in Efforts = 28 – 24 = 4 Man Days = 14% (approx.) for one module
4.2 With Change in Requirement
When change in requirement is invoked, efforts required for testing lifecycle is less as compared to the efforts for first time testing of an application. And also Unit level tuning for CR depends on how changes made impact the working of application.
Example: -
Considering 2 CR’s are made, there are following 3 scenarios for effort saving: -
Scenario 1:
» Performance Testing (Unit Level Tuning) is performed in DEV Environment considering major changes impacting the application for both the CRs.
Scenario 2:
» Performance Testing (Unit Level Tuning) is not performed in DEV Environment considering minor changes not impacting the application for both the CRs.
Scenario 3:
» CR1 - Performance Testing is performed in DEV Environment considering major changes impacting the application.
» CR2 - Performance Testing is not performed in DEV Environment considering minor changes not impacting the application.
Note: Value may vary based on the data used for calculation of effort saving.
5.0 Risk Mitigation
Agile Performance Testing has positive outcome in risk mitigation with respect to: -
§ Customer Satisfaction
Customers/users have higher expectation about the quality of application.
§ Code Tuning
Unit Level tuning is done early so no need for code optimization at later stage on Testing Lifecycle.
§ Application Rejection
Reduces risk of the application being rejected because of issues such as Memory Leaks, Hard Coded Values in code, Database Locking
§ Release Dates
Iterative Testing leads to constant awareness of the application performance and therefore more confidence in meeting release dates.
§ Performance Bugs
Less Number of Performance bug(s) in subsequent phase(s).
§ Performance Testing Efforts
Performance Script can be re-used thus saving 60% of the efforts of Performance Cycle.
6.0 Advantage
One Good decision at Start can cover a lot
§ Earlier detection of performance bottlenecks
» Reduces the effort and duration for performance tuning and retesting
» Project management has constant awareness of the application performance and therefore more confidence in meeting release dates.
§ Faster resolution of identified issues
» Bottom-up approach to identify bottlenecks helps narrowing down the root-cause faster
§ Better performance tuning
» Development team gets more time for performance tuning activities
§ Cost Effective
» Identified bottlenecks are resolved much earlier in Testing Lifecycle saving expenses required to a great extend.
7.0 Pre-requisite, Challenges and Risk
§ Pre-requisites
» Full time Performance Tester.
» Must be technically sound.
» Closer interactions between developers & testers
» Access to monitor all the servers.
§ Challenges
» Defining accurate performance requirements.
§ Risks
» Incorrect performance requirements may jeopardise the testing
8.0 Conclusion
Applications are backbone of business; hence performance of application becomes one of the key differentiators in success or failure of business. Thus performance testing is conducted to evaluate the performance of a system with specified performance requirements.
In current approach, performance testing happens just before application going live with very short timeframe allocated for it. When major issues are found at the end of test life cycle, it becomes much more expensive to resolve them which may hold up important releases.
This calls for the solution which states - “Investigate performance early and Validate performance last”
Agile Performance Testing does just that. With Agile Performance Testing, Project management is constantly aware of the application performance and therefore more confidence in meeting release dates.
Agile Performance Testing has positive outcome in risk mitigation because systems are getting more and more complex with a dynamic technology landscape and high standards of customer satisfaction
No comments:
Post a Comment