Mobile apps are here to stay but the challenges to test these remain the same. We have already gone through the list of challenges when testing a mobile app — OS and device fragmentation, form factor variations, bandwidth issues, network speeds, recording user actions and load requirements which are varied in mobile. Testing web apps is equally challenging, since there are other set of factors like browser compatibility, form factors, geo location features etc. specific to the web that need to be considered.
No matter what the type of app is, native, hybrid or web based, it has to go through a very rigorous performance testing cycle to ensure that it can perform, under all conditions.
There are also some major performance application issues on the server and client side which need to be addressed. Key issue — mobile application testing for performance testing is NOT the same as traditional performance testing. Mobile is a whole new game and the testing requirements for all attributes are much steeper due to the unpredictability.
On the server side we need to understand variations in response times, streaming resource intensive packets, delays in delivery of messages, application crashes etc. On the client side — we need to address the usual discrepancy of application behavior on various platforms and handsets, memory and CPU consumption, loading speed and battery issues.
Let’s first clear some of the definitions that we often come across in the performance testing world. It’s important to understand the key differences of these terms. It’s mostly used interchangeably and considered to be same, but actually it’s not as their goals are different.
| Testing Type | Key Goals |
| Performance Testing | · Establish a smooth, controlled, stable environment with a clear defined set of expectations (e.g. expected load, acceptable response time & throughput & record repeatable measurements · Eliminate required bottlenecks, while increasing load & establish a baseline for various load levels |
| Load Testing | · Increase load constantly via automated tools to operate at predefined level & hence it’s part of the process of Performance testing [Incremental or Sequential] · Aim to break the software, but keep the software working at the highest load possible on peak hours
· Rectify bugs identified to ensure performance testing baseline is met |
| Stress Testing | · Test the software under unpredictable scenarios like increasing load three times, sudden failure of DB connectivity etc. and try to crash the software or see how software reacts under unexpected failures/scenarios. |
There are some tools in the market that can help you with the performance/load, and stress testing conundrum in the mobile and web based apps space.
- Recent announcement by MicroFocus about SilkPerformer 9.0 (2012) supports Mobile Web & Native apps testing on iOS, Android & BB platforms
- HP LoadRunner supports mobile web & native apps testing on iOS, Android, BB & WM platforms
- NeoLoad supports mobile web & native mobile apps
- JMeter supports mobile web app testing on simulators
- IBM Rational Performance Tester (RPT) supports mobile web app testing as well
But no matter what your mobile app is like, you need a strong testing team behind you to first plan the detailed diagnostics and determine the capacity. Then, bring the know-how to run the tests by selecting the right load generation and monitoring tools to make sure that your app can handle the worst, at all times.
