Deciding whether to automate or emulate testing for iOS 8

Test automation has many fans and is gaining momentum in mobile app testing. The tools have changed but the central question remains: Is using either approach effective enough to significantly improve mobile application testing.
customer experience testing melbourne

Contact Us We cut software testing from weeks to days. Let’s talk for 15 minutes to see if we can accelerate your digital delivery too. Schedule a call with our CEO Ash Conway.

Simulate testing with emulators

iOS Emulators provide a simulated test environment without the overhead of purchasing and managing multiple devices and you’ve likely been using them for several years, but are they effective enough to be useful? Do they find defects that affect the end users experience?

Emulators are useful for testing application behavior that is not device or carrier specific early in the development cycle. However, their effectiveness for testing beyond the basics is suspect. Relying too much on emulators for testing is risky. It’s difficult to execute all the testing types needed for a new iOS release. Consider doing a risk analysis on your testing types - basic emulation (smoke testing), performance, functional, load and then decide on the method used for each.  

Automating functional testing

The advantage of automated testing tools is speed and regression testing repeatability.

A few disadvantages are their dependency on the platform so duplicate tests may be needed for each device or platform type. Additionally, automated tests need tuning to get timings correct. Most importantly, automated test maintenance is significant when application code changes frequently. The maintenance effort involved is the leading killer of automated testing projects. 

Another consideration is ensuring automated tests have valid verification points included.  Otherwise, you may be running the exact same test multiple times which wastes valuable time and fails to discover defects.

Automated testing is stronger and less frail when built at the unit testing level rather than the GUI level. GUI level automated tests frequently produce false failures when a software application changes frequently as they require constant maintenance.  If your mobile apps change frequently, plan on how to handle test maintenance. Both emulators and automated tests are useful as complements to manual and device testing, but not as a total replacements even when testing time is limited.

Bug Newsletter

Keep up to date with the latest in beta, bug & product testing.

User acceptance testing tools & teams

Quickly assess the quality of your software, websites & mobile apps