Mobile testing using Benchmarks

Hi,

We know that many devices vary between each other in their HW capabilities, meaning – the fact that 2 devices by 2 different or similar OEM’s, running the same OS/different OS version will score differently in a benchmark run or a performance test.

This is usually due to hardware differences, defects related to memory allocation, and others.

The use of mobile benchmarks had become quite significant and useful and can allow the application developers and testers to do objective testing on many different devices using a benchmark tool and get unique scores.

By comparing the overall score among the devices under test they can reach a conclusion which device has the higher performance and decide to either use it as “Gold” device for its porting platform, or if the score is extremely low, use it for testing when that device serves as the low performer in the list allowing the team to gain some prerequisite information on how their application would run on the device/s.

The benchmarks are usually downloaded for free from the Android or App store markets, and allows at the end of the quick test run to compare the device score with other devices which ran in the past and see the overall position of the device under test in compare to the market.

Usually the tests involve mathematical calculations (such as finding the prime number from a list), string sorting or number sorting, 2D/3D gaming with FPS calculations, reading and writing to a memory of the device (SD Card etc.), and some other CPU tests.

I strongly recommend to use such tools as part of the testing and as part of the decision for porting a new/existing device.

Information on commonly used benchmarks is available in the web, however i have good experience with the following ones:

– For Android – PassMark and Antutu (Which also provides Battery saver tool and CPU master to control the CPU speed)

http://www.androidbenchmark.net/passmark_chart.html

http://www.antutulabs.com/AnTuTu-Benchmark

 

– For iOS – PassMark and iBench

http://itunes.apple.com/us/app/ibench/id385703033?mt=8

Good Luck

Regards,

Eran Kinsbruner

Latest Mobile world news (July 2012)

Hi,

I’ve collected few topics which are “hot” these days in the mobile world and which i find interesting and relevant to the global audience.

  • Apple states that NFC support  (Near Field communication) might be included in the next iPhone 5 device

http://www.businessinsider.com/iphone-5-may-include-nfc-chip-for-mobile-payments-2012-6

  • RIM (research in motion) is considering selling its business to Facebook or Amazon (Treat this as a rumor rather than a fact)

http://www.businessinsider.com/rim-considers-selling-its-handset-business-2012-6

  • Time spent on mobile apps vs. web browsing is growing year by year – see below latest trends (Minutes per day):

  • Microsoft announces on the new Windows 8 OS with Internet Explorer 10 release with new features, enhancements and more, as well as new Metro customizable UI

http://windows.microsoft.com/en-US/windows-8/release-preview

  • Check out the list of 20 best mobile phones in the world according to techradar site:

http://www.techradar.com/news/phone-and-communications/mobile-phones/20-best-mobile-phones-in-the-world-today-645440?artc_pg=22

Enjoy 🙂

Regards,

Eran Kinsbruner

Automating mobile tests with SeeTest by Experitest

Hi,

In this short post i will introduce the SeeTest mobile automation testing tool which penetrates the mobile land and provides a very good automation and LAN Cloud solution for iOS/Android and other mobile OS’s.

The tool can be downloaded for short trial via the main web site: http://experitest.com 

The ramping up and connection of a device or simulator is quite easy, see short video which demonstrates the basic steps to get you up and running:

The tool provides built in high level functions (Actions like ‘launch browser’, ‘send url’ etc.) which ease the test development and automation life cycle.

The automation is highly reliable, and has various methods of object recognition and recording, as well as object editing for more precise operations (both OCR, Image and Native object recognition and analysis).

The tool has simple integration into the most common IDE’s (QTP, Jsystem, python, and others), and allows to deploy the initial code which was recorded into that IDE.

The tool has a mobile cloud solution which is unique, and provides remote access from within the local area network to any device which is connected, and by such allows running various scripts on multiple handsets/simulators as part of a regression test cycle or a continuous integration process etc.

For more info regarding pricing model or technical details, either contact me or the SeeTest support through the web

Good Luck

Eran Kinsbruner