July 2012 – Continuous Testing and DevOps Blog

July 25, 2012August 1, 2012

Cross Platform mobile development tool – Titanium

All

This is a Testing blog however i recently get a lot of questions around Mobile development/porting for cross platform projects (Android/iOS etc.).

For this cases there are several good and free tools out there which can be used such as PhoneGap (http://phonegap.com/download/), EggPlant (http://www.testplant.com/products/eggplant/for-cross-platform-testing/) and Titanium.

In this short post i will just point you out to the tool and give a very high level details about it for you guys to go and try it out for yourself

Titanium is a free tool by Appcelerator (http://www.appcelerator.com/platform/titanium-sdk) which allows quite quickly to develop a cross platform mobile application which can be than deployed on iOS and Android Phones/Emulators and also PC Web browsers.

The tool allows to develop in Java Script and customize your resources pending the operating system you which to deploy on:

You need to have on your machine the Android SDK and point its location in the Titanium SDK to allow execution and debugging on the Android Emulator.

You have through the SDK options to create new project from samples or templates (HTML5, Tabbed APP and more).

Feel free to give it a try and comment

Regards,

Eran Kinsbruner

July 12, 2012

Mobile project planning

Up until now i covered in the various posts the aspects of Mobile testing, trends, tools for automation, mobile platforms etc.

All of the above does not help much if the mobile project is not well planned in advanced with SMART milestones, efficient choices of devices (lead/gold devices), right tools and Agile ALM (Application life-cycle management).

In this short post i will advise the most important project milestones and how to define them.

First milestone:

– Product definition and feature list –> In this milestone the product management ought to chose the supported platforms and devices for the project (based on thorough market research world wide and based on the projected release date which should be relevant for the selected devices), the project scope (the feature set should be well defined, the supported languages/localizations should be defined, the required certifications (Google CTS, Google Market, Apple App Store certification and more) should be clear, the target customers and of course the projected release date.

Not to forget the UI/UE aspect which is a critical aspect in any mobile project (Not only because the OS requirements but also for the end users).

Second Milestone:

– Core team build up and preliminary work plan implementation, staffing etc.

– Purchasing/ordering of equipment as needed, tools for development and automation should be brought up for approval

– High level Test Plan and Product specification should be developed during that milestone and approve by the core team

Third Milestone:

– Product development

– Unit test development

– Test development on the lead/gold devices per platform (Manual and Automated)

– Documentation creation (As needed and as defined by product)

– Sanity testing on supported devices, emulators, and cloud based handsets.

fourth Milestone:

– Feature complete

– Test development complete

– Full cycle execution on lead devices

– Porting kick off –> Building of subset devices (Either from the lead devices family or new families) versions for testing (Only sanity testing is required for devices within the lead devices family, while full cycles on new family members)

** Family can be defined by a group of devices from the same OEM with the same OS and Screen resolution (e.g. Samsung Galaxy S and Samsung galaxy S Plus are related to the same family –> http://www.gsmarena.com/samsung_i9001_galaxy_s_plus-3908.php ; http://www.gsmarena.com/samsung_i9000_galaxy_s-3115.php).

Fifth Milestone:

– In this stage the project should be well tested, debugged, ported to at least several devices and families, as well as documentation should be ready, certification status should be GREEN, and the bug trends for the project should also be minimal, quality criteria is mostly met.

– Project release to the various markets for approval is being done

– Based on approval and if needed pre-customer approval the project is released!!

The above is just in a nutshell list of items which i feel are a must in a common project plan for a mobile project, i might have missed few items, or the order can be different in few companies, but the above should bring you to a safe release if followed properly.

Thanks,

Eran Kinsbruner

July 9, 2012July 9, 2012

Free iOS automation testing tool – Test Studio for iOS

I will not write too much in this post, just wanted to provide a link for a free mobile automation tool for iOS web/GUI applications which is available in the App Store

http://www.telerik.com/automated-testing-tools/ios-testing.aspx?utm_source=twitter&utm_medium=sm&utm_content=FRDTNCKNWJ&utm_campaign=ad

The tool provides a simple record/replay mechanism on the iOS native platform and fits basic and not too complex web and native applications.

Soon it should allow sync of the test results and more in the cloud.

Test Studio for iOS does not use image based element detection. It relies on object based recording instead.

I recommend to give it a try

Regards,

Eran Kinsbruner

July 8, 2012

Integrating a new Android AVD (Emulator) into Android SDK

In this post i would simply show the quick and simple steps on how to add a new OEM Emulator into your Android SDK and run a sample Android application (APK) on it or on a specific Emulator.

The assumption for this post is that you’ve already downloaded and installed on your Windows/Linux machine the latest Android SDK.
If you have not yet done so, you may download the SDK from: http://developer.android.com/sdk/index.html

** Please note that the SDK requires a JDK (Java Development Kit) to be installed on your PC.

Once you have the SDK installed, you will have a basic skin/AVD (Android Virtual Device) to work with, which comes with the SDK.

As application developer/tester you know that 1 emulator will not give you enough confidence about your application functionality and behavior (without getting into the debate of the use of real device – which i tend to recommend to start ASAP and in the worse case, no later than Beta milestone), so you need to get as much coverage as possible using as many emulators as possible.

Most OEM’s develop and provide for free many emulators which you can simply add to your sdk (We have Samsung, Motorola, LG and HTC as well as Sony Ericsson who provide Tablets and Smartphones emulators which you can download and deploy in the SDK).

From your SDK installation path, simply paste the ZIP with the AVD files into the folder:

{SDK_HOME}/platforms/android-xx/skins –> When android-xx stands for the version of Android on which you wish you run your application on (e.g. – Android-10 stands for Android 2.3.3 Ginger Bread, while android-15 stands for Android ICS 4.0.3).

See example of a folder into which i created few simulators (or AVD’s):

The problems which people often ran into or ask about are, what is the method to install and run an application through the ADB command line on a specific emulator while i have 2 or more running?

The answer is simple:

From the command line assuming you have few AVD’s running, run the command “adb devices”

You will get an output in the below form:

“C:\android-sdk_r18-windows\android-sdk-windows\platform-tools>adb devices
List of devices attached
emulator-5560 device
emulator-5564 device”

If you than want to install an APK on 1 of the above emulators (e.g. emulator-5560), simply run this command:

“adb -s emulator-5560 install “demo.apk”

That’s it

I know that it is a quick and short post, if you have any problems or questions about the above – feel free to ask

Regards,

Eran Kinsbruner

July 5, 2012July 5, 2012

JAMO Solutions – Mobile test automation tool

Hi,

I am sure that most mobile testers heard about JAMO solutions, but if not than i will briefly introduce this solution.

In past blogs i mentioned few powerful automation tools such as PerfectoMobile, Keynote Device anywhere, SeeTest by ExperiTest and FoneMonkey (MonkeyTalk).

Except of the Monkey Talk – all others are commercial tools and not free.

Jamo is also a commercial and not free solutions which comes in various licensing models from a company based in Belgium (http://www.jamosolutions.com/editions.html).

The solutions is based on a plug in to various of IDE’s such as QTP (HP), Eclipse, and Visual Studio, which than allows connectivity over USB or WiFi through a Device manager to the mobile platform (iOS, Android, BlackBerry and Windows Mobile) on which an agent is also installed to provide the full control and automation via record/replay, easy to input text utils etc.

For Android OS device automation using eclipse, you can see a short demo of the tool and its methodology:

In the same manner as the video above users can automate their scenario’s through QTP or Visual studio for the various OS platforms.

For more information or inquiries please contact Jamo through their web site:

http://www.jamosolutions.com/home.html

Regards,

Eran

July 3, 2012

Mobile testing using Benchmarks

Hi,

We know that many devices vary between each other in their HW capabilities, meaning – the fact that 2 devices by 2 different or similar OEM’s, running the same OS/different OS version will score differently in a benchmark run or a performance test.

This is usually due to hardware differences, defects related to memory allocation, and others.

The use of mobile benchmarks had become quite significant and useful and can allow the application developers and testers to do objective testing on many different devices using a benchmark tool and get unique scores.

By comparing the overall score among the devices under test they can reach a conclusion which device has the higher performance and decide to either use it as “Gold” device for its porting platform, or if the score is extremely low, use it for testing when that device serves as the low performer in the list allowing the team to gain some prerequisite information on how their application would run on the device/s.

The benchmarks are usually downloaded for free from the Android or App store markets, and allows at the end of the quick test run to compare the device score with other devices which ran in the past and see the overall position of the device under test in compare to the market.

Usually the tests involve mathematical calculations (such as finding the prime number from a list), string sorting or number sorting, 2D/3D gaming with FPS calculations, reading and writing to a memory of the device (SD Card etc.), and some other CPU tests.

I strongly recommend to use such tools as part of the testing and as part of the decision for porting a new/existing device.

Information on commonly used benchmarks is available in the web, however i have good experience with the following ones:

– For Android – PassMark and Antutu (Which also provides Battery saver tool and CPU master to control the CPU speed)

http://www.androidbenchmark.net/passmark_chart.html

http://www.antutulabs.com/AnTuTu-Benchmark

– For iOS – PassMark and iBench

http://itunes.apple.com/us/app/ibench/id385703033?mt=8

Good Luck

Regards,

Eran Kinsbruner

July 1, 2012July 1, 2012

Latest Mobile world news (July 2012)

Hi,

I’ve collected few topics which are “hot” these days in the mobile world and which i find interesting and relevant to the global audience.

Apple states that NFC support (Near Field communication) might be included in the next iPhone 5 device

http://www.businessinsider.com/iphone-5-may-include-nfc-chip-for-mobile-payments-2012-6

RIM (research in motion) is considering selling its business to Facebook or Amazon (Treat this as a rumor rather than a fact)

http://www.businessinsider.com/rim-considers-selling-its-handset-business-2012-6

Time spent on mobile apps vs. web browsing is growing year by year – see below latest trends (Minutes per day):

Microsoft announces on the new Windows 8 OS with Internet Explorer 10 release with new features, enhancements and more, as well as new Metro customizable UI