OK, past someone else's document
Keep It Functional – iPhone Test Automation
How we implement automated tests for the native iPhone App by Daniel Knott
Currently all the XING Mobile Apps generate more than 20% of the overall XING traffic and this number will likely increase in the near future. In my last devblog post, I dealed with Android Apps and how to implement automated tests with the framework Robotium. This article describes how to implement automated tests using the tool KIF (Keep It Functional).
KIF is an open source test framework developed by the company square. It is an iOS integration test framework that allows you to implement test cases with objective C that can be executed, currently only, against the iPhone/ iPad simulator. The KIF tests are integrated to your Xcode workspace, there is no need for additional servers or services that must be started.
KIF is like a Grey Box Test Tool because you need knowledge of the structure of the App you want to test. KIF use the provided accessibility labels from iOS to interact with the UI elements in the App. The labels are normally used to make Apps available for people with visual disabilities.
With KIF you are able to simulate real user interaction like tap, click, drag, enter text and many more. Also you are able to verify elements on the screen. It is really fast in executing the test cases on the simulator. Also it is really easy to integrate to your xCode and Continuous Integration environment.
KIF is like a Grey Box Test Tool because you need knowledge of the structure of the App you want to test. KIF use the provided accessibility labels from iOS to interact with the UI elements in the App. The labels are normally used to make Apps available for people with visual disabilities.
With KIF you are able to simulate real user interaction like tap, click, drag, enter text and many more. Also you are able to verify elements on the screen. It is really fast in executing the test cases on the simulator. Also it is really easy to integrate to your xCode and Continuous Integration environment.
Don’t submit your tests to Apple
It is really important that the KIF tests are not part of your production code, otherwise Apple will deny your submission because of undocumented APIs. Why this?
KIF uses these undocumented Apple APIs, to interact with the App, but that is not a problem when it comes to developing the tests. Most of the available iOS test frameworks use these undocumented Apple APIs.
KIF uses these undocumented Apple APIs, to interact with the App, but that is not a problem when it comes to developing the tests. Most of the available iOS test frameworks use these undocumented Apple APIs.
Installation
To install and configure KIF please follow the provided information on the github page. These instructions are always up to date to the latest version of KIF.
Accessibility Inspector
If your project is configured with KIF you have also to activate the “Accessibility Inspector” on the iPhone or iPad simulator to have access to the labels. Do the following steps on the simulator:
- Open the Settings
- Click on General
- Click on Accessibility
- And activate the Inspector
Afterwards you see the inspector with a colorful layer within your simulator. You can drag the layer wherever you want. Clicking the small (x) on the left side opens the inspector. If you now do a single tap on an app you get information about the accessibility label and some more information (Screenshot). If you do a double click you can e.g. enter the app. If you close the inspector layer with the (x) again, you have again single click on the simulator.
NOTE: The inspector must be activated to use KIF for testing. Now you can start implementing your test classes.
Code Example
KIF has three main classes that can be used to implement tests within your project. There is a test runner the
KIFTestController, a scenario the KIFTestScenario and a step the KIFTestStep. The KIFTestController is like a test suite where all the test scenarios are listed that should be executed. A KIFTestScenario is like a container that includes several steps that should be done within a test. A step is the smallest and simplest action that represents user interaction like tap, enter text or wait for views.
The following listings show the Controller h-file and m-file. The m-file includes the list of possible scenarios.
XINGTestController.h
#import #import "KIFTestController.h" @interface XINGTestController : KIFTestController {} @end
XINGTestController.m
#import "XINGTestController.h" @implementation XINGTestController - (void)initializeScenarios { [self addScenario:[KIFTestScenario scenarioLoginWithWrongCredentials]]; // Add here additional scenarios } @end
Now you need to implement the scenario scenarioLoginWithWrongCredentials. You need again an h-file and an m-file for that. The h-file includes the scenario definitions. The m-file is the class where the test methods are implemented.
KIFTestScenario+XINGLogin.h
#import #import "KIFTestScenario.h" @interface KIFTestScenario (Login) + (id) scenarioLoginWithWrongCredentials; @end
KIFTestScenario+Login.m
#import "KIFTestScenario+Login.h" #import "KIFTestStep.h" @implementation KIFTestScenario (Login) + (id)scenarioLoginWithWrongCredentials{ KIFTestScenario *scenario = [KIFTestScenario scenarioWithDescription:@"Wrong credentials"]; // enter wrong credentials in the username and password field [scenario addStep:[KIFTestStep stepToEnterText:@"wrongusername" intoViewWithAccessibilityLabel:@"Username"]]; [scenario addStep:[KIFTestStep stepToEnterText:@"wrongpassword" intoViewWithAccessibilityLabel:@"password"]]; // click the done button on the keyboard [scenario addStep:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"done"]]; // verify that the error message is shown with a localized string [scenario addStep:[KIFTestStep stepToWaitForViewWithAccessibilityLabel:LocalizedString (@"FAILED_MESSAGE")]]; // click the button on the error message [scenario addStep:[KIFTestStep stepToTapViewWithAccessibilityLabel:LocalizedString (@"OK_BUTTON")]]; return scenario; } @end
Now you have a really simple test that tries to login with wrong credentials against an app. After pressing the done button on the keyboard KIF verifies with the
stepToWaitForView… method that the right error message is shown. The following screenshots show the login screen where KIF tries to login with the wrong credentials. Please also have a look at the provided screencast videos at the end of this article. You get a really good impression how KIF is working.
Continuous Integration
It is highly recommended to integrate KIF to a CI server like Jenkins to build up a regression testsuite to be sure that the existing functionality is still working after a commit. To integrate and execute your KIF tests from the CI server you must be able to start the simulator from the command line. To do this, you need an additional tool called WaxSim. One Note from the KIF github page: “Note that the Square fork of WaxSim provides a number of bug fixes and some useful additional functionality. Your CI script should resemble something like the this:”
Adapt the following script to your environment and add it to your Jenkins build configuration:
Adapt the following script to your environment and add it to your Jenkins build configuration:
#!/bin/bash killall "iPhone Simulator" set -o errexit set -o verbose # Build the "Integration Tests" target to run in the simulator # xcodebuild -target "Integration Tests" -configuration Release -sdk iphonesimulator build # Run the app we just built in the simulator and send its output to a file # /path/to/XINGApp.app should be the relative or absolute path to the application bundle that was built in the previous step /path/to/waxsim -f "iphone" "/path/to/XINGApp.app" > /tmp/KIF-$$.out 2>&1 # WaxSim hides the return value from the app, so to determine success we search for a "no failures" line grep -q "TESTING FINISHED: 0 failures" /tmp/KIF-$$.out
Store this script as a shell script within your KIF test folder and call the script from the CI server. The iPhone simulator should start automatically and should execute the tests on the simulator.
That is the way, how we at XING use KIF to automate our iPhone App. Besides all the manual testing we use the automated tests to check, that code changes doesn’t affect to current functionality.
We hope that this post gives you an impression on how to build a test automation suite for your iPhone or iPad apps!
We hope that this post gives you an impression on how to build a test automation suite for your iPhone or iPad apps!
Screencasts
The following screencasts show the functionality of KIF. The first video shows the example code mentioned in this blog post. The second video shows how KIF is starting the XING handshake with some testusers on our testsystem.
Login with wrong credentials
Handshake
The latest XING iPhone App is available here.
Further information can be found in this really good google group about KIF.
This post based on sources from:
- http://corner.squareup.com/2011/07/ios-integration-testing.html
- https://github.com/square/KIF
- https://github.com/square/waxsim
Daniel Knott works as a Manager Quality Assurance in XING’s mobile team. He likes to automate test cases using technologies such as Robotium, KIF(Keep It Functional), Selenium and Java.
No comments:
Post a Comment