san jose selenium meetup 22 mar 2012: the restless are getting native

The Restless Are Getting Native

Lessons Learnt While Automating an iOS App

Dante BrionesPrincipal Consultant

Cochiva

Friday, March 23, 12

What makes a good automation tool?

Friday, March 23, 12

I want tests that:

• are robust

• are easy to maintain

• clearly express intent

• are written in a real programming language

• run in any environment

• run on simulator or real device

Friday, March 23, 12

These are important, but...

Friday, March 23, 12

What’s the point of automated tests?

• To catch regression bugs

• To do so faster and more reliably than a human

Friday, March 23, 12

Developer

CommitSource Repo

Production

Build & Deploy

UserUses

Friday, March 23, 12

Developer

CommitSource Repo

Production

Build & Deploy

UserUses

Test Environment

TesterFeedback

Tests

Build & Deploy

Friday, March 23, 12

Developer

CommitSource Repo

Production

Build & Deploy

UserUses

Test Environment

TesterFeedback

Tests

Build & Deploy

Continuous Integration

Environment

Build & Deploy

Feedback

Friday, March 23, 12

Developer

CommitSource Repo

Production

Build & Deploy

UserUses

Test Environment

TesterFeedback

Tests

Build & Deploy

Continuous Integration

Environment

Build & Deploy

Feedback

Feedback

Local Test Environment

Tests

Friday, March 23, 12

What’s the biggest win of automated tests?

FASTER FEEDBACK

Friday, March 23, 12

Development Team Automation Team

Tests

Test Environment

Deploy Run

Feedback

Friday, March 23, 12

Friday, March 23, 12

Development / Automation Team

Tests

Test Environment

DeployRun

Feedback

Friday, March 23, 12

Something to consider

"If a test fails in Jenkins,

and no one is there who cares about it,

should it still break the build?"

Friday, March 23, 12

</digression>

Friday, March 23, 12

iOS Automation: Tools

• NativeDriver http://nativedriver.googlecode.com/

• KIF https://github.com/square/KIF

• Frank http://www.testingwithfrank.com/

• MonkeyTalk http://www.gorillalogic.com/

• LessPainful https://www.lesspainful.com/

• iCuke https://github.com/unboxed/icuke

• UISpec http://uispec.googlecode.com/

Friday, March 23, 12

iOS Automation: Architecture

Device / Simulator

Application

HTTP Server

Test Code

Request / Response

Synthesized Events

StateQuery

Friday, March 23, 12

iOS Automation: Architecture

Device / Simulator

Application

Test Code

Friday, March 23, 12

Why NativeDriver?

Familiar API

Friday, March 23, 12

Why NativeDriver?

Java

Friday, March 23, 12

Why NativeDriver?

Android

Friday, March 23, 12

Why NativeDriver?

“Hybrid” Apps

Friday, March 23, 12

iOS Automation Toolchain

Device / Simulator

Application (Obj-C)

HTTP Server

Test Code (Java)

WebDriver Protocol

Synthesized Events

StateQuery

Friday, March 23, 12

NativeDriver Test Support Code

Friday, March 23, 12

NativeDriver Test Code

Friday, March 23, 12

Toolchain: The ugly bits

• How do I build from the command line?

• How do I launch the simulator from the command line?

• How do I shut down the simulator cleanly?

Friday, March 23, 12

Toolchain: The ugly bits

• How do I browse native widgets currently onscreen?

• How do I browse widgets in a UIWebView?

• How do I solve intermittent test failures?

• What do I do if NativeDriver doesn’t support an element I need to interact with?

Friday, March 23, 12

Challenges

Friday, March 23, 12

Cultural Differences

Football Football

Friday, March 23, 12

Web Test Automation

• Selenium, Watir, Capybara, HTMLUnit, etc.

• Test automation is commonplace

Friday, March 23, 12

iOS Test Automation

Real Developers Don’t Write Tests

Friday, March 23, 12

Challenges:Language Choice

Friday, March 23, 12

Recommendations

Friday, March 23, 12

Recommendation:Show Value Early

Friday, March 23, 12

Recommendation: Surprise and Delight

Friday, March 23, 12

Friday, March 23, 12

Friday, March 23, 12

Recommendation: Smooth Rough Edges

Friday, March 23, 12

Recommendation: Know Your Limits

Friday, March 23, 12

The Future of NativeDriver

Friday, March 23, 12

Friday, March 23, 12

The Future of Mobile Testing...

...is in your hands!

Friday, March 23, 12

Thank you!

Dante Brionesdante@cochiva.com

@dantebriones

Friday, March 23, 12