OpenCV on Android Platforms

Marco Moltisanti Image Processing Lab – http://iplab.dmi.unict.it

moltisanti@dmi.unict.it http://www.dmi.unict.it/~moltisanti

Outline

• Intro

• System setup

• Write and build an Android app

• Write and build an OpenCV app for Android

Marco Moltisanti – OpenCV on Android Platforms

A couple of words about Android / 1

• Android is a Linux-based OS, oriented to mobile devices;

• Android, Inc. was founded in 2003 by Andy Rubin, Rich Miner, Nick Sears and Chris White;

• Google acquired Android, Inc. in 2005; • In 2007, Open Handset Alliance, a consortium

including Google, HW manufacturers and TC operators, unveiled Android as its first product, a mobile device platform.

• First commercial device running Android was HTC Dream, released on October, 2008.

Marco Moltisanti – OpenCV on Android Platforms

A couple of words about Android / 2

• Each major release is named after a dessert: first was Cupcake (1.5), current is Jelly Bean (4.1)

Distribution of Android Releases: http://developer.android.com/about/dashboards/index.html

Marco Moltisanti – OpenCV on Android Platforms

A couple of words about Android / 3

• Applications for Android OS can be acquired through an app store, such as Google Play or the Amazon App Store;

• Moreover, it is possible download and install the app’s APK file from a third party site.

Marco Moltisanti – OpenCV on Android Platforms

Android Apps / 1

• An Android app is packaged and distributed as an Android Package, i.e. a file with apk extension. All the code and data in a single apk file is considered a to be one application.

• Each app lives in a sandbox inside the Linux-based Android OS.

• Each app has a unique ID and runs on its own Linux process.

• Each process has its own Virtual Machine.

Marco Moltisanti – OpenCV on Android Platforms

MyApp

Android Apps / 2

A single screen with a user interface

Run in background

Manage data Listen to system

wide announcements

Marco Moltisanti – OpenCV on Android Platforms

Android Apps: file Manifest

• The system must know the existence of the application’s components.

• A «manifest» file, named AndroidManifest.xml, is needed in order to: – Declare components;

– Identify user permissions (e.g. Internet Access);

– Declare minimum API level;

– Declare needed external lib;

– …

Marco Moltisanti – OpenCV on Android Platforms

Android Apps

• An exhaustive view of Application’s architecture would take too much time…

• Further informations available at

http://developer.android.com/guide/components/index.html

Marco Moltisanti – OpenCV on Android Platforms

Let’s start!

• First of all, we need to setup our work environment.

– Developer’s Kits, Virtual Machines, API, …

Marco Moltisanti – OpenCV on Android Platforms

Ingredients

Java SDK 6 or higher

Eclipse IDE

A C/C++ Compiler

(Visual C++ / MinGW / Cygwin)

Android SDK Android NDK

Marco Moltisanti – OpenCV on Android Platforms

Recipe / 1

• Download and install Java SDK from http://www.oracle.com/technetwork/java/javase/downloads/jdk7u9-downloads-1859576.html

• Let JAVA_FOLDER be the folder where we installed our JDK. Put the «JAVA_FOLDER\bin» directory in the System Path.

Marco Moltisanti – OpenCV on Android Platforms

Recipe / 2

• Download and install Android SDK from http://developer.android.com/sdk/index.html

• Download and unzip Android NDK from http://developer.android.com/tools/sdk/ndk/index.html

Marco Moltisanti – OpenCV on Android Platforms

Recipe / 3

• Download and unzip Eclipse IDE for Java Developers from http://www.eclipse.org/downloads/

• Install the Android Development Tools (ADT) plugin for Eclipse: http://developer.android.com/sdk/installing/installing-adt.html

• Install the C/C++ Development Tooling (CDT) for Eclipse

Marco Moltisanti – OpenCV on Android Platforms

Recipe / 4

• A good quick start guide is available at http://docs.opencv.org/doc/tutorials/introduction/android_binary_package/android_dev_intro.html

Marco Moltisanti – OpenCV on Android Platforms

Android application structure

Jni C/C++ application source code Control the C++ build process

Android.mk and Application.mk Written with Makefile syntax

Libs Compiled libs

Res Resources of the application Images

xml files describing UI layout

Src Java code of the application

AndroidManifest.xml essential information about application to the Android system:

Name of the Application Name of main application’s package Components of the application Required permissions

default.properties text file containing information about target Android platform and other build details

Project root folder

Present if we need C/C++ code

Marco Moltisanti – OpenCV on Android Platforms

Hello World! / 1

• Click on the highlighted button (if you don’t have it, you have to reinstall ADT plugin) to create a new Android App.

Marco Moltisanti – OpenCV on Android Platforms

Hello World! / 2

• Fill the form and click next.

• Go forward to the end of wizard… we don’t care in this moment about icons and stuff!

Marco Moltisanti – OpenCV on Android Platforms

Hello World! / 3

Marco Moltisanti – OpenCV on Android Platforms

Run, Hello World, run! / 1

• To run our HelloWorld! App we have to setup a Virtual Device.

• This can be done using Android Virtual Device Manager, that comes with the SDK.

• Launch AVD Manager and create a new device

Marco Moltisanti – OpenCV on Android Platforms

Run, Hello World, run! / 2

• From Eclipse, click Run (CTRL+F11) and select the Android App modality

• Eclipse will launch the device emulator and run your app.

Marco Moltisanti – OpenCV on Android Platforms

A bit deeper… / 1

• The user interface (UI) of an application is stored in a xml file, which contains each component’s settings.

• The UI can be created modifying the xml or using the tools provided by Eclipse.

• Example!

Marco Moltisanti – OpenCV on Android Platforms

A bit deeper… / 2

• The logic part is in a java file, with the same name of the activity we’re working on.

• In our example, it’s in MainActivity.java

• It is possible to access layout elements using the findViewById method.

• public View findViewById (int id)

• id can be retrieved using R.id.componentName

Marco Moltisanti – OpenCV on Android Platforms

Events Handling

• Nobody likes a static UI!

• Here we use the Java event handling paradigm: Event Listeners!

• Each event has its own listener – onClickListener

– onCheckedChangeListener

– …

• Few lines of code are better than a thousand words!

Marco Moltisanti – OpenCV on Android Platforms

OpenCV for Android

There are 3 ways to use OpenCV on Android 1. Basic level

• OpenCV Java API only + Android SDK.

2. Advanced level • OpenCV native interface + Android NDK.

3. Hacker level • We don't suggest this way.

• 1 & 2 use prebuilt OpenCV package:

– It allows to use both Java and C++ OpenCV API

• 3 needs OpenCV builded from sources.

Marco Moltisanti – OpenCV on Android Platforms

Basic level

• OpenCV exposes almost all its functionality to Java including camera

• You don't even need to dive into NDK stuff

• You simply add a reference to the OpenCV, import its classes and move forward

• The overhead is equal to a cost of one or several JNI calls

• This approach works in many cases, but sometimes JNI calls can consume too much

Marco Moltisanti – OpenCV on Android Platforms

Advanced level

• The Android way is writing all your code in Java

• Sometimes it is not enough and you need to go to a native level and write part of your application in C/C++: – For example: you already have some computer

functionality which is written in C++ but do not want to rewrite the C++ code to Java.

• Add a class with native methods and wrapping your C++ functionality into the Java part of your Android application

• C++ code of OpenCV is cross-platform, that means that you can port your desktop code to mobile devices

Marco Moltisanti – OpenCV on Android Platforms

Hacker level

• We don't recommend this way, but it is useful if you want to work with OpenCV sources.

– Implement some new features

– Optimize the performance

– Submit a bugfix

– Want to have an access to the latest OpenCV features

We won’t try this approach!

Marco Moltisanti – OpenCV on Android Platforms

Install OpenCV Manager on the VD / 1

• Launch Android Virtual Device (AVD) Manager

• Start the Virtual Device

Marco Moltisanti – OpenCV on Android Platforms

Install OpenCV Manager on the VD / 2

• Let ANDROID_SDK be the folder where we installed the SDK;

• Let ANDROID_OPENCV be the folder where we put the opencv4android libs;

• Open a shell: – cd ANDROID_SDK\platform_tools

– adb install

ANDROID_OPENCV\apk\OpenCV_2.4.3_Manager_2.0_platform.

apk

– Wait for success!

Marco Moltisanti – OpenCV on Android Platforms

And now?

String code = "Let’s go to write down the code!";

System.out.prinltn(code);

Marco Moltisanti – OpenCV on Android Platforms