Content is available under Creative Commons, Attribution 3.0 license

iLabs mobile toolbox: Java MEapplications without intolerable pain.

Describes version 0.53b of the toolkit

2

iLabs

• Research project at Telenor R&I

• Three year funding, 1 year left.

• Focus:– How to develop new forms of collaboration through open source.

• Method:– Get dirty

– Real open source project: ilabs mobile toolbox

– Be analytical– Think about both concrete and abstract issues.

– Report to everyone– Top management, peers, nerds, suits, ..

3

Speakers

Bjørn Remseth. Cand. Scient. Research scientist at Telenor Research & Innovation. Bjørn has more than twenty years of experience as a sysadmin, developer, systems architect and research scientist.

Stian Børresen MSc. Ten years experience in 3D graphics, web, mobile and rich client development.He is currently working as a consultant specializing in mobile application development using java.

Else Nordhagen. Dr. Scient. Research Scientist at Telenor Research & Innovation. More than thirty years of experience with object oriented design and implementation, 3D graphics, theoretical computer science and GUI design (from Smalltalk in the late seventies, to JME today).

4

Outline of this talk:

• Why we are doing this, history so far, design goals.

• ”Hello GUI” – simple example application

• A tiny bit of detail:– Important classes and object structures

• “Application style” examples– OpenMiniChatter – XMPP chat

– OpenMiniBlogger – blog to blogger/blogspot.

– OpenMiniBrowser – simple HTML rendering

• “Widget style” example– A Twitter client.

• Links and contact info

• Questions

5

Why iLabs Toolbox?

Øystein Myhre (original author) with project manager Hilde Lovett

En early facebook/gtalk client.

We like mobile applications, would like to have more, but they painful to write, so we would like to:

• Make it easy: Remove stupid technical barriers.• Be inspiring: Facilitate and motivate the production of

useful mobile services.

Current core developers

6

In short

• It should be as simple to write an app that works on a mobile phone as it is to make content for the web

• We’re not quite there yet ;)

7

History so far:• Telenor R&I had several closed source Social Networking

example applications written in JME.• First open source release June 07, second massively reworked

release in March 2008. Bimonthly release cycle (java.net, freshmeat, sourceforge). August 08: 0.52

• Used by iLabs and some other projects, e.g.– Facebook client at the University of Colorado– Green touch, application at the Oslo School of Architecture– Vaccination registration application for HISP, developed at the Informatics

Department at Univ. Of Oslo– WellCom controlling set top boxes from your phone developed at Telenor

R&I in a EUREKA project.

8

Java versions

9

Design goals:

• Hide heterogeneity of devices

• One binary per application (no conditional compilation!)

• Remove common JME pitfalls.– Screen handling (Forms, Canvas)

– Event handling (Buttons, Menus)

• Simplify layout control of subwindows

• Simplify drawing of text, pictures etc.

• Adding real menus, tabbed views etc.

• Do not require an external proxy

• Leverage existing code for HTML, XMPP, ATOMS, Blogger...

!Challenged!

10

Development model

• Main license: Lesser Gnu Public License (LGPL)– To allow derivative works.

• Contributors:– 2-3 full time developers internally in Telenor R&I– Handful contributors worldwide.

• Openness: We accept anything that is remotely useful. If we ever get to the point where we have to reject something, we will start rejecting things but we are not there yet.

• Subversion repository: Used religiously.• Java coding standard/checkstyle: Used but with some

lapses.• Issue tracker: Read continously by developers. Telenor team

decides what is done by Telenor team. Everyone else decides what they do.

• Scholarships: We do scholarships. Talk to us if you have a cool project and you are a student. ;)

11

Example 1: How to make your own app (HelloGUI1). public final class HelloGUI extends

AbstractIlabsMIDlet {

public HelloGUI() {

}

public void initialize() {

ApplicationView view = new ApplicationView("Hello GUI!");

setCanvas( new ViewCanvas(getDisplay(),view));

view.setRightSoftbuttonAction(

new ExitAction("Exit"));

getCanvas().show();

}

}

• Create a new midlet by subclassing AbstractILabsMidlet

• Override method “initialize” in your midlet1.Set up your main

view, and a canvas to draw on.

2.Add menu items (Action instances) if any.

3.Show the canvas

That wasn’t very painful was it?

12

Example 2: (HelloGui3):

13

class HelloGUI3 extends AbstractILabsMidlet public final void initialize() { ApplicationView view = new ApplicationView("Hello GUI 3!");

view.addMenuItem(new ShowSelection("Item 1")); view.addMenuItem(new ShowSelection("Item 2")); view.addMenuItem(new ShowSelection("Item 3")); view.addMenuItem(new ShowSelection("Item 4"));

view.setRightSoftbuttonAction(new ExitAction("Exit")); setCanvas(new ViewCanvas(getDisplay(), view)); getCanvas().show(); }

User provided

Built in

14

Action that change Canvas:

class ShowSelection extends DisplayChangeAction {

ShowSelection(final String label) { super(label); }

protected void onExecute() {

final Alert alert =

new Alert("Hello GUI3 alert",

"You selected : " + getLabel(),

null, AlertType.INFO);

alert.setTimeout(Alert.FOREVER);

setCurrent(alert, getCanvas());

}

}

15

Boilerplate to get things going:

/** * Creates a new instance of HelloGUI3. */ public HelloGUI3() { }

16

Important concepts• (abstract) Graphics item – Something

that can be drawn.

• (interface) InputHandler – Something that interprets user input.

• View – Something on the screen that can respond to User input. – Often hierarchical. – Often both GraphicsItem and InputHandler.

• Action – Command pattern that does something

17

HelloGUI3, hierarchical decomposition

ApplicationView

TitleBar

MenuView

PopUpMenu

Clock

Action 4Action 3

Action 2Action 1

SimpleList

View

18

Example 3: OpenMiniChatter• A Google Talk client

• Introducing features:– Menus– Forms

– Username /Password

– Tabbed view– Switching between chats

– HtmlView – for chat content

– XMPP – Network protocol

• XML parsing: – MarkupReader - a pull parser– MarkupElement - creates an

object structure

• Persistent storage, simple extention to JME

Cool

19

OpenMiniChatter – dry run:

20

OpenMiniChatter - dry run - use of TabView

21

Example 4: OpenMiniBlogger• General components + Feed protocol

• Used towards {Blogger,blogspot}.com

• Phone-specific browser extension: – Camera input plugin

• XML parsing: – MarkupReader - a pull parser

– MarkupElement - creates an object structure

• Persistent storage, simple extention to JME

• Error handling mechanism that avoid the need to create ”millions” of different types of exception classes.

• URL encoding and parsing

• …

Cool

22

Blogger – dry run

23

What the HTML renderer sees<form

action=“"

id="cameraform"

method="post"

enctype="multipart/form-data"><object name="image_file"

classid=

"clsid:1b9982cf-4be8-42dd-9599-d0f6786d128d"

width="60"

height="80">

</object><input type="submit" name="update" value="Update"/>

</form>

What you seeLive camera view

24

Blogger – upload and view:

25

Example 4: OpenMiniBrowser

• Fast

• Simple

• Simple Html rendering– No

– css

– tables

– scripting

– But with – Forms

– cookies

– With extensions..

26

Example 4: OpenMiniBrowser public void initialize() {

mView = new ApplicationView("Open Minibrowser");

mCanvas = new ViewCanvas(Display.getDisplay(this), mView);

mHtmlView = new HtmlView(this);

mView.setMainAreaView(mHtmlView);

mHtmlView.setMargins(2, Font.getDefaultFont().charWidth(' '));

mView.addMenuItem(new OpenURLAction(mHtmlView, mHomePage, "Home"));

mView.addMenuItem(new EnterURLAction(mHtmlView, "Open url"));

mView.addMenuItem(new BackUrlAction(mHtmlView, "Back"));

mView.addMenuItem(new ExitAction("Exit"));

//a clock, displaying hours and seconds.

mClock = new Clock(System.currentTimeMillis(), false);

mView.addClock(mClock);

mView.layout(0, 0, mCanvas.getWidth(), mCanvas.getHeight());

mCanvas.show();

// Start a timer, update the clock once every minute.

final Timer timer = new Timer();

timer.scheduleAtFixedRate(new ClockTask(),

MILLIS_IN_ONE_MINUTE, MILLIS_IN_ONE_MINUTE);

mHtmlView.open(mHomePage);

}

27

Lessons so far

28

Lesson I:• It’s not that hard to write useful, portable JME programs

anymore ;)

• Most layout code in our demo programs end up being based on HTML, even when we’re not really doing web browsing.

• But we can make it even easier:

29

Where we started - where we ended up:

1. Plain JME programming

2. Social Networking : easiest to render HTML content from others

3. HTML : also used to make the user interface

4. AHA: nice, UI just like the web :-)

5. => Server side scripting as mobile programming method.

6. Enabling developer to make their own mobile applications without writing JME

JMEapp.

.jad&.jarUI

elements

.html

30

xml & html

J2ME programs vs. server side XML

J2MEApp. J2ME

Widger

App.

App.

App.

Widgets

J2MEApp. J2ME

App.

.jadand.jarfiles

.jad&.jar

31

WIDGER: Widlets through markup

• A Widger (an application)

• that runs runs Widlets(pieces of functionality living in the widger sandbox)

• that holds Views (showing some kind of content) that have

• Actions (that makes things happen).

Documentation: http://demo.ilabs.no/widgerdoc/index.php

32

UngWeb in Møre og Romsdal Fylkeskommune

33

UngeWeb: Widlet for ONE news item:

34

XML for widlet for one news item:

<?xml version="1.0" encoding="windows-1252"?><widlet ver="0.2" id=”tabbednyhet"> <view type="tabs" id="main">

<view type="panel" id="panel1" title="Nyhet 1" icon="/img/FolderExplorer.png" show="true"> <view type="web" id="web1" url="nyhet1.html"></view> <action type="loadiwg" label="Nyheter" url="nyheter.iwg" button="right"/> </view>

<view type="panel" id="panel2" title="Kommentarer" icon="resource://FolderExplorer.png"> <view type="web" id="web2" url="nyhet1_kommentarer.html"></view> <action type="reloadurl" label="Oppdater" for="web2" /> <action type="loadiwg" label="Nyheter" url="nyheter.iwg" button="right"/> </view>

<view type="panel" id="panel3" title="Skriv selv" icon="resource://BlogView.png"> <view type="web" id="web3" url="nyhet1_skrivSelv.html"></view> <action type="loadiwg" label="Nyheter" url="nyheter.iwg" button="right"/> </view>

</view></widlet>

35

Integrate with authoring tools:

UngWeb

Edit with eZ publish

Generates HTML and .iwg

Widger

36

Lesson II: Supporting protocols is hardhard

• Protocol code is painful and error prone to write in JME, – e.g. HTTPS :-(

– FaceBook, Twitter, FEIDE (ID login), ...

• Protocols change over time

• Workaround: exernal proxies that offload complexity

Proxy

Widger

The serviceserver

37

Two servers and one client:

The TwitterServer

Widger/Twitter adapter (in example demo running on an iLabs server)

Widger running theTwitter Widlet

38

Widget Twitter dry run: Server side login and formatting using forms and simple html.

In this case all pages are created on our server, not twitters.

Saves bytes and fixes problems with phone heterogeneity and weaknesses.

39

Server-side scripting:Advantages and Disadvantages:Advantages

• Shorter development time• Simpler client programs

– handles more phones– smaller– faster

• No need to download & install application again when updating service.

• Less network traffic on client due to tailored content on your server

• Better tools on server side for program development than JME...

• Easy to upgrade application

Disadvantages

• You need a server• .. that scales• Need to handle security issues

on the server• Limited functionality on

widger, today, but you can always extend it since it is open source

40

WIDGER: Widgets through markup

• A Widger (an application)

• that runs Widgets(pieces of functionality living in the widger sandbox)

• that holds Views (showing some kind of content) that have

• Actions (that makes things happen).

Documentation: http://demo.ilabs.no/widgerdoc/index.php

41

Example 5: Browser Widget :<?xml version="1.0" encoding="windows-1252"?><widget ver="0.2" id="httpwidget"> <view id="mainview" type="application"> <titlebar title="Http widget"/> <subview type="web" id="web1" url="resource://widgets/httpwidgetindex.html"> </subview> <action type="openurl" label="Home" url="resource://widgets/httpwidgetindex.html"for="web1"/> <action type="enterurl" label="Open url..."

for="web1"/> <action type="backurl" label="Back" for="web1"

button="right"/> <action type="exit" label="Exit"/> </view></widget>

Browser specific actions, built into widger

Default exit action

42

Result:

v.s:

Just another widget

Hand crafted JME application

43

Example 6: The Twitter Widget

44

Two servers and one client:

Twitter Widget (in Widger)

Widger/Twitter adapter (in example demo running on an iLabs server)

Twitter server

45

Widget Twitter client:

<widget> <view name="twitterview" type="appview"> <titlebar title="TwitterME" bordercolor="#000000" bgcolor="#94E4E8" textcolor="#ffffff"/> <view type="web" name="web1" url=”[...]/twitterlogin.html"> </view> <action type="exit" label="Exit"/> </view></widget>

[...] == long url

46

Widget Twitter client:

<widget ver="0.2" id="twitter"> <view type="application" id="main"> <titlebar title="TwitterME" … <subview type="web" name="web1" url="resource://widgets/twitterlogin.html">

</subview> <action type="loadiwg" label="Widgets" url="resource://widgets/masterwidget.iwg"/> <action type="exit" label="Exit"/> </view></widget>

Colors removed

47

twitterlogin.html (Plain old HTML, no surprises):<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <title>Twitter</title> <meta http-equiv="Content-Type" content="text/html; charset=windows-1252"> </head> <body> <p><img src="twitter.png"/></p><form action="http://demo.ilabs.no/test/widgets/twitter/twitter.php" id="signinform" method="post"> <p><b>Username:</b><br> <input name="username" type="text" title="Enter username" value="" /> </p> <p> <b>Password:</b><br> <input type='password' name="pwd" title="Enter password" value="" /> </p> <p> <input type="submit" name="signin" value="signin" /> </p> <p><center><img src="/widger/img/twitterbird.png"/></center></p> <p>Twitter is a service for friends, family, and co-workers to communicate and stay connected through the exchange of quick, frequent answers to one simple question: What are you doing?</p> </form> </body> </html>

Nothin

g ex

iting

happ

ening

her

e ple

ase

mov

e on

48

twitterME user interface (Plain old HTML, one small surprise):<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> … <textarea name="status"></textarea> <!-- <INPUT TYPE="file" name="image_file" id="image_file"/> --> <object name="image_file" classid="clsid:1b9982cf-4be8-42dd-9599-d0f6786d128d" width="60" height="80"></object> <input type="submit" name="update" value="Update"/> </form> <hr> </p> <h3>Showing the latest private posts</h3><hr>

<p><img src="http://s3.amazonaws.com/twitter_production/profile_images/55933057/cnn.twitter_normal.gif"/><b>cnnbrk : </b><br>Embattled…

Camera input

49

Widget Twitter dry run: Server side login and formatting using forms and simple html.

In this case all pages are created on our server, not twitters.

Saves bytes and fixes problems with phone heterogeneity and weaknesses.

50

Where we are now.. And where to go..• It works! on a lot of phones.• Its rather clean, easy to use and extend.• Documentation in the code and examples - and working

applications. But no tutorials.• A handful of users outside of the core team. • Server side configuration enables new groups to make

applications for mobile phones!• Improve look & feel (CSS, simple tables, etc.)• Lots of ideas for making it better!• Extend with phone specific functions, like the camera plugin,

and telco assets (OMTP standardization initiative)• Further develop the server side configuration idea• Report progress and grow the developer community (that’s why

we are here ;)

51

Tested for devices:• SonyEricsson, typical phones:

– k660i (Java platform 8)w880i (Java platform 7)k550i (Java platform 7)w810i (Java platform 6) : camera ”strange”K530i

• Nokia – Series 60 N93, Nokia N9, N73, N63, E51, E61– Series 40: They all have limited memory and usually have problems

reading large web-pages, in particular if there are large pictures– 6131– 6280 – did NOT work, at least with models with early firmware

• HTC – all models, limited functionality for versions with no camera access

from JME• Motorola – newer phones, require separate build• Others – not tested

52

How do we compare with other toolkits?• JME out of the box

– We are better in every way ;)– We have a a browser ;)

• JME Polish (http://www.j2mepolish.org/cms/)– They have nicer layout, CSS-like stylesheets– They use conditional compilation (one binary per configuration)– They have less flexible dynamic structure (e.g. variable no. Of

tabs, -only- build time configuration!)

• LWUIT (Lightweight UI Toolkit, https://lwuit.dev.java.net/)– They are more heavyweight – They do look nice.– They borrow some elements from Swing, but isn’t quite Swing.– They are supported by Sun (but in a strange way, no JCP, etc.)

53

Links:

• iLabs main page: http://www.ilabs.no/

• Toolkit homepage: https://ilabsmobiletoolbox.dev.java.net/

• Subversion repository homepage: https://ilabsmobiletoolbox.dev.java.net/source/browse/ilabsmobiletoolbox/

• Issue tracker: https://ilabsmobiletoolbox.dev.java.net/servlets/ProjectIssues

• Promotion: http://freshmeat.net/projects/ilabstoolbox/

• Downloadshttp://sourceforge.net/projects/ilabsmobiletool/

• Demo applications downloadable from phones:– http://m.ilabs.no

54

Questions?

Please .. ?

55

Thank you for your attention

56

XTRA

57

Example XML for a browser:<widlet> <view name="ONM08" type="appview"> <titlebar title="ONM 08" bordercolor="#FF9900" bgcolor="#444444" textcolor="#ffffff"/>

<view type="web" name="web1" url="http://....html"></view>

<action type="openurl" label="Home” url="http://....html" control="web1"/> <action type="useropenurl" label="Open url" control="web1"/> <action type="backurl" label="Back" control="web1"/> <action type="exit" label="Exit"/> </view></widlet>