ibm software | ibm - add multimedia to your midlets · 2010. 1. 15. · j2me to in order to...

Add multimedia to your MIDletsAn overview of the Mobile Media API (MMAPI)

Skill Level: Intermediate

Soma Ghosh ([email protected])Senior DeveloperUS Foodservice

29 Nov 2005

This tutorial demonstrates the Mobile Media API (MMAPI), which extends thefunctions of Java™ 2 Platform, Micro Edition (J2ME) by providing audio, video, andother time-based multimedia support to resource-constrained devices.

Section 1. Before you start

This tutorial describes the Java 2 Platform, Micro Edition (J2ME) Mobile Media API(MMAPI). You develop a MIDlet that takes advantage of the multimedia capabilitiesMMAPI provides.

About this tutorial

The MMAPI allows easy and simple access and control of basic audio andmultimedia resources on devices with limited memory and processing capabilities.This tutorial is intended to get you started. You need to have basic knowledge ofJ2ME to in order to understand the material.

The tutorial takes a basic approach to MMAPI, starting with an explanation of theMMAPI architecture, packages, and objects, and then showing you how to create anaudio- and video-capable MIDlet.

Prerequisites

For this tutorial, you need to install the J2ME Wireless Toolkit 2.0 (WTK; see

Add multimedia to your MIDlets© Copyright IBM Corporation 1994, 2008. All rights reserved. of 21

mailto:[email protected]

http://www.ibm.com/legal/copytrade.shtml

Resources for a link). MMAPI comes with the Toolkit. The J2ME WTK provides adevelopment and emulation environment for executing MIDP (Mobile InformationDevice Profile) and CLDC (Connected Limited Device Configuration)-basedapplications. J2ME 2.0 requires that you install JDK 1.4.

To execute the sample MIDlet, you'll need to download the MIDI file thataccompanies this tutorial -- pattern.mid (see the downloads section for a link) -- andput it in <WTK Home>\apps\MediaMIDlet\res before you execute the program.

Section 2. Introduction to the Mobile Media API (MMAPI)

This section offers a general overview of MMAPI. You'll look at MMAPI features,multimedia processing, and the MMAPI architecture.

What is MMAPI?

MMAPI is an implementation of the Java Community Process's JSR 135. As asimple and lightweight optional package, it gives Java developers access to nativemultimedia services available on a given device.

MMAPI features

The MMAPI brings the following capabilities to J2ME:

• Scalability: The API brings scalable sound and multimedia supportcapabilities to J2ME. It is aimed at the CLDC, Connected DeviceConfiguration (CDC), and profiles based on CDC and CLDC.

• Small footprint: The API allows easy and simple access and control ofbasic audio and multimedia resources on devices with limited memoryand processing capabilities.

• Support for tone generation, playback, and recording of time-basedmedia: The API supports any time-based audio or video content.

• Protocol and content independence: The API is not biased toward anyspecific content type or protocol.

• Ability to subset: Developers can limit application support to particulartypes of content -- like basic audio, for example.

• Extensibility: New features can be added easily without breaking olderfunctions. More importantly, additional formats can be easily supported,and the framework is in place for additional controls.

developerWorks® ibm.com/developerWorks

Add multimedia to your MIDlets of 21 © Copyright IBM Corporation 1994, 2008. All rights reserved.


• Options for implementers: The API offers features for differentpurposes. The API is designed to allow implementers to leave somefeatures unimplemented if they cannot be supported in a particularapplication.

Multimedia processing

There are two parts to multimedia processing:

1. Protocol handling: Reading data from a source such as a file or astreaming server into a media-processing system.

2. Content handling: Parsing or decoding the media data and rendering itto an output device such as an audio speaker or video display.

MMAPI architecture

To facilitate protocol and content handling operations, the MMAPI provides thefollowing high-level object types:

• DataSource

• Player

• Control

• Manager

I'll discuss each in turn in the following sections.

DataSource

A DataSource encapsulates protocol handling by hiding the details of how the datais read from its source. javax.microedition.media.protocol.DataSourceis the abstract parent class for all data sources in the Mobile Media API.

Player

A Player reads the data from a DataSource, processes it, and renders it to anoutput device. A Player knows how to interpret media data, be it MP3 audio data ora QuickTime movie.

Players are represented by implementations of thejavax.microedition.media.Player interface. Utility methods in DataSourceenable the Player object to handle the content.

Control

ibm.com/developerWorks developerWorks®



One or more Controls can be used to modify the behavior of a Player. You canget the Controls from a Player instance and use them while the Player isrendering data from media. For example, you could use a VolumeControl tomodify the volume of a sampled audio Player.

Controls are represented by implementations of thejavax.microedition.media.Control interface; specific control sub-interfacesare in the javax.microedition.media.control package.

To enable video capability in a MIDlet, you need to obtain a VideoControl. TheinitDisplayMode() method initializes the mode that determines how the video isdisplayed and must be called before video can be displayed. Potential modesinclude USE_GUI_PRIMITIVE, USE_DIRECT_VIDEO, and implementation-specificmodes. USE_GUI_PRIMITIVE defines how the GUI is displayed. The modeUSE_DIRECT_VIDEO defines how the video is displayed.

Manager

MMAPI specifies a third object, a factory mechanism known as the Manager, thatenables an application to create Players from DataSources and InputStreams.The javax.microedition.media.Manager class is the access point forobtaining system-dependent resources such as Players for multimedia processing.

The relationship between the various components is illustrated in Figure 1.

Figure 1. MMAPI components




Section 3. Using the Mobile Media API

In this section, I discuss MMAPI's key methods.

playTone()

The Manager.playTone () method, illustrated in Listing 1, plays a single tone ora very short sequence.

Listing 1. playTone()

publicstaticvoidplayTone(intnote,intduration,intvolume)throws




MediaException

The duration is specified in milliseconds, and the volume ranges from 0 (silent) to100 (loud). The note is specified as a number, as in a MIDI file, where 60 is middle Cand 69 is a 440 Hz A. Listing 2 illustrates playTone() in action.

Listing 2. playTone() in action

try {Manager.playTone(60,200, 90);} catch(MediaExceptionex) {System.out.println("can'tplaytone");}

createPlayer()

The real magic of the Mobile Media API is exposed through Manager'screatePlayer() method. There are three different versions of this method; whichversion you'll use depends on the way you want get to the media data.

The method in Listing 3 lets you create a Player by accessing media data from anInputStream. The type argument specifies the content type of the input media.For example, you'd use audio/midi for an audio MIDI. If null is used, Managerattempts to determine the type.

Listing 3. Creating a Player from an InputStream

public static Player createPlayer(InputStream stream, String type) throwsIOException, MediaException

The version of createPlayer() in Listing 4 lets you create a Player from aDataObject, an object that speaks a protocol to get access to media data.

Listing 4. Creating a Player from a DataSource

public static Player createPlayer(DataSource source) throwsIOException, MediaException

The simplest way to obtain a Player is to use the version of createPlayer() inListing 5 and just pass in a string that represents media data.

Listing 5. Creating a Player from a URL

public static Player createPlayer(String locator) throwsIOException, MediaException




MMAPI determines which protocol to use and gets the media data to the Player.For instance, you might specify an audio file on a Web server, as in Listing 6.

Listing 6. Specifying an audio file on a server

Player p = Manager.createPlayer("http://webserver/music.mp3");

The application uses the methods of the returned Player to control the retrieval andplayback of time-based media.

Life cycle methods: States of a Player

The Player's life cycle consists of five states:

1. UNREALIZED: When a Player is created, it is in the UNREALIZED state.

2. REALIZED: Calling realize() moves the Player to the REALIZEDstate and initializes the information it needs to acquire media resources.For example, if a Player is rendering an audio file from an HTTPconnection to a server, that Player reaches the REALIZED state afterthe HTTP request is sent to the server, the HTTP response is received,and the DataSource is ready to begin retrieving audio data.

3. PREFETCHED: Calling prefetch() moves the Player to thePREFETCHED state, establishes network connections for streaming data,and performs other initialization tasks. The state is achieved when thePlayer has read enough data to begin rendering.

4. STARTED: Calling start() causes a transition to the STARTED state,where the Player can process data. When the data is being rendered,the Player's state is STARTED. When it finishes processing (that is,when it reaches the end of a media stream), it returns to the PREFETCHEDstate.

5. CLOSED: Calling close() moves the player to the CLOSED state. Theapplication uses the methods of the returned Player to control theretrieval and playback of time-based media.

The Player's five states and the state transition methods are summarized in Figure2.

Figure 2. States of a Player




getControl()

A Player provides controls specific to the particular types of media it processes. Anapplication uses getControl() to obtain a single control, or getControls() toget an array of them.

Public Control getControl(java.lang.String controlType)

The controlType specifies the type of Control to be returned. For example, if aPlayer invokes getControl() with type MIDIControl, it gets back aMIDIControl. Similarly, when VideoControl is specified, the Player gets aVideoControl. If a control type is not supported, getControl() returns null.

Supported formats

MMAPI supports several audio and video formats. Formats supported by bothMMAPI and the J2ME Wireless Toolkit are:

• Audio: PCM and WAV

• MIDI: Type 0 (single track), Type 1 (multiple tracks), and SP-MIDI

• Video: MPEG-1

The Mobile Media API doesn't require any specific content types or protocols, butyou can find out at run time what formats are supported by a particular application bycalling Manager's getSupportedContentTypes() andgetSupportedProtocols() methods.

If you ask Manager to give you a Player for a content type or protocol that is notsupported, it will throw an exception.




Section 4. MMAPI in action: Building a sampleapplication

The sample MIDlet

In this section, you learn to develop a MIDlet with multimedia capabilities. It showsthe application of the various MMAPI functions I discussed in previous sections. TheMIDlet plays a simple tone, an audio clip from an URL, a video clip from an URL,and audio from a resource file. The source code is in Listing 7; it should be saved ina file named MediaMIDlet.java in the <WTK Home>\apps\MediaMIDlet\src directory.

Listing 7. A multimedia MIDlet

import javax.microedition.midlet.*;import javax.microedition.lcdui.*;import javax.microedition.media.*;import javax.microedition.media.control.*;import java.util.*;import java.io.*;/*** This is a demo midlet to show the basic audio functionalities, to* play a Tone, URL wave file and Resource file.** @version 1.6**/

public class MediaMIDlet extends MIDlet implements CommandListener, Runnable{

private Player player = null;




private Command exitCommand = new Command("Exit",Command.EXIT, 1);

private Command playCommand = new Command("Play",Command.ITEM, 1);

private ChoiceGroup menuList = new ChoiceGroup("Play List",Choice.EXCLUSIVE);

private Display display;

private Form form = new Form("The Mobile Media");

private static String mode="";

public MediaMIDlet() {super();display = Display.getDisplay(this);initPlayList();form.append(menuList);form.addCommand(exitCommand);form.addCommand(playCommand);form.setCommandListener(this);display.setCurrent(form);

}

public void run(){

try{if ("URL".equals(MediaMIDlet.mode)){

player =Manager.createPlayer("http://java.sun.com/products/java-media/mma/media/test-wav.wav");

player.realize();

player.prefetch();player.start();

} else if ("RESOURCE".equals(MediaMIDlet.mode)){

String loc = "pattern.mid";InputStream is = getClass().getResourceAsStream(loc);

player = Manager.createPlayer(is, "audio/midi");player.realize();


} else if ("VIDEO".equals(MediaMIDlet.mode)){

player =Manager.createPlayer("http://java.sun.com/products/java-media/mma/media/test-mpeg.mpg");

player.realize();VideoControl videoControl = (VideoControl)player.getControl(

"VideoControl");if(videoControl != null) {

Item video = (Item)videoControl.initDisplayMode(videoControl.USE_GUI_PRIMITIVE, null);

Form v = new Form("Playing Video...");v.append(video);

display.setCurrent(v);


}

}




player.realize();


} catch (IOException iex) {iex.printStackTrace();

} catch (MediaException ex) {System.out.println("can't create player");

} catch (Throwable t) {player = null;

}

}

public void startApp() {

}

public void pauseApp() {

}

public void start() {Thread t = new Thread(this); t.start();

}

/*** Destroy must cleanup everything not handled* by the garbage collector.

*/public void destroyApp(boolean unconditional) {

display.setCurrent(null);}

public void commandAction(Command c, Displayable s) {

if (c == exitCommand) {destroyApp(true);notifyDestroyed();

} else if (c == playCommand) {

int i = menuList.getSelectedIndex();if (i == 0) { // Simple tone

try {Manager.playTone(60, 200, 90);

} catch (MediaException ex) {System.out.println("can't play tone");

}

} else if (i == 1) {MediaMIDlet.mode="URL";start();

} else if (i == 2) {MediaMIDlet.mode="RESOURCE";start();

} else if (i == 3) {MediaMIDlet.mode="VIDEO";start();




}}

}

private void initPlayList() {

menuList.append("Play Simple Tone",null);menuList.append("Play URL",null);menuList.append("Play Resource",null);menuList.append("Play Video",null);

}}

Analyzing the Multimedia MIDlet

Now let's see the details of how the code works. The lines in Listing 8 indicate therequired packages for MIDlet media function.

Listing 8. MMAPI imports

import javax.microedition.midlet.*;import javax.microedition.lcdui.*;import javax.microedition.media.*;import javax.microedition.media.control.*;import java.util.*;import java.io.*;

The MIDlet implements the CommandListener and Runnable interfaces, asillustrated in Listing 9.

Listing 9. MMAPI interfaces

public class MediaMIDlet extends MIDlet implements CommandListener, Runnable {

CommandListener ensures that the MIDlet responds to commands like Play andExit, as you can see in Listing 10.

Listing 10. MIDlet command implementation

private Command exitCommand = new Command("Exit", Command.EXIT, 1);private Command playCommand = new Command("Play", Command.ITEM, 1);...public void commandAction(Command c, Displayable s) {

if (c == exitCommand) {destroyApp(true);notifyDestroyed();

} else if (c == playCommand) {// Play media from URL or Resource}

}




When a MIDlet implements a CommandListener, it must overwrite thecommandAction(Command c, Displayable s) method.

The application creates the commands Play and Exit. The Exit command quitsthe application. On receiving the Play command, the application determines theselected menu option and behaves accordingly.

The MIDlet implements Runnable to create a thread. In order to avoid a deadlockon networking and I/O operations, you should run a Player on a different threadfrom the MIDlet. To implement Runnable, you need to overwrite the run() method.All this is illustrated in Listing 11.

Listing 11. Multithreaded approach

public void commandAction(Command c, Displayable s) {// Get selected optionstart();

}public void start() {

Thread t = new Thread(this); t.start();}...public void run(){

// Create Player based on media}

The initPlayList() method builds the initial menu, menuList, that is displayedon launch of the MIDlet, as shown in Listing 12.

Listing 12. MIDlet List of options.

private ChoiceGroup menuList = new ChoiceGroup("Play List",Choice.EXCLUSIVE);...private void initPlayList() {menuList.append("Play Simple Tone",null);menuList.append("Play URL",null);menuList.append("Play Resource",null);menuList.append("Play Video",null);}

• When the first option, Play Simple Tone, is selected,Manager.playTone() is called and a simple tone is played.

• When the second option, Play URL, is selected, the mode is set to URL.

• When the third option, Play Resource, is selected, the mode is set toRESOURCE.

• When the fourth option, Play Video, is selected, the mode is set toVIDEO.

With all the options, the start() method is called to create a new thread. Therun() method of the thread determines whether to create a Player from a URL,Resource, or Video, depending on the mode value:




• If the mode is URL, Manager.createPlayer(url) is called. ThecreatePlayer() method takes care of determining the type of mediadata.

Listing 13. MIDlet options: Play URL

if ("URL".equals(MediaMIDlet.mode)){player =

Manager.createPlayer("http://java.sun.com/products/java-media/mma/media/test-wav.wav");player.realize();

player.prefetch();player.start();}

• If the mode is RESOURCE, the MIDlet forms an InputStream from theindicated location. In that case, Manager.createPlayer() takes theInputStream and related audio type as parameters and creates thePlayer.

Listing 14. MIDlet options: Play RESOURCE

if ("RESOURCE".equals(MediaMIDlet.mode)){

String loc = "pattern.mid";InputStream is = getClass().getResourceAsStream(loc);

player = Manager.createPlayer(is, "audio/midi");player.realize();

player.prefetch();player.start();}

• If the mode is VIDEO, Manager.createPlayer(url) is called. Afterthe player has been created, additional steps are required to initialize thephone screen to display the video, as illustrated in Listing 15.

Listing 15. Initialization for a video display

if ("VIDEO".equals(MediaMIDlet.mode)){

player =Manager.createPlayer("http://java.sun.com/products/java-media/mma/media/test-mpeg.mpg");

player.realize();VideoControl videoControl = (VideoControl)player.getControl("VideoControl");

if(videoControl != null) {Item video = (Item)videoControl.initDisplayMode(

videoControl.USE_GUI_PRIMITIVE, null);




Form v = new Form("Playing Video...");v.append(video);

display.setCurrent(v);


}

In all cases, MediaException and IOException should be properly handled.MediaException is thrown only if the Player cannot be created. On the otherhand, IOException is thrown if an invalid URL or InputStream cannot beformed. A Throwable object takes care of any thread-related exceptions.

Listing 16 outlines the basic methods in the MIDlet that need to be defined:startApp() to take care of any startup activities, pauseApp() to take care ofactivities while the MIDlet is paused, and destroyApp() to perform cleanupactivities not handled by the garbage collector.

Listing 16. MIDlet-related methods

public void startApp() {}

...

public void pauseApp() {}...public void destroyApp(boolean unconditional) {

display.setCurrent(null);}

Now you understand how the code works. In the next section, you'll see what theapplication looks like in action.

Section 5. Running the MIDlet

The J2ME Wireless Toolkit comes with MMAPI, along with a mobile device emulatoron which you can run the sample code. Start by invoking the Wireless Toolkit'sKtoolbar; on Windows, you can do this by choosing Start > Sun Java WirelessToolKit 2.3 Beta > KToolbar. Create a project by selecting New Project. Enter aname for the project as shown in Figure 3.

Figure 3. Create a project




After the project is created, create a Java file with the same name as the MIDletclass name in the <WTK Home>\apps\MediaMIDlet\src directory.

Build the project by choosing Build in KToolbar, and then choose Run to run it.Figure 4 illustrates the running MIDlet in the emulator. Navigate to each of theoptions, and then select each one and press Play to hear the different tones. Youcan also select the Video option to play a video clip, as shown in Figure 5.

The MIDlet developed here plays the following resources when options 2 and 4 areselected:

• http://java.sun.com/products/java-media/mma/media/test-mpeg.mpg

• http://java.sun.com/products/java-media/mma/media/test-wav.wav

In order to run option 3, the MIDI file available in this tutorial's downloads sectionshould be saved the <WTK Home Directory>\apps\MediaMIDlet\res directory.

Figure 4. Multimedia MIDlet menu




Figure 5. Multimedia MIDlet playing a video




Section 6. Summary

In conclusion

You've now seen how a MIDlet can incorporate a number of multimedia capabilities:playing various tunes, playing a WAV file from an URL, and playing a local MIDI file.You also saw how a video clip can be displayed on the phone screen. While theseexamples are simple, they have illustrated the powerful abilities that MMAPI can addto your mobile Java applications.




Downloads

Description Name Size Download method

Resource file wi-multimediasource.zip1KB HTTP

Information about download methods



http://download.boulder.ibm.com/ibmdl/pub/software/dw/library/wi-multimediasource.zip

http://www.ibm.com/developerworks/library/whichmethod.html


Resources

Learn

• developerWorks Wireless technology area: Stay connected to the wirelessworld.

• developerWorks Java technology zone: Keep up to date on events in the Javaarena.

• IBM's Pervasive Computing: Check out the latest developments.

• Stay current with developerWorks technical events and Webcasts.

Get products and technologies

• WebSphere Studio Application Developer: Build Java applications.

• Build your next development project with IBM trial software, available fordownload directly from developerWorks.

• J2ME Wireless Toolkit: To use MMAPI, you'll need to download the J2MEWireless Toolkit.

Discuss

• Participate in developerWorks blogs and get involved in the developerWorkscommunity.

About the author

Soma GhoshA holder of a Master's degree in Computer Science and Engineering, Soma Ghoshhas, over the past ten years, developed a wide range of Java applications in theareas of e-commerce and networking. She believes that wireless commerce is thewave of the near future, and is interested in and drawn to wireless initiatives fromexisting desktop components and models. Soma is an open source Java developerand currently associated with US Foodservice Corporation. She has authored severalarticles on J2ME for developerWorks, and she can be reached [email protected].



http://www.ibm.com/developer/wireless

http://www.ibm.com/developer/java

http://www.research.ibm.com/thinkresearch/pervasive.shtml

http://www.ibm.com/developerworks/offers/techbriefings/?S_TACT=105AGX03&S_CMP=art

http://www-306.ibm.com/software/awdtools/studioappdev/

http://www.ibm.com/developerworks/downloads/?S_TACT=105AGX03&S_CMP=art

http://java.sun.com/products/sjwtoolkit/download-2_3.html

http://www.ibm.com/developerworks/blogs/

mailto:[email protected]


ibm software | ibm - add multimedia to your midlets · 2010. 1. 15. · j2me to in order to...

Documents