Taking Pictures with MMAPI

y Jonathan Knudsen
July 2003
Download: [source code]

The Mobile Media API (MMAPI) enables MIDlets to play and record audio and video data. The exact capabilities depend on the device implementation. Currently the only device that supports MMAPI is the Nokia 3650 mobile phone. This article describes how to build a simple camera application using MMAPI.

Tools for MMAPI Development

The first challenge of building an MMAPI application is finding the right tools. The J2ME Wireless Toolkit 2.0 includes support for MIDP 2.0, MMAPI 1.0, and WMA 1.1. At first glance, it looks like this is all you need — but the Nokia 3650 runs MIDP 1.0, not MIDP 2.0. To develop MIDP 1.0 applications, you'll need the 1.0.4 version of the J2ME Wireless Toolkit and an emulator that supports MMAPI.

Sun offers such an emulator, available from the MMAPI home page. While this emulator will allow you to build MMAPI applications, it does not support image capture, so it can't be used to test a camera application.

Nokia has an emulator that does support image capture, part of the Nokia Series 60 MIDP Concept SDK Beta 0.2, available from Forum Nokia, Nokia's developer web site. The file name is nS60_jme_concept_sdk_b0_2_014.zip. Linux developers should be able to use the Nokia Developer's Suite for J2ME, Version 2.0. The following installation instructions apply to Windows only.

Install the Nokia SDK in the <J2ME Wireless Toolkit>\wtklib\devices directory (or copy the Nokia_Series_60_MIDP_Concept_SDK_Beta_0_2 directory there after installation is complete). Next time you run the J2ME Wireless Toolkit, you'll have an additional emulator available, Nokia_Series_60_MIDP_Concept_SDK_Beta_0_2. When this emulator is selected, you can build MMAPI applications and test them as well. The emulator itself looks very much like the Nokia 3650.

MMAPI Device Testing

There's no substitute for testing on real devices. In the long term, many devices will support MMAPI, even though the Nokia 3650 is the only MMAPI device available just now.

There are several ways to deploy a MIDlet on the Nokia 3650. I transfer files via infrared between my laptop and the device. You can also use Bluetooth or do an OTA installation.

Getting a Video Capture Player

The first step in taking pictures (officially called video capture) in a MIDlet is obtaining a Player from the Manager. A special locator, capture://video, indicates that pictures will be captured from the camera using a default image size.

mPlayer = Manager.createPlayer("capture://video");

If the device does not support video capture, a MediaException will be thrown. You can check ahead of time to see whether video capture is supported; if it is, the system property supports.video.capture will be true.

The Player needs to be realized to obtain the resources that are needed to take pictures. If you haven't learned or don't remember what "realized" means, read through Mobile Media API Overview or The J2ME Mobile Media API.

mPlayer.realize();

Showing the Camera Video

The video coming from the camera can be displayed on the screen either as an Item in a Form or as part of a Canvas. A VideoControl makes this possible. To get a VideoControl, just ask the Player for it:

mVideoControl = (VideoControl) mPlayer.getControl("VideoControl");

If you wish to show the video coming from the camera in a Canvas, initialize the VideoControl, then set the size and location of the video in the Canvas, then make the video visible. The following example (the constructor of a Canvas subclass) shows how to place the video two pixels in from the sides of the Canvas. If the incoming video cannot be placed that way, the constructor tries using the full screen. Finally, it calls setVisible() to make the camera video visible.

public CameraCanvas(SnapperMIDlet midlet, VideoControl videoControl) { int width = getWidth(); int height = getHeight(); mSnapperMIDlet = midlet; videoControl.initDisplayMode( VideoControl.USE_DIRECT_VIDEO, this); try { videoControl.setDisplayLocation(2, 2); videoControl.setDisplaySize(width - 4, height - 4); } catch (MediaException me) { try { videoControl.setDisplayFullScreen(true); } catch (MediaException me2) {} } videoControl.setVisible(true); }

Showing the camera video in a Form is slightly different. Instead of calling VideoControl's initDisplayMode() method with USE_DIRECT_VIDEO, use the USE_GUI_PRIMITIVE value instead. On MIDP devices, you'll get back an Item you can place in a Form to be displayed.

Form form = new Form("Camera form"); Item item = (Item)mVideoControl.initDisplayMode( GUIControl.USE_GUI_PRIMITIVE, null); form.append(item);

Capturing an Image

Once the camera video is shown on the device, capturing an image is easy. All you need to do is call VideoControl's getSnapshot() method. You'll need to pass an image type, or null for the default type, PNG. You can find out ahead of time the image types that are supported. The video.snapshot.encodings system property contains a whitespace-delimited list. The Nokia 3650 supports PNG, JPEG and BMP.

The getSnapshot() method returns an array of bytes, which is the image data in the format you requested. What you do at this point is up to you: you might save the bytes in a record store, send them to a server, or create an Image from them so you can show the user the picture just taken, as in this example:

byte[] raw = mVideoControl.getSnapshot(null); Image image = Image.createImage(raw, 0, raw.length);

The image returned by the Nokia 3650's MMAPI implementation is 160 x 120 pixels. Although the camera is capable of 640 x 480 (which is what you get with the native camera application), the MMAPI implementation on this phone can't handle the higher resolution.

Creating a Thumbnail Image

One seemingly simple thing I wanted to do in this article was talk about how to create a thumbnail image, a smaller version of the image captured from the camera. MIDP 2.0 includes methods for obtaining the raw pixel values of an Image, which would make a true scaling transformation possible. Unfortunately MIDP 1.0 does not provide access to the pixel data.

The solution I implemented is neither elegant nor strictly correct (in an image-processing sense), but it approximates the behavior I wanted and doesn't involve parsing the PNG format. I create a new (blank) image for the thumbnail. For each pixel of that image, I set the clipping region to encompass that single pixel, and draw the source image at an appropriately scaled location.

private Image createThumbnail(Image image) { int sourceWidth = image.getWidth(); int sourceHeight = image.getHeight(); int thumbWidth = 64; int thumbHeight = -1; if (thumbHeight == -1) thumbHeight = thumbWidth * sourceHeight / sourceWidth; Image thumb = Image.createImage(thumbWidth, thumbHeight); Graphics g = thumb.getGraphics(); for (int y = 0; y < thumbHeight; y++) { for (int x = 0; x < thumbWidth; x++) { g.setClip(x, y, 1, 1); int dx = x * sourceWidth / thumbWidth; int dy = y * sourceHeight / thumbHeight; g.drawImage(image, x - dx, y - dy, Graphics.LEFT | Graphics.TOP); } } Image immutableThumb = Image.createImage(thumb); return immutableThumb; }

The Snapper Example

The source code download for this article is an example application called Snapper. Snapper allows you to take pictures and shows in a Form thumbnails of all the pictures you've taken. The code excerpts in this article are from the Snapper example. The application has only two classes. SnapperMIDlet provides application logic and screen flow as well as most of the MMAPI code for creating a Player and getting a snapshot. CameraCanvas is a Canvas subclass that sets itself up with a VideoControl to show the video coming from the camera.

Snapper is smart enough to figure out whether video capture is supported; try running it on Sun's MMEmulator, for example, and you'll receive a polite message:

Running Snapper on a Device Without Video Capture

If you use the Nokia Concept emulator instead, you'll see a screen like this:

Snapper's Main Screen

Choose the Camera command (the Nokia implementation places this in the Options menu). On a real device, you'll see live video coming from the camera; the emulator shows a single still picture. In either case, CameraCanvas draws a green border around the video.

Snapper's Camera Screen

To take a picture, choose Capture or press the Fire button (in the center of the arrow buttons). Snapper creates a thumbnail and shows it on the MIDlet's main screen.

Snapper's Main Screen With a Thumbnail

Each time you take another picture, a thumbnail image is added to the main screen.

Future Directions

The Snapper example is simple, but there are many interesting places you could take it:

• You could save the full-size images in persistent storage, so that the user could browse through them later.
• You could allow the user to remove images, or change their order.
• You could send the full-size images to a server for storage and provide a web interface for browsing them.
• In MIDP 2.0, you could allow for standard transformations of the images: blurring, sharpening, color inversion, geometric transformations, among others.

Summary

The Mobile Media API is a flexible and powerful API that allows the rendering and capture of audio and video data. This article describes how to use MMAPI's video capture capability, what tools you'll need, and how to use them to write MIDlets that take pictures. The Snapper example allows you to take pictures and creates smaller thumbnail images. With a little work, the Snapper example could easily be the basis of a networked image-sharing application.