Java and Imaging

Paintpaint() methodAWT Coordinate SystemColor & ColorModelthe java.awt.Color classthe java.awt.image.ColorModel class

paint() method Java Peer to draw, inherit Canvas, override paintredraw screen WINDOW_paint() EXPOSEDupdate() Eventrepaint() clear component window

AWT Coordinate System

The java.awt.Color classEncapsulate RGB colorshave static final Color object of frequent-used colorspublic static final Color black;public Color(int red, int green, int blue);public Color brighter();public Color darker();public int getRed();

java.awt.image.ColorModelencapsulate the METHODS of TRANSLATION from PIXEL VALUESget alpha, red, green, blue value from pixelimplemented by DirectColorModel, IndexColorModelpublic abstract class ColorModel;public colorModel(int nbits);public abstract getAlpha(int pixelvalue);public static ColorModel getRGBdefault();// return default ColorModel 0xAARRGGBB

DrawingThe Graphics classSimple DrawingWorking With TextFontFontMetrics

java.awt.Graphics classcontain current graphics context and methods for drawinggraphics context : fgcolor, bgcolor, font, etc.abstract class - actual java implementation and native library is provided by virtual machine

Drawing Methods of Graphics

Simple Drawingimport java.awt.*;import java.awt.event.*;public class DrawingCanvas extends Canvas implements MouseListener {protected Point pt;public DrawingCanvas() { pt = new Point(50, 50); addMouseListener(this);}public Dimension getPreferredSize() { return new Dimension(100, 100);} public void mouseClicked(MouseEvent e) { pt = e.getPoint(); repaint();}public void mouseEntered(MouseEvent e) { }public void mouseExited(MouseEvent e) { }public void mousePressed(MouseEvent e) { }public void mouseReleased(MouseEvent e) { }public void paint(Graphics g) { g.setColor(Color.blue); g.drawRect(pt.x, pt.y, 10, 10);}}

java.awt.Fontload Font object from font namepublic Font(String name, int style, int size);public static Font getFont(String nm);

java.awt.FontMetricshas methods to get information from Font objectpublic FontMetrics(Font fn);public getAscent();public getDescent();

Using imagesThe Image classLoading imagesLoadingKeep Track of Image LoadingWith the MediaTracker classWith the ImageObserver interface*Displaying images

Using images - Manipulating images*imagefilter

java.awt.Image classencapsulate imageprogrammer cant know about internal data structure representaing imagepublic abstract Graphics getGraphics();public abstract int getWidth();public abstract int getHeight();

public Image createImage(ImageProducer);public Image createImage(int w, int h);

Loadingautomatic loading about JPEG and GIF imagesbackground loadinguse methods in java.applet.Applet or java.awt.Toolkitload via filename or urlpublic abstract Image Toolkit.getImage(URL url);public abstract Image Toolkit.getImage(String file)

Toolkit Component.getToolkit() or static Toolkit Toolkit.getDefaultToolkit()

java.awt.MediaTrackerkeep track of image loadingpublic MediaTracker(Component comp);public addImage(Image img, int id);public boolean checkAll();public boolean checkID(int id);public boolean isErrorAny();public void waitForAll();

java.awt.MediaTracker - MediaTracker tracker;tracker = new MediaTracker(this);Toolkit tk = Toolkit.getDefaultToolkit();for (int i = 1; i

java.awt.image.ImageObserver interface

loading

imageUpdate() ImageObserver imagewidth knownheight knownloading

java.awt.image.ImageObserverpublic interface ImageObserver { public abstract boolean imageUpdate(image img, int infoflags, int x, int y, int width, int height);} If image knows width and height, or loading pixels, call this methods with infoflags. if return true, no more call this methodsComponent implements ImageObserverif not completely loaded yet, repaint()

java.awt.image.ImageObserverpublic int Image.getWidth(ImageObserver observer)If the width is not known yet then the ImageObserver will be notified later and -1 will be returned. public boolean drawImage ( Image img, int x, int y, int width, int height, Color bgColor, ImageObserver observer )return immediately. return ture if image is fully initialized.if not, return false, and let img notify observer

Displaying images public abstract boolean drawImage(Image img, int x, int y, ImageObserver observer);public abstract boolean drawImage(Image img, int x, int y, int width, int height, ImageObserver observer);public abstract boolean drawImage(Image img, int x, int y, Color bgcolor, ImageObserver observer);public abstract boolean drawImage(Image img, int x, int y, int width, int height, Color bgcolor, ImageObserver observer);

ImageFilter -

startProduction() setPixels()ImageProducerImageConsumer

ImageFilter - ImageConsumer

ImageFilter filter = new RotateFilter(angle);ImageProducer producer = new FilteredImageSource(souceImage.getSource(), filter);Image resultImage = createImage(producer);

ImageProducerImageFilter

Improving AnimationEliminating FlashingOverrideing update() methodClippingDouble BufferingSpeed up Image Loading with Clipping

Overriding update() methoddefault update() clear all component's Background and call paint()Components member methodpublic void update(Graphics g) { DoActualDrawing();}public void paint(Graphics g) { update(g);}

Clipping public abstract void clipRect(int x, int y, int width, int height); Clipping the Drawing Area

public void update(Graphics g) { Graphics g2 = g.create(x, y, w, h); paint(g2);}

Double Bufferingupdate drawing area one timeDimension offDimension;Image offImage;Graphics offGraphics;public void update(Graphics g) {if( (offGraphics == null) || ( d.width != offDimension.width) || (d.height != offDimension.height) ) { offDimension = d; offImage = createImage(d.width, d.height); offGraphics = offImage.getGraphics(); } DoActualDrawing(offGraphics); drawImage(offGraphics, 0, 0, this);}

Speed up Image Loading with Clippingwhen loading images using URLs, most of the time is taken up by initiating HTTP connections. make an image strip int stripWidth = imageStrip.getWidth(this);int stripHeight = imageStrip.getHeight(this);int imageWidth = stripWidth / numImages;g.clipRect(0, 0, imageWidth, stripHeight);g.drawImage(imageStrip, -imageNumber*imageWidth, 0, this);

New Features of Java1.1Enhancement on Graphics ClipCropping, Scaling, and FlippingMemoryImageSourcePixelGrabber

Enhancement on ClippingNew methods of GraphicsShape getClip();void setClip(Shape clip);void setClip(int x, int y, int w, int h);void getClipBounds();

Enhancement on ClippingGraphics g2 = g.create();g2.clipRect(10, 10, 100, 100);g2.drawImage(img2, 10, 10, this);g2.dispose();

Shape oldclip = g.getClip();g.clipRect(10, 10, 100, 100);g.drawImage(img2, 10, 10, this);g.setClip(oldclip);

Java Media APIsJava2DJava Advanced Imaging (JAI)Java Media Framework (JMF)Java3D

java.awt.imageImage is an abstract classImage objects are constructed in a platform specific mannerTwo types of image objects:Images created from an image sourceImages created by an AWT componentComponent.createImage()

Processing Images in AWTMemoryImageSourceImageProducerPixelGrabberMemoryImageSource

JAI modelsThe producer/consumer model AWT modelThe immediate mode model AWT modelThe pipeline modelThe JAI model

Push modelPull model

AWT Push modelJava 2D Immediate modePull ModeImage ImageProducer ImageConsumer ImageObserver BufferedImage Raster BufferedImageOp RasterOp RenderableImage RenderableImageOp RenderedOp RenderableOp TiledImage

BufferedImageColor ModelRasterColorSpaceSampleModelDataBufferjava.awt.image.BufferedImageEncapsulates the methods for translating a pixel value to colorcomponents (ex. red/green/blue) and an alpha component for rendering

DataBufferjava.awt.image.DataBufferDataBuffer stores pixel data as one or more arrays of primitive data types.Java2D buffer types:ByteIntegerShortUnsigned shortJAI buffer types:DoubleFloat

Sample ModelSampleMode describes how pixels are stored in the DataBuffer.Packed ModelsSingle-pixel packed modelMultiple-pixel packed sample modelComponent ModelsComponent sample model

ColorModelConverts raster pixel samples into color components for displayingPixel samples are packed into a short integer (16 bits). Each sample would be 5 bits.If displayed on a color monitor (0-255) per channel, the resulting image would appear dark.Color model handles the conversion.

ColorSpacesDevice independent/dependentConverts between devicesSupports many color spaces including:CMYKHSV (HSB)CIEXYZsRGBICC Color ProfileCreate ColorSpace via factory

Code ExampleCreating an Image

Alpha ChannelA mixing factor to control the linear interpolation of foreground and background colors.Varies from 0 to 10 indicates no coverage (clear)1 indicates full coverageEach sample is multiplied by the alpha channel

TransformationsDefinitions:Transformation is a function that maps an object from one point to another.Affine transform preserves:Finiteness ParallelismTransformations:TranslationScalingRotationReflectionShear

Translationyx

Scalingyx

Rotationyx

Reflectionyx

Shearyx

TransformationsTransformations can be concatenated:C = Translate * Scale * Rotate * ShearJava Classes:AffineTransform ( java.awt.geom )AffineTransformOp ( java.awt.image )

InterpolationWhat is Interpolation?Interpolation is a process of generating a value of a pixel based on its neighbors.Types of Interpolation:Nearest-neighborLinearBilinearTrilinearCubic

Interpolation - LinearP(x+1)pxPxpxPxP(x+1)dxxx+1Interpolated PixelBuilding Imaging Applications with Java p272

Bilinear InterpolationdydxP(x,y)P(x+1,y)P(x,y+1)P(x+1,y+1)P(x,y)

Trilinear InterpolationdydxdzP(x,y,z)P(x,y,z)P(x,y+1,z)P(x,y,z+1)

Cubic Interpolationpx = ax3 + bx2 + cx + d

Convolution KernelsImage I(i,j)Convolution kernel is a matrix f(m,n)

Example: f(m,n) =

Convolution Kernel ExampleI(3,3) = (1*5) + (1*17) + (1*6) + (2*11) + (1*11) + (1*1) +(1*5) + (1*7)= 74f(m,n)=

15687851768910111112231578910432

Smoothing FiltersWeighted average of the neighboring pixels.Often approximate a Gaussian function.

Edge DetectorsLaplacian

Sharpening

Java2D Image OperationsAffineTransformOpConvolveOpLookupOpRescaleOpColorConvertOpBandCombineOp (RasterOp)

Code ExampleInterpolation/Edge Detection

Image I/OThe Java Image I/O API provides a pluggable architecture for working with images stored in files and accessed across the network. It offers substantially more flexibility and power than the previously available APIs for loading and saving images.

Image I/O Key FeaturesRead & write multiple image formatsJPG, PNG, GIF, etc.Expose image metadata uniformlyW3C DOMProvide Access to thumbnail imagesSupport multi-image filesProvide pluggable architecture for additional formatsHandle large image files

Image I/O Packagesjavax.imageioImageIO, ImageReader, ImageWriterjavax.imageio.eventjavax.imageio.metadataIIOMetadata, IIOMetadataFormatImpl, IIOMetadataNodejavax.imageio.spiJavax.imageio.stream

Custom Plug-insImplement:Reader - extends ImageReaderWriter - extends ImageWriterProvider interfaces:ImageReaderSpiImageWriterSpiOptionally provideImage and stream metadata classes extending IIOMetadata

Image I/O Code ExamplesReading & writing imagesQuerying for supported image typesImplementing a plug-in

Java Advanced Image

JAI ConceptsMulti-tiled imagesDeferred executionNetwork imagesImage property managementImage operators with multiple sourcesThree-dimensional image data

Image Processing DAGsconstantfileloadaddDisplay Widgetsrotate

JAI Operator CategoriesPoint OperatorsArea OperatorsGeometric OperatorsColor Quantization OperatorsFile OperatorsFrequency OperatorsStatistical OperatorsEdge Extraction OperatorsMiscellaneous Operators

Point Operators

AbsoluteAddAddCollectionAddConstAddConstToCollectionAndAndConstBandCombineBandSelectClampColorConvertCompositeConstantDivideDivideByConstDivideComplexDivideIntoConstExpInvertLogLookupMatchCDFMaxMinMultiplyMultiplyConstMultiplyComplexMultiplyConstNotOrOrConstOverlayPatternPiecewiseRescaleSubtractSubtractConstSubtractConstSubtractFromConstThresholdXorXorConst

Area OperatorsBorderBoxFilterConvolveCropMedianFilter

Geometric OperatorsAffineRotateScaleShearTranslateTransposeWarp

Color QuantizationErrorDiffusionOrderedDither

File Operators

AWTImageBMPEncodeFileLoadFilestoreFormatFPXGIFIIPIIPResolutionJPEGPNGNMStreamTIFFURL

Frequency Operators

ConjugateDCTDFTIDCTIDFTImageFunctionMagnitudeMagnitudeSquaredPeriodicShiftPhasePolarToComplex

API

PlanarImage

RenderOP

Java.awt.image.RenderedImage

Java.awt.image.BufferedImage

ParameterBlock RenderableImageOp class sources objcet source vector, RenderedImages RenderableImage object. ParameterBlock object short s;add(s);short s;add( new Short(s));

ParameterBlock Side EffectParameterBlock get set methods referenceParameterBlock addSource(ParameterBlock pb, RenderableImage im) { ParameterBlock pb1 = new ParameterBlock(pb.getSources()); pb1.addSource(im); return pb1; }:pb1 pb share source Vector(a change in either is visible to both)ParameterBlock pb1 = new ParameterBlock( pb.getSources().clone() );

ParameterBlock clone method shadowClone methodclone method ParameterBlock source Vectors Sources , , Side Effect

shadowClone vectors reference references Source VectorParameter VectorSource VectorParameter VectorObjects

ParameterBlock

add method this objectsParameterBlock pb = new ParameterBlock(); op = new RenderableImageOp("operation", pb.add(arg1).add(arg2) ); objects

More api

import java.awt.Frame;import java.awt.image.renderable.ParameterBlock;import java.io.IOException;import javax.media.jai.Interpolation; // bilinear operator usageimport javax.media.jai.JAI;import javax.media.jai.RenderedOp; // operatorimport com.sun.media.jai.codec.FileSeekableStream; // import javax.media.jai.widget.ScrollingImagePanel; decode JAI (GIF,JPEG,TIFF,BMP,PNM,PNG) RenderedImage. bilinear interpolation 2

EX1public class JAISampleProgram { public static void main(String[] args) { // Step 1: FileSeekableStream stream = null; try { stream = new FileSeekableStream("as01.jpg"); } catch (IOException e) { e.printStackTrace(); System.exit(0); }

// Step 2: RenderedOp image1 = JAI.create("stream", stream);

FileSeekableStream

scale operator // Step 3: Interpolation interp = Interpolation.getInstance( Interpolation.INTERP_BILINEAR);

// Step 4: ParameterBlock params = new ParameterBlock(); params.addSource(image1); params.add(2.0F); // x scale factor params.add(2.0F); // y scale factor params.add(0.0F); // x translate params.add(0.0F); // y translate params.add(interp); // interpolation method

// Step 5: operator scale RenderedOp image2 = JAI.create("scale", params);

Interpolation bilinear scale scale operator scale

API // Scale int width = image2.getWidth(); int height = image2.getHeight();

// attach scrolling panel ScrollingImagePanel panel = new ScrollingImagePanel( image2, width, height);

/* Create a frame to contain the panel. */ Frame window = new Frame("JAI Sample Program"); window.add(panel); window.pack(); window.show(); }} ScallingImagePanel JAISampleProgram.jpx

Rendered Graphicsthe simplest form of rendering in JAI Rendered graphs are useful when it is necessary to work directly with the pixels Immediatethe image source: have been evaluated at the moment it is instantiated and added to the graph a new operation is added to the chain:compute its results immediately.

Rendered GraphicsA Rendered graphnodes are usually instances of the RenderedOp (any subclass of PlanarImage )Image sources objects that implement the RenderedImage interface

Example import javax.jai.*; import javax.jai.widget.*; import java.awt.Frame;

public class AddExample extends Frame { ScrollingImagePanel imagePanel1;

public AddExample(ParameterBlock param1, ParameterBlock param2) { RenderedOp im0 = JAI.create("constant", param1);. RenderedOp im1 = JAI.create("constant", param2); RenderedOp im2 = JAI.create("add", im0, im1);

// Display the original in a scrolling window imagePanel1 = new ScrollingImagePanel(im2, 100, 100); // Add the display widget to our frame. add(imagePanel1); } } Create a constant image Create another constant image Add the two images together.

Renderable GraphsA Renderable graphis made up of nodes implementing the RenderableImage interface

Instance of RenderableOP

Examplereads a TIFF file, inverts its pixel values, then adds a constant value to the pixels

RenderedOp sourceImg = JAI.create("TIFF",Jing01.tiff);

ParameterBlock pb = new ParameterBlock(); pb.addSource(sourceImg); pb.add(null).add(null).add(null).add(null).add(null); RenderableImage ren = JAI.createRenderable("renderable", pb);

ParameterBlock pb1 = new ParameterBlock();pb1.addSource(ren);RenderableOp Op1 = JAI.createRenderable("invert", pb1);

ParameterBlock pb2 = new ParameterBlock();pb2.addSource(Op1); // Op1 as the sourcepb2.add(2.0f); // 2.0f as the constantRenderableOp Op2 = JAI.createRenderable("addconst", pb2);

AffineTransform screenResolution = ...;RenderContext rc = new RenderContext(screenResolution);RenderedImage rndImg1 = Op2.createRendering(rc);

// Display the rendering onscreen using screenResolution. imagePanel1 = new ScrollingImagePanel(rndImg1, 100, 100);

Rendered Image RenderableImage operator addconst operator Get a rendering

Geometric TransformationRotation// Scale int width = image2.getWidth(); int height = image2.getHeight(); // Create the rotation angle (45 degrees) and convert to // radians. int value = 45; float angle = (float) (value * (Math.PI / 180.0F)); // Create a ParameterBlock and specify the source and // parameters ParameterBlock pb = new ParameterBlock(); pb.addSource(image2); // The source image pb.add(0.0F); // The x origin pb.add(0.0F); // The y origin pb.add(angle); // The rotation angle pb.add(interp); // The interpolation // Create the rotate operation RenderedOp RotatedImage = JAI.create("Rotate", pb, null); rotate Rotation

Java provides a rich set of media capabilities Raster - rectangular array of pixels and is composed of: - Data Buffer - actual data - Sample Model - knows how extract to interpret the data buffer - ColorModel - contains methods for converting the pixel data into color.- Fractions correspond to partial coverageeach sample is multiplied by the alpha channel when compositing Often pre-multiplied values are stored to speed renderingPixels of an image occupy integer coordinates. When images are rendered or manipulated, the destination pixels may lie between the integer coordinates. Neighboring pixels contribute a certain weight to the value of the pixel being interpolated. This weight is often inversely proportional to the distance at which the neighbor is located.

Different ways to do interpolation.Nearest-neighbor: the interpolating pixel is assigned the value of the nearest neighbor. This technique is fast, but it may not produce accurate imageLinear: immediate neighbors of the pixel to be interpolated are used to determine the value of the pixelBilinear interpolation is performed in one direction, and the result is applied to the linear interpolation in the other direction

Immediate neighbors of the pixel to be interpolated are used to determine the value of the pixel. Distance-to-weight relationship is linear.Linear interpolation if performed in one direction, and the result is applied to the linear interpolation in the other direction.Computed in a 3x3 neighborhood. Bilinear interpolation is first performed in the xyz plane, then linear interpolation is applied to the resulting value in the z direction. Performed in a neighborhood of four pixels. BufferedImageOpThis interface describes single-input/single-output operations performed on BufferedImage object.Can handle only single source images.RasterOpThis interface describes single-input/single-output operations performed on Raster objectPart of core Java APIs in 1.4JSR 15 - Image I/O Framework SpecificationRead & write - previous AWT was limited to only JPG/GIF formats with no clean way to add support for additional formatsExpose metadata - images often contain metadata, different formats have different ways of handling metadata, Image I/O provides a clean interface for retrieving and modifying metadata. Previously there was no way to access metadata.Thumbnails are often very useful when you want to quickly see a large imageImage I/O also makes if fairly easy to add support for new formats. Example would be support for DICOM imagesImages come in all different sizes - depending on the type of data it can be mega-bytes. For instance at Turbo the graphics for a our CD labels are over 20 megs - printing a decent label label requires a dpi much higher than 72! 72 dpi is good for graphics on the web, but not for your family photos - especially if you want to print them.The main package of the Java Image I/O APIImageIO - convenience methods for locating ImageReaders and ImageWriters2. A package of the Java Image I/O API dealing with synchronous notification of events during the reading and writing of images.3. A package of the Java Image I/O API dealing with reading and writing metadataIIOMetadata - abstract class, for image and stream metadata IIOMetadataFormatImpl - concrete implementation IIOMetadataNode - implements the w3c DOM Element interface4. A package of the Java Image I/O API containing the plug-in interfaces for readers, writers, transcoders, and streams, and a runtime registry.5. Stream - A package of the Java Image I/O API dealing with low-level I/O from files and streams.

To implement your own image io plugin you must: - implement your own reader and writer - provide two classes implementing the image reader/writer service provider interface that ImageIO classes use to locate plugins for a specific image - optionally provide implementations of iiometadata to hold stream and image data. We are going to explore three code examples demonstrating image I/O. The first example will demonstrate how to read and write images out.Then we are going to look at the supported image types - with image I/O you can query for supported formats.Finally, going to go over how you would implement your own custom codec to read/write images.Multi-tiled - images can be diced up as tiles and the processing performed on the tileDeferred execution - can create a chain of operators, with processing deferred until you request an image/tileNetwork images - has a model for distributing image processing using RMIImage property management - arbitrary data attached to an imageMultiple sources- operators can operate on multiple images (complicated when you think about it, images may have different data buffers, color models, color spaces etc).Three dimensional imaging - image can be composed of a stack of separate images, think of a CAT scanOperations can be hooked together into a DAG.Point operators - you to modify the way in which the image data fills the available range of grey levelsArea operators - perform geometric transformations which result in repositioning of pixels within an imageGeometric operators - modify orientation, size, and shape of an imageColor quantization - (aka dithering) used to reduce the appearance of amplitude contouring on monochrome frame buffers with fewer than 8 bits of depth or color frame buffers with fewer than 24 bitsFile operators - file input/output - now uses Image IO APIFrequency operators - decompose and image from spatial-domain to frequency domainEdge Extraction - identify edges in an imageMisc. Operators - operators that dont fall nicely into the different categories