Graphics with Canvas, SurfaceView, and multitouch processing (panning and multitouch

zoom)

www.eecis.udel.edu/~bohacekGraphicsWithCanvas_2012.pptx

Approaches for Graphics

• Load image from /res/drawable– Best for static images

• OpenGL ES– 3D graphics (i.e., transforms such as spin can be applied to

graphical objects)– Best for game-type animation

• Draw on Canvas or SurfaceView– Canvas for drawing within the UI thread– SurfaceView is faster, and better for detailed graphics

• Note, if your main thread take too long, the OS will kill it, and it will be difficult to debug.

Drawable shapes• Make new app, edu.udel.eleg454.Graphics1• In onCreate is setContentView(R.layout.main)• Instead of the view generated by R.layout.main, we use our own, which extends View• In Graphics1 class, add• private class MyView extends View {

public MyView(Context context) {

super(context);}

@Overrideprotected void onDraw(Canvas canvas) {ShapeDrawable mDrawable = new ShapeDrawable(new OvalShape());mDrawable.getPaint().setColor(0xff74AC23);mDrawable.setBounds(10, 10, 310, 60);mDrawable.draw(canvas);}

}• Then, in onCreate, replace setContentView(R.layout.main); with setContentView(new

MyView(this));

• Run

• Besides OvalShape, ArcShape, PathShape, RoundRectShape, Shape, and BitMaps

View Widget• The previous method required us to replace setContentView(R.layout.main);• This resulted in the entire view being controlled by our view object

– E.g., we could not have a button in the view where we place the button with the layout editor• To fix this, we add a view widget• Move MyView to separate class• Make new class

– In package explorer, under /src– Find edu.udel.eleg454.TestGraphics1– Right click on edu.udel.eleg454.TestGraphics1– Select: new->class– Dialog opens

• Name: MyView• Superclass: View (then select browser to get full name: android.View)• OK

• Move functions from private class MyView to this new MyView– Move public MyView(– Move onDraw

• Also, in MyView addpublic MyView(Context context, AttributeSet attrs) {super(context, attrs);}

View Widget

• Go to main.xml graphical layout editor – Drag button to the screen– Leave id as button1

• Go to main.xml (not the editor)• Find second <Button …> </Button>• Before <Button>, add

– <edu.udel.eleg454.Graphics1.MyView android:layout_width="wrap_content" android:layout_height="wrap_content“ android:id="@+id/View01"></edu.udel.eleg454.Graphics1.MyView>

– Note that edu.udel.eleg454.TestGraphics1.MyView is the name of the separate class. If another name is used, then this name should be changed

– Save and go back to graphical view. There should be a box labeled MyView. Drag the box to make it larger

• Run

Canvas Drawing• Canvas has many drawing functions, e.g., drawPath(Path path, Paint paint)• In onDraw, add the following• Path: sequence of graphical objects

– Path path = new Path();– Make line between two points

• path.moveTo(10,10); // starting place• path.lineTo(160,160);

– add circle somewhere• path.addCircle(160,160,20, Path.Direction.CCW);

• Paint – for setting color and line width– Paint paint = new Paint();– paint.setDither(true);– paint.setColor(Color.RED);– paint.setStyle(Paint.Style.FILL_AND_STROKE);– paint.setStrokeJoin(Paint.Join.ROUND);– paint.setStrokeCap(Paint.Cap.ROUND);– paint.setStrokeWidth(10);

• Draw viewcanvas.drawPath(path, paint);

• run

Change graphics• At the end of TestGraphics1Activity.onCreate• MyView myView = (MyView) findViewById(R.id.View01);• Button button = (Button)findViewById(R.id.button1);• button.setOnClickListener(new View.OnClickListener() {});

– Let eclipse add unimplemented methods• In onClick, add

– myView.redraw();• In MyView

– Add class variable• int radius = 20;

– In onDraw, change• path.addCircle(160,160, 20, Path.Direction.CCW);• To• path.addCircle(160,160,radius, Path.Direction.CCW);

– Add function• public void redraw() {• radius = 80;• invalidate(); // this is needed to force redraw• }

• run

notes• Use invalidate() to force redraw

– In Graphics1, add variable• MyView myView;

– In Graphics1.onCreate(), add• myView = (MyView) findViewById(R.id.View01);

– Then myView.invalidate(); will force redraw

• Is canvas documentation for more graphics– E.g., drawBitmap has several functions

• Avoid declaring and setting variables in onDraw, instead, setting them elsewhere and access them from draw

• Use invalidate() to force redraw• Use SurfaceView for faster screen drawing

SurfaceView• Faster• You can draw on a SurfaceView from other threads, not just to UI thread

– When drawing with the UI thread, if the drawing takes a long time, then everything else must wait for the drawing to complete, • e.g., the user cannot press any buttons• If you put a long activity in the UI thread, a message will pop up saying that the app has stopped

responding• If you put a long drawing activity when starting the app, the system just kill it (thinking that it did

not start correctly)

• But, SurfaceViews are not transparent, nothing behind the view can be seen• Differences

– In Canvas approach, your onDraw function is called and has argument canvas. You can draw on this canvas. You can force a redraw can calling invalidate.• Invalidate will result in onDraw being called from the UI thread

– With surfaceview, you get a canvas and can draw on it whenever you want. Usually you draw on it from a new thread• E.g., You start the thread from the UI thread

SurfaceViewFun• Make a new app, SurfaceViewFun and package name

edu.udel.eleg454.SurfaveViewFun• Make new class

– Right click on edu.udel.eleg454.SurafceViewFun– Select New -> class– Name: MySurfaceView– SuperClass: SurfaceView

• Go to res/layout/main.xml• Open graphical view• Click on “Advanced”• Drag SurfaceView• In xml view, find the <SurfaceView ….• Change <SurfaceView … to <edu.udel.eleg454.SurfaceViewFun.MyView• Go back to graphical view and check that the surfaceView is now labeled

MySurfaceView. If not, then something is wrong.

MySurfaceView

• Open MySurfaceView• Eclipse will ask to add some unimplemented

functions. Add all three of these– public MySurfaceView(Context context, AttributeSet

attrs, int defStyle) – public MySurfaceView(Context context, AttributeSet

attrs) {– public MySurfaceView(Context context) {

• Each on has content– super(context, attrs); // eclipse migth add this part– ini(context); // a function to make

• Change – public class MySurfaceView extends SurfaceView

• To– public class MySurfaceView extends SurfaceView

implements SurfaceHolder.Callback– Add unimplemented functions

• Add member variables– SurfaceHolder surfaceHolder; // needed fro drawing– MyThread myThread; // will make MyThread next

Thread class• The whole reason to use a surfaceView is to draw on a different thread then the UI

thread. Let’s make a thread class.• Make new subclass in MySurfaceView and extend Thread, i..e, add

– class MyThread extends Thread {};• This will be the thread we use for drawing• We will draw an oval, by the drawing is slightly animated

• In MyThread, add member variables– SurfaceHolder surfaceHolder = null; // this a key variable as it allow use to get a canvas

– boolean done;– long startTime;– Canvas canvas;– RectF rectForOval = null;– float duration = 5*1000;– Paint drawingPaint;– float arcSweep;

MyThread member functions• Constructor

– public MyThread(SurfaceHolder _surfaceHolder) {• surfaceHolder = _surfaceHolder;• drawingPaint = new Paint();• drawingPaint.setColor(Color.BLUE);

– }• We will draw an oval, but it will be slightly animated• This is the function that will run in the thread.

– @Override– public void run() {

• Log.e("surface","running thread");• // some initialization• setOval();• startTime = System.currentTimeMillis();• arcSweep = 0;• done=false;• while (!done) { // draw until done

– updateArcSweep();– drawCurrentArc();

• }• Log.e("SurfaceViewFun","MyThread.run has finished");

– }

• Add the following functions to MyThread• public void setOval() {

– rectForOval = new RectF(10,10,300,600);• }• void updateArcSweep() {

– long currentTime = System.currentTimeMillis();– arcSweep = (float) ((currentTime-startTime)/duration*360.0);– if (currentTime-startTime>duration) {

• done = true;• arcSweep = 360;

– }• }• void drawCurrentArc() {

– canvas = surfaceHolder.lockCanvas(null); // must lock before drawing– canvas.drawColor(Color.BLACK); // clear the screen– canvas.drawArc(rectForOval, 0, arcSweep, true, drawingPaint); // draw new stuff– surfaceHolder.unlockCanvasAndPost(canvas); // must unlock when done

• }

MySurfaceView• Initialize (this function is called by each of the MySurfaceView constructors

– public void ini(Context context) {• Log.e("SurfaceViewFun","ini");• surfaceHolder = getHolder(); // MySurfaceView extends SurfaceView, which has member function getHolder. We use the

surfaceHolder to get the canvas that we will draw on• surfaceHolder.addCallback(this); // MySurfaceView implements SurfaceHolder.CallBack. This allow MySurfaceView to get

message about when the surface is ready for drawing and whether the surface has change (e.g., change orientation)• myThread = new MyThread(surfaceHolder); // make thread• setFocusable(true);

– }• In surfaceChanged, add

– Log.e("SurfaceViewFun","surface changed. Width: "+width+" height: "+height);• Start thread

– Since we extend SurfaceHolder.Callback , we implement surfaceCreated. – When this function is called, the surface is ready for drawing (the surface might not be ready for drawing when the

SurfaceView constructor is called. We need to wait for the surfaceCreated function)– In surfaceCreated, add

• Log.e("SurfaceViewFun","created"); • myThread.start(); // starts thread. Will lead to MyThread.run being called in its own thread

• Run it

playing• Compare directly writing onto canvas vs using a surfaceview and a thread• In surfaceCreated,change

– myThread.start();– To– myThread.run();

• Change – class MyThread extends Thread {

• To– class MyThread {

• Change – @Override– public void run() { …

• To– //@Override– public void run() { ….– ]that is, comment out the override statement

• So now there is no thread, we writing from the UI thread.• Run it• Undo what we did so it draws in a thread

Multi-touch and zoom• View objects process touches. • We need to implement this processing to capture multi-touch. • Also, to process some touches, we need GestureDetectors to help• In summary

– Override onTouchEvent– Extend ScaleGestureDetector.SimpleOnScaleGestureListener– Extend GestureDetector.SimpleOnGestureListener

• There is a bunch of code, but only a couple of critical spots• In MySurfaceView::ini, add

– iniTouchHandling(context);• At the end of MySurfaceView, add

– private float mPosX =0, mPosY = 0; // will indicate how much we have panned. Use these to adjust the graphics

– float mScaleFactor = 1.f; // indicate the scalling. Use this to adjust graphics– private float mLastTouchX;– private float mLastTouchY;– private static final int INVALID_POINTER_ID = -1;– private int mActivePointerId = INVALID_POINTER_ID;– GestureDetector mTapListener;– ScaleGestureDetector mScaleDetector;– public void iniTouchHandling(Context context) {

• mScaleDetector = new ScaleGestureDetector(context, new ScaleListener());• mTapListener = new GestureDetector (context, new TapListener());

– }

onTouchEvent

• @Override• public boolean onTouchEvent(MotionEvent ev) {

– // let our gesture detectors process the events– mScaleDetector.onTouchEvent(ev); – mTapListener.onTouchEvent(ev);– final int action = ev.getAction();– switch (action) {– case MotionEvent.ACTION_DOWN: {

• final float x = ev.getX();• final float y = ev.getY();

• // Remember where we started• mLastTouchX = x;• mLastTouchY = y;• mActivePointerId = ev.getPointerId(0);• break;• }

– //More on next slide

onTouchEvent (continued)

• case MotionEvent.ACTION_MOVE: {– final int pointerIndex = ev.findPointerIndex(mActivePointerId);– final float x = ev.getX(pointerIndex);– final float y = ev.getY(pointerIndex);

– if (!mScaleDetector.isInProgress()) {• // Calculate the distance moved• final float dx = x - mLastTouchX;• final float dy = y - mLastTouchY;

• // Move the object• mPosX += dx;• mPosY += dy;

• // Remember this touch position for the next move event• mLastTouchX = x;• mLastTouchY = y;

• // Invalidate to request a redraw• // invalidate(); // use this is a regular canvas is being used• myThread.setOval();• if (myThread.done==true)

– myThread.drawCurrentArc();

– }– break;

• }

The current amount that we have panned

With new values of mPosX and mPosY, we need to update the graphics-We can remake the oval-If we are still drawing, the the new oval will be drawn-If we have finished drawing, tehn we need to redraw-If we are not using a surface view (i.e., we are directly drawing on the canvas, then in order to have the new values of mPosX and mPosY be shown, we need to call invaliate for the view

onTouchEvent (end)

• case MotionEvent.ACTION_UP: {– mActivePointerId = INVALID_POINTER_ID;– break;

• }• case MotionEvent.ACTION_CANCEL: {

– mActivePointerId = INVALID_POINTER_ID;– break;

• }• case MotionEvent.ACTION_POINTER_UP: {

– // Extract the index of the pointer that left the touch sensor– final int pointerIndex = (action & MotionEvent.ACTION_POINTER_INDEX_MASK) – >> MotionEvent.ACTION_POINTER_INDEX_SHIFT;– final int pointerId = ev.getPointerId(pointerIndex);– if (pointerId == mActivePointerId) {

• // This was our active pointer going up. Choose a new• // active pointer and adjust accordingly.• final int newPointerIndex = pointerIndex == 0 ? 1 : 0;• mLastTouchX = ev.getX(newPointerIndex);• mLastTouchY = ev.getY(newPointerIndex);• mActivePointerId = ev.getPointerId(newPointerIndex);

– }– break;

• }• } // and switch statement• return true;• } // ends onTouch

ScaleGestureDetector.SimpleOnScaleGestureListener

• private class ScaleListener extends ScaleGestureDetector.SimpleOnScaleGestureListener {• @Override

– public boolean onScale(ScaleGestureDetector detector) {• mScaleFactor *= detector.getScaleFactor();• // Don't let the object get too small or too large.• mScaleFactor = Math.max(0.1f, Math.min(mScaleFactor, 5.0f));

• //invalidate(); // use this for a regular canvas (i.e., not a SurfaceView)• myThread.setOval();• if (myThread.done==true)• myThread.drawCurrentArc();

• return true;– }

• }

•Since mScaleFactor has changed, the graphics should be updated•Also, if we are drawing directly on the canvas, then we need to invalidate sot nat onDraw is called

GestureDetector.SimpleOnGestureListener• private class TapListener extends GestureDetector.SimpleOnGestureListener {• @Override• public boolean onDoubleTap(MotionEvent e) {

– Log.e("SurfaceViewFun","double tap "+e.getX()+" "+e.getY());– return true;

• }• @Override• public void onLongPress(MotionEvent e) {

– Log.e("SurfaceViewFun","got long press at location x="+e.getX()+" y="+e.getY());• }• @Override• public boolean onFling(MotionEvent e1, MotionEvent e2, float velocityX, float velocityY) {

– Log.e("SurfaceViewFun","fling: started at ("+e1.getX()+" ,"+e1.getY()+"). Ended at ("+e2.getX()+" ,"+e2.getY()+"). With velocity ("+velocityX+" ,"+velocityY+")");

– return true;• }• @Override• public boolean onScroll (MotionEvent e1, MotionEvent e2, float distanceX, float distanceY) {

– Log.e("SurfaceViewFun","scroll: started at ("+e1.getX()+" ,"+e1.getY()+"). Ended at ("+e2.getX()+" ,"+e2.getY()+"). With total distance ("+distanceX+" ,"+distanceY+")");

– return true;• }• } Our app does not use these touches. But if you want them, here they are

Make graphics use mPosX, mPosY, and mScaleFactor• Need to change the oval• public void setOval() {

– synchronized(surfaceHolder) { // make sure that we are not drawing while updating the oval• rectForOval = new

RectF(10*mScaleFactor+mPosX,10*mScaleFactor+mPosY,300*mScaleFactor+mPosX,600*mScaleFactor+mPosY);

– }• }• drawCurrentArc also need to be synchronized• void drawCurrentArc() {

– canvas = surfaceHolder.lockCanvas(null);– canvas.drawColor(Color.BLACK);– synchronized(surfaceHolder) {

• canvas.drawArc(rectForOval, 0, arcSweep, true, drawingPaint);

– }– surfaceHolder.unlockCanvasAndPost(canvas);

• }

• Run

mPosX, and mPosY translate the ovalmScaleFactor scales the oval

We should make sure that no other thread is reading RectF ad we are resetting it. We synchronized with surfaceHolder