![Page 1: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/1.jpg)
Ken HinckleyPrincipal ResearcherMicrosoft Research
Michel PahudBill Buxton
Direct Display Interaction via Simultaneous Pen and
Multi-Touch Input
![Page 2: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/2.jpg)
• Handhelds• Slates• Drafting Boards• Tabletops• Wall Displays
• Touch, or• Pen.
DIRECT INPUT A SHIFT TO
![Page 3: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/3.jpg)
ISN’T THIS ABOUTTHE FUTURE OF TOUCH?
WHY THE PEN?
![Page 4: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/4.jpg)
To help explain, let me introduce my nemesis.
![Page 5: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/5.jpg)
MR. I.M.A. BIGBODY, CORPORATE DENIZEN,THIRD RATE INC.
![Page 6: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/6.jpg)
“I CAN TYPE FASTERTHAN I CAN WRITE.”
![Page 7: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/7.jpg)
![Page 8: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/8.jpg)
TYPETHIS.
![Page 9: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/9.jpg)
OR THIS.
![Page 10: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/10.jpg)
OR MAYBETHIS.
![Page 11: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/11.jpg)
YOU CAN’T TYPE THE SOUL OFA NEW IDEA.
![Page 12: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/12.jpg)
BUT IT’S NOT ALL ABOUTTHE PEN.
![Page 13: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/13.jpg)
FIRST YOU NEED TO GRAB THE NAPKIN.
![Page 14: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/14.jpg)
“JUST TELL ME WHICH IS BEST, TOUCH OR PEN.”
![Page 15: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/15.jpg)
![Page 16: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/16.jpg)
PROPERTIES SHARED BY
PEN AND TOUCH
![Page 17: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/17.jpg)
PEN IS BEST!
Mechanical intermediary: familiar tool, lever arm, affords precision!
![Page 18: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/18.jpg)
TOUCH IS BEST!
No mechanical intermediary!
Nothing to lose!
No time to unsheathe!
![Page 19: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/19.jpg)
PEN IS BEST!
Definite, intentional pressure to leave a mark
![Page 20: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/20.jpg)
TOUCH IS BEST!
Just rest your hand…
No force required
BUT…“Midas Touch”
![Page 21: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/21.jpg)
EVERY COIN HAS TWO SIDES.
![Page 22: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/22.jpg)
OF HANDSAND
HANDEDNESS
![Page 23: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/23.jpg)
Q: Which hand do you write with, Right or Left?
![Page 24: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/24.jpg)
A: Yes
• Guiard, 1987: NPH defines frame of reference.• Right vs. Left is the wrong question to ask.
![Page 25: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/25.jpg)
What is the logic of the division of labor
between pen and touch in interface design?
![Page 26: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/26.jpg)
Pen WritesTouch Manipulates
![Page 27: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/27.jpg)
fluidity of annotation + manipulation navigation zooming object creation
![Page 28: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/28.jpg)
Pen + Touch Yields New Tools
![Page 29: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/29.jpg)
• Video• Dell XT2 Demo
![Page 30: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/30.jpg)
PEN + TOUCH GESTURES
![Page 31: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/31.jpg)
SUMMARY
![Page 32: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/32.jpg)
IF YOU’RE ONLY THINKING ABOUT TOUCH, YOU’RE BLOWING IT.
![Page 33: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/33.jpg)
GET OFF THE MULTI-TOUCHESCALATOR.
![Page 34: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/34.jpg)
![Page 35: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/35.jpg)
Not pen. Not touch.
Not pen OR touch.Pen AND Touch,
designed in concert
![Page 36: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/36.jpg)
“TOUCH OR PEN ARE JUST NEWWAYS TO CONTROL THE MOUSE,
SO WHAT’S THE BIG DEAL?”
![Page 37: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/37.jpg)
YOU’RE FIRED!
YOU’RE FIRED!
YOU’RE FIRED!
YOU’RE FIRED!
YOU’RE FIRED!
“TOUCH OR PEN ARE JUST NEWWAYS TO CONTROL THE MOUSE,
SO WHAT’S THE BIG DEAL?”
![Page 38: Ken Hinckley Principal Researcher Microsoft Research Michel Pahud Bill Buxton Direct Display Interaction via Simultaneous Pen and Multi-Touch Input](https://reader035.vdocuments.site/reader035/viewer/2022062517/56649f225503460f94c3b0a8/html5/thumbnails/38.jpg)