![Page 1: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/1.jpg)
Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman &
Desney Tan
oronto
![Page 2: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/2.jpg)
InkSeine : Ink + Seine
-n: a fishing net -v: to fish with a seine
= insanely cool ink search!
InkSeine lets the user fish for useful information directly from ink notes.
Fishermen catching salmon on the Columbia River using a seine.(courtesy Wikipedia)
![Page 3: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/3.jpg)
XLibris—pen-based system for active reading(reading + critical thinking + learning)
InkSeine combines pen-based note taking with searching, linking, collecting & sensemaking:
![Page 4: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/4.jpg)
Ink – excels at capturing user’s thoughts & ideas with minimum distraction from formatting issues
Ink
Sear
ch
Gathe
r
![Page 5: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/5.jpg)
Search – digging through the file system on a tablet is really a pain Active note taking benefits
from low overhead access to supporting materials & related docs
Focus on personal search, but support Web as well
Ink
Sear
ch
Gathe
r
![Page 6: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/6.jpg)
Gather – Make it easy to use what you find by pulling content directly into unstructured notes Thumbnail hyperlinks to
documents Clippings from documents Queries that can be revisited
later
Ink
Sear
ch
Gathe
r
![Page 7: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/7.jpg)
InkSeine unifies these three elements in a fluid user experience that is integrated directly with inking
Ink
Sear
ch
Gathe
r
![Page 8: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/8.jpg)
Leverage preexisting ink to initiate search No context switch to “search app” – stay in the flow No tedious copying of information to a “search box”
Queries as first class objects, commingled with ink notes Flag some ink for search for later Visible and salient search history as part of notes Queries can be copied, pasted, moved, etc.
Interleave inking, searching & gathering No barrier between inking / searching – note ideas while
searching, side-by-side searching, span app boundaries for content gathering
Tightly couple queries with application content Queries naturally persist in originating context Easy re-visitation of prior queries Results of queries become new content
![Page 9: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/9.jpg)
Continuum of time cost – can defer at any point: 1 second: Capture thought “I should search on this” 5 seconds: Trigger search & see initial results list 10s: Open a document from result list, or drag result into
notes 10s-30s:
Inspect details of a few results Apply filters to results Add a word to the query Grab a snapshot from a short document Correct occasional reco errors – but Tablet’s reco is amazingly
good! Longer: sufficient depth for common info needs; not mired
by last 10% of complex queries that require full featured search tools Scroll through longer results lists Inspect result documents to see if they meet information needs Sideways searches based on Title, Date, or Author of a result
![Page 10: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/10.jpg)
InkSeine system
VIDEO
![Page 11: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/11.jpg)
![Page 12: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/12.jpg)
Paper Prototype
![Page 13: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/13.jpg)
Paper Prototype
![Page 14: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/14.jpg)
Paper Prototype
![Page 15: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/15.jpg)
Findings
Existing desktop search tool difficult to use Difficult text entry
Enter and correct text
![Page 16: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/16.jpg)
Findings
Existing desktop search tool difficult to use Difficult text entry
Enter and correct text Annoying context switch and flow interruption
Switch among apps and context
![Page 17: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/17.jpg)
Findings
Existing desktop search tool difficult to use Difficult text entry
Enter and correct text Annoying context switch and flow interruption
Switch among apps and context Tedious information gathering
Difficult to link to documents and gather pieces of information
![Page 18: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/18.jpg)
Notebook Control
![Page 19: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/19.jpg)
In Situ Search
![Page 20: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/20.jpg)
Breadcrumb
Breadcrumb
![Page 21: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/21.jpg)
Breadcrumb
![Page 22: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/22.jpg)
Trigger Personal Search
![Page 23: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/23.jpg)
Search Panel
![Page 24: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/24.jpg)
Query Area
Query Area
![Page 25: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/25.jpg)
Result List
Result List
![Page 26: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/26.jpg)
Result List
Result List
Focus Result
![Page 27: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/27.jpg)
File and Timeline Filters
Filters
![Page 28: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/28.jpg)
Gestures for Filters
Tap - Select Cross – Multi-select
Disjunctive cross – Multi-select Cross & Turn Right – Exclude
![Page 29: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/29.jpg)
Work Flow & OcclusionSelection
Correction
Search
![Page 30: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/30.jpg)
Work Flow & Occlusion
![Page 31: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/31.jpg)
Work Flow & Occlusion
![Page 32: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/32.jpg)
Crossing Widgets
![Page 33: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/33.jpg)
Resize Search Panel
![Page 34: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/34.jpg)
Examine Document
![Page 35: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/35.jpg)
Document & Tracking Menu
![Page 36: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/36.jpg)
Multiple Search Panels
![Page 37: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/37.jpg)
First Round ObservationsGesture discovery problem
![Page 38: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/38.jpg)
First Round ObservationsGesture discovery problemContext preservation & occlusion problem
![Page 39: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/39.jpg)
First Round ObservationsGesture discovery problemContext preservation & occlusion problemRecognition problem for out-of-vocabulary word
UISTList?, Dist?, Gist?, Mist?, Hist?
?
![Page 40: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/40.jpg)
Second Round Observations
Highlighter hintsSmart positioning of search panelPersonalized indexer
![Page 41: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/41.jpg)
Summary
In situ search experienceOptimum workflow / maximum flexibilityEnable rich tradeoffsGather contentMinimize search screen real estateSpan application boundariesTailored to pen input
![Page 42: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/42.jpg)
Future DirectionsPull in materials for searches from notes
![Page 43: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/43.jpg)
Future DirectionsPull in materials for searches from notesSupport vector-based searches
![Page 44: Ken Hinckley, Shengdong Zhao, Raman Sarin, Patrick Baudisch, Edward Cutrell, Michael Shilman & Desney Tan oronto](https://reader033.vdocuments.site/reader033/viewer/2022061304/5513ca1c5503466f748b49a5/html5/thumbnails/44.jpg)
Questions?