the chartex project: a case study in interdisciplinary hci christopher power advanced topics in...
TRANSCRIPT
The ChartEx Project: A Case Study in Interdisciplinary HCI
Christopher Power
Advanced Topics in Interactive Technologies
January 31, 2013
Interdisciplinary Research
• Normally in HCI, we like to think of ourselves as interdisciplinary …
• … in our own group we have:– 1 Psychologist– 2 Computer Scientists– 1 Mathematician
• And yet we all think of ourselves as HCI researchers
Interdisciplinary Research (2)
• There are of course a huge number of fields that contribute to HCI:– Computer Science, Engineering, Psychology,
Sociology, Anthropology, Semiotics, Philosophy, Business and Management
• But is that what interdisciplinary really means?– That is all contained within our field – we are all
working in the same broad domain, often on the same problems
Interactive Systems for Science Research
• Interactive systems are everywhere and some fields get a lot of attention, notably scientific fields
Interactive Systems for Humanities?
• A huge amount of money and effort are changing physical resources into digital representations
• Where do we start in trying to design new interactive systems for these digital artefacts?
Digital Cultural Heritage
• What is digital cultural heritage?– Emerging discipline bringing together humanities
researchers with purveyors of digital “stuff”• Up until now, has been driven by digitization and
preservation– Massive push in the EU over the last 10 years to digitize
collections in archives, museums and libraries– Full digitization: > 100 Billion Euros (Poole, 2010)– Massive investment already committed to an organization
called Europeana to archive and provide access to the millions of Euros of investment in digitization already spent
• So what is the problem?
Digital Cultural Heritage (2)• We can’t do anything with them!• The most advanced technology we have for interacting with them?
The equivalent of a file browsing search:– http://discovery.nationalarchives.gov.uk/SearchUI/– http://www.hrionline.ac.uk/causepapers/
• Sometimes small projects come up with gorgeous, enriching web interactions – however, these are often ad-hoc approaches and not reproducible – nor do we understand the design principles for them
• And what if we want to do more than just display documents? Surely people are doing things with the documents – why aren’t we supporting those tasks?
• But unlike home and office systems, we have no context or knowledge of what those tasks are!
A history lesson …
• About 4 years ago we were approached by a medieval historian with this great idea:– Create an online tool that would allow her to use her
domain knowledge to reason about documents and synthesize new knowledge about history – in particular about locations in space and time (e.g. where was the blacksmith shop on Goodram Gate?)
– What she really didn’t want was a computer just running off and coming up with a bunch of stuff automatically
– It was about helping her, the historian, do the thing she does best – tell stories about the past from the artefacts we have from history
A history lesson…(2)
• We immediately saw a parallel with an HCI idea from around 2000 from Beaudoin-Lafon: instrumental interaction
• Beaudoin-Lafon argued that many of the systems we build deskill people; we pursue systems that do everything automatically, when instead we should be finding ways to let the use contribute their knowledge to the system – where the system becomes an instrument (i.e. like a scalpel for a doctor)
A history lesson … (3)
• We liked the idea so much that we jumped in with both feet. It ticked all the boxes– Really hard problems that would stretch our HCI
theories and practices– Creation of new interesting interactive technologies– Fun! It isn’t boring like life insurance, or banking, or
scary like safety in aerospace or medicine – it is about enriching our lives and our culture
• So after about 3 years of trying, we got a grant to try to build this interactive system
ChartEx: The Charter Excavator
• Received money from the Digging into Data Challenge– Money to try to do something with all of this
digitized data – from the UK, Canada, US and Ned.
• Our main purpose was to come up with an interactive system that would allow people to interact with medieval charter documents and reason about the people and places in them
Wait … what’s a charter document?
• Charters (or charter deeds) are legal documents that describe the transfer of land holdings from one owner to another (oversimplified – the historians would scold me)
• They are some of the most important remaining legal documents that we have, especially in Europe, as they can help work out where people lived, or groups lived, or wealth distribution, or spheres of influence … you get the idea
• To work with these documents is a highly specialized skill• Some of the best holdings of charters in the world:
University of York and the University of Toronto
Charter Document• 408. Grant by Thomas son of Josce goldsmith and citizen of York to his
younger son Jeremy of half his land lying in length from Petergate at the churchyard of St. Peter to houses of the prebend of Ampleford and in breadth from Steyngate to land which mag. Simon de Evesham inhabited; paying Thomas and his heirs 1d. or [a pair of] white gloves worth 1 d. at Christmas. Warranty. Seal.
• Witnesses: Geoffrey Gunwar, William de Gerford[b]y,' chaplains,Robert de Farnham, Robert le Spicer, John le plastrer, Walter de Alna goldsmith, Nicholas Page, Thomas talliator, Hugh le bedel, John de Glouc', clerks, and others.
• January 1252 [1252/3].• SOURCE: VC 3/Vi 326 (161 mm. x 137 mm.)• ENDORSEMENT: Petergat', Donacio facta vicariis de domo que fuit Thome
aurifabri; Simonis Evesham.• SEAL: Slit.' Hole in MS.• NOTE: See 403.
ChartEx Tool Chain (1)
• In order to do anything with all of this data we have, we need to know what is in the documents – images aren’t going to work
• We need to process and mine some of the information from the documents to get at the key entities that historians use when doing their work– We’ll get back to those in a minute …
ChartEx Tool Chain (2)Charter Documents
Analysed individualdocuments
Language processing
Data Mining
Analysed integrateddocuments
Researcher’s workbench
Questions questions …
Classic problem in a different domain
Historian View of Problem Technologist View of Problem
Questions questions … (2)
• And there we learned our first rule of interdisciplinary work:– In true interdisciplinary work there is no common
vocabulary – everyone speaks a different language
• For technology people the solution is to grab their terminology (Linked Data is a favourite in this domain) – not useful for historians
• What do we do in HCI when we hit a problem like this?
Answers!
• Participatory design– This was the point of participatory design – getting
domain experts working to design systems (think about Scenario Based Design)
• But we weren’t yet trying to design the interface – we were working at the level of the content of the documents
• Well … that’s kind of different – but maybe we can try some of our participatory stuff out on those documents to see if we can understand how people work with their contents
Participatory Design of a Content Model
• Proviso: This is one of the weirdest things I’ve ever done as an HCI practitioner.
• Historians work with documents in interesting ways – one thing they do is diplomatic markup – this is a process where the historian marks different entities and relationships in a document and gives them semantic meaning– Example: If I had the name John in a document, he is
(likely) a person. John could be the owner of some land, or he could be a witness, or he could be related to some other person. These things can be marked in the document.
Participatory Design of a Content Model
• We had a group markup session!– Had a moderator/scribe and 2-3 historians– Each historian brought their 3 favourite charters– We then chose a type of “thing” (entity) that we
wanted marked up: Actors (People, Institutions), Places, Sites, Events
– For each entity type, people pointed to parts of the document that were of that type
– For each entity, the historians indicated the relationships and roles that entity was playing in the document.
Identifying Entities
• Initial examination charters for entities about actors, sites and events………..
• 408. Grant by Thomas son of Josce goldsmith and citizen of York to his younger son Jeremy of half his land lying in length from Petergate at the churchyard of St. Peter to houses of the prebend of Ampleford and in breadth from Steyngate to land which mag. Simon de Evesham inhabited; paying Thomas and his heirs 1d. or [a pair of] white gloves worth 1 d. at Christmas. Warranty. Seal.
Identify Relationships between Entities
• Initial examination of charters for relationships between entities …
• 408. Grant by Thomas son of Josce goldsmith and citizen of York to his younger son Jeremy of half his land lying in length from Petergate at the churchyard of St. Peter to houses of the prebend of Ampleford and in breadth from Steyngate to land which mag. Simon de Evesham inhabited; paying Thomas and his heirs 1d. or [a pair of] white gloves worth 1 d. at Christmas. Warranty. Seal.
Participatory Design of a Content Model (2)
• We ended up with a collection of documents that had a robust collection of information– Gave a clue about the types of things that historians were using to
reason about documents• A content analysis on the documents gave us a high level set of
entities, an a set of relationships that can occur between relationships
• From that analysis– a content model (technical details omitted) for documents was
produced– a set of markup guidelines were created for working with future
documents• More importantly – we now had a common understanding of
what we was interesting in these documents
Document processing
50documents
Trainingdocuments
Unmarked Documents
Newly markeddocuments
LanguageProcessing
Interactive System Design
• We needed to know more than just content of the documents, we needed to know what people did with them when doing research
• We immediately ran into an interdisciplinary problem:– Humanities people are trained VERY differently from
scientists – so we couldn’t even pose questions in a way that made sense to historians
• A further problem is: research is tacit knowledge; it is often very hard for researchers to describe how they synthesize new information
Reaching into the toolbox
• In your modules, you are learning a set of tools that allow you to overcome challenges like this.
• What elicitation methodology might help us overcome these problems?
Contextual Inquiry• To deeply understand the tasks
of historians, we undertook a contextual inquiry
• We asked the historians to find a problem they had recently solved and retrospectively walk us through the solution– One historian identified a set of
shops on a street in York– Another identified particular
trends in witness lists in Essex– Another traced the holdings
and influence of an Abbot in Catalonia
Contextual Inquiry (2)• We recorded the sessions and then did partial transcriptions
of the videos– Identified key actions and tasks undertaken– Utterances where people talked about things that were important
in either the documents or their notes• Distilled a set of basic requirements that fell into three broad
activities:– Searching for documents in collections– Interacting with individual documents to understand their contents– Relating information between documents
• Tied into all of this was ideas around their confidence in particular pieces of data (e.g. the Abbot in Document A is a Bishop in Document B)
Interaction Design
• We began by dividing the interface it the three activities
• Started with the easy one: searching for documents in collections– Built a prototype for an interface that was
centred on the metaphor of file boxes, sorting documents, carousels of documents
• Then we had our first co-design workshop to evaluate it with our historians …
Interaction Design FAIL
• They really didn’t like it!• It emerged there were 3 key problems:
1. Despite repeatedly saying they wanted something new, they all wanted to go back to the file/folder metaphor of the desktop interface
2. The document search was nothing like the searches they were using in existing archive systems
3. They couldn’t get to “the document” easy enough• One interesting thing was that people had a hard
time expressing what the problems were on the fly during the demonstration
Interaction Design: Round 2
• Because of the feedback we had, we knew that we had to start getting down to working with the documents
• We moved to looking at the tasks of working with each document and relationships between documents more holistically
• We focused on making the document the key pivot point in the interface
Prototype Demonstration
• Low-fidelity prototype prepared with OmniGraffle
• Exported to web pages and later power point to demonstrate functionality
Evaluation of Round 2
• This time, we did evaluation in two phases– Demonstration of the interface functionality– Co-design workshop approximately 2 weeks
afterwards• This gave people some time to think
about what worked and didn’t work
Outcome of Evaluation• Overall, the historians liked what they saw
– Very positive about being able to investigate relationships between the documents
– Liked that they could keep their own copies and update those copies with their own knowledge (e.g. confidence)
• One really interesting outcome– Some historians wanted to be able to go from a high level view and
immediately see the relationships – then drive down to document level– This is something they couldn’t do previously, and shows how our
elicitation methods capture some things, but we need the evaluation steps!
• One funny outcome: – “The document takes up too much space – we shouldn’t have to have it
up.”
Implementation
• Taking our low-fidelity prototype, we moved into an implementation phase
• Web application using a variety of technologies with the CodeIgniter framework as the backbone to speed up development
• Working application went live in October, 2013 with about 400 documents being available from the rest of the toolchain
Evaluation
• Conducted a collaborative heuristic evaluation (CHE) on the prototype with 3 of our most brutal evaluations– Highest praise ever from one of them:
• “Actually, that wasn’t too bad guys.”
• About 20 problems ranging in the cosmetic to major range, with no universally agreed usability catastrophes– This makes for very happy developers
Evaluation (2)
• Conducted a user evaluation with 14 people (hopefully more to come)
• Initial group 4 senior researchers and a group of 11 PhD students in medieval history
• Each was given the opportunity to learn the interface and then answer questions that required a bit of research using the tool about one of our document sets
Evaluation (3)
• Overall the participants were able to complete the tasks and successfully answer the questions with near 100% accuracy
• They thought the workbench was a good step forward in trying to provide a way to get into the documents – more support needed in recording relationships that were found (not surprising)
• Really interesting – they saw right through the data mining right away!
Lessons Learned• About interdisciplinary work
– The gap between science and humanities is much larger than we expected
– The difference in training is important – there is very little shared language to build from
• About “technology think”– There is a major danger to move down to the implementation
level – for technology people it is comfortable and for the potential users it is arcane
– Technology people can often push users into solutions that are not appropriate
– Even within technology partnerships there remains gaps between interaction designers and “hard core” technologists
Lessons Learned (2)
• About interaction design– Elicitation techniques like contextual inquiry
are good to start with – but this project emphasizes how important evaluation is to refining early prototypes
– A lot of what is done in the historians heads is about building relationships, and essentially building up mental models of the past – there isn’t a ton of design work doing that well
Lessons Learned (3)
• About HCI as a contributor to digital cultural heritage– We have a unique perspective that other
technology people don’t have – we know how and want to understand the users
– We have techniques that no one else has to understand these complex domains
– DCH is an almost untouched field in terms of good interfaces – we NEED to be there to avoid the types of problems we see in all other systems that are driven by technology and not the user
Late Breaking News!
• We got another Digging into Data project just last week!
• It will be working with archaeology this time!
Late Breaking News!
• What will actually happen is that we are working with the Archaeological Data Service to increase access and use of their picture libraries
• Expect projects from me coming up in February on interaction with picture archives
Questions?
Resources
• Poole, Nick (2010). The Cost of Digitising Europe’s Cultural Heritage, A Report for the Comité des Sages of the European Commission, Collections Trust. http://ec.europa.eu/information_society/activities/digital_libraries/doc/refgroup/annexes/digiti_report.pdf
• Beaudouin-Lafon, M. (2000, April). Instrumental interaction: an interaction model for designing post-WIMP user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 446-453). ACM.