experiment on knowledge acquisition

AuraWikiSubject Matter Expert - User Experience

Review

Jesse Wang, Wil Smith

Oct 2012

Project Goals“Crowd Source UT Authoring”

Can Subject Matter Experts Author Useful Universal Truths?Can We Speed Up Encoding a Textbook with Input from Subject Matter Experts?Can We Create a UT Authoring Portal for Multiple Textbooks?Can Existing Social Networks Provide Subject Matter Experts Capable of UT Authoring?Could Gamification be Applied to An Existing Portal to Add Non-Domain Experts?

Meeting GoalsThis is an end user meeting and we are doing it to hear your comments on the current system.

The team wants to explain the authoring process, and demonstrate the current application state.

Before we continue development on the project we need to review how the application, in the unfinished state, performs with our targeted user base.

MID PROJECT FEEDBACK. We do these meetings when releasing production products to ensure the final application complies with customer needs and is not “the engineering best way”.

Review SchedulePhase Method Time required

Introduction & Training • Goals of Project Halo and Aura Wiki

• Training for UT Authoring• What is a UT?• What is a concept /

context?• How do I formulate UTs

• System demonstration & hands-on

0:45

Assignment: UT Authoring

• You work by yourself using our system

• Support is provided by us

1:30

Questionnaire & Discussion

• Online questionnaire• Group discussion

0:45

How do we author a universal truth?

Subject Matter Expert - User Experience Review

From Sentence to Universal TruthsWhat is a UT? How to write UTs for a sentenceWhat it a concept? How to choose concepts for a UT

What is a UT?There is no exact definition of “Universal Truth”, but a good UT…

expresses a single fact (which can be compound, but must be indivisible).

is as simple as possible, but not simpler (to quote Einstein).

is context-independent, i.e. meaningful on its own, without surrounding sentences or other context.

is universally true. For example, a UT could start with "All Plants..." but not with "Some plants...".

is unambiguous.

is precise.

How to Write UTs for a Sentence

For each relevant sentence, write one or more UTs

There is no one correct set of UTs for a given sentence. The source sentences are open to interpretation, and it is up to you to determine what the most appropriate UTs are

The interpretation of the source sentences, and the paraphrasing into UTs, should be guided by your knowledge of the subject matter (and by common sense)

The following methods are not exhaustive. When in doubt, refer back to the definition of a “good UT”.


In the simplest case, the sentence has only one UT, and it is exactly the same as the UT.

Sentence: Matter is made up of elements.

UT: Matter is made up of elements.


Sometimes part of a sentence is irrelevant, even though the sentence as a whole has been deemed relevant. In such cases the irrelevant part is simply dropped.

Sentence: Some symbols are derived from Latin or German; for instance, the symbol for sodium is Na.

UT: Sodium has the symbol Na.


Often, a sentence contains several facts. In this case, the sentence needs to be split into several UTs.

Sentence: Pure sodium is a metal, and pure chlorine is a poisonous gas.

UT: Pure sodium is a metal. UT: Pure chlorine is a poisonous gas.


When a reference like "this", "that", "these", "those", "which", etc occurs in a source sentence, they should be replaced with the referent (the word that "this","that", etc refers to).

Sentence: Organisms are composed of matter, which is defined as anything that takes up space and has mass

UT: Organisms are composed of matterUT: Matter has mass and volume

Here, "which" clearly refers to "Organisms", and "is defined as" is unnecessarily verbose, and thus simplified.


Sometimes the referent is in a previous sentence in the same paragraph.

Sentence: These selective channels affect the rate at which water moves osmotically across the membrane

UT: Aquaporins are selective channelsUT: Aquaporins affect the rate at which water moves osmotically across the membrane


There are many other words that are used as linguistic flourishes or to bind sentences in a paragraph together and give the text some "flow". Such words ("another" in the example below) should be eliminated, but can sometimes be used to extract or infer "incidental" information from a sentence.

Sentence: Water (H20), another compound, consists of the elements hydrogen (H) and oxygen (O) in a 2:1 ratio

UT: Water consists of hydrogen and oxygen in a 2:1 ratioUT: Water is a compoundUT: Hydrogen has symbol HUT: Oxygen has symbol O

Here, the first UT contains the "main" information of the sentence, and the last three are "incidental".


Ideally, a UT should not contain words like "sometimes", "usually", "most", "some", "generally", etc. There are a couple of ways to handle them. One way is to generalize the weaker statement into a universal statement, usually just by dropping the word in question.

Sentence: Branching generally enables plants to harvest sunlight for photosynthesis more effectively.

UT: Branching enables plants to harvest sunlight for photosynthesis more effectively.


Another method of eliminating "some" etc is to instantiate the sentence by providing a name for the case where the sentence is true.

Sentence: Some prokaryotes even contain a row of tiny magnetic particles that allow the cells to orient in Earth's magnetic field

UT: Magnetotactic bacteria contain a row of tiny magnetic particles that allow the cells to orient in Earth's magnetic field

Here we replaced "some prokaryotes" with "magnetotactic bacteria". We only do this when the new name occurs somewhere in the textbook.


"some", etc are allowed when they do not refer to the main concept of the sentence (see "Choosing Concepts for a UT", below). For example, this is a good UT:

UT: In vascular plants, some substances may use more than one route

The UT as a whole states a fact about all vascular plants (the main concept), and "some" refers to substances within all vascular plants. One way to remember this distinction is to keep in mind that Universal Truths are just that -- universally true statements, i.e. statements that are always true of some type of entity, event, etc. If it is possible, we can also use Instantiation (see previous slide) in this type of case.


When "for example", "such as", etc occurs in a sentence, encode the specific information in the example.

Sentence: Some plants have horizontally oriented leaves; others, such as grasses, have leaves that are vertically oriented.

UT: Grass has vertically oriented leaves

As discussed above, the "some" part of the sentence is problematic.


In other cases, the "for example" does not add any new information.

Paragraph: We symbolize atoms with the same abbreviation used for the element that is made up of those atoms. For example, the symbol C stands for both the element carbon and a single carbon atom.

Here, the second sentence does not add any information, since its informational content is subsumed by the general "rule" of the first sentence (assuming we have already encoded that carbon has the symbol C).


Conditional statements should usually be rephrased to a non-conditional form.

Sentence: If we place a cell in a solution that is hypotonic to the cell, water will enter the cell faster than it leaves

UT: A cell in a hypotonic solution will gain water faster than it loses it

Here, the condition has been changed into a context, "in a hypotonic solution". See also the section on "Contextual Universals" below.

Review: What is a UT?

Or rather a good UT…

expresses a single fact (which can be compound, but must be indivisible).

is as simple as possible

is context-independent, i.e. meaningful on its own, without surrounding sentences or other context.

is universally true. For example, a UT could start with "All Plants..." but not with "Some plants...".

is unambiguous.

is precise.

Summary: Methods of UT

AuthoringCopy, remove (partial, irrelevant info), splitting

Reference resolution (from context)

Incidental extraction e.g. “Water, another compound, …”

Specialization, generalization

InstantiationE.g. “some prokaryotes” “magnetotactic bacteria”

Example-encodingE.g. “some plants…” “grass”

Condition to context

What is a Concept?

A concept is a noun or a noun phrase that describes a type of entity or event, in singular form

You may need to create a new concept for a UT if an appropriate one does not already exist.

Good concept names: Cell, Eukaryotic Cell, Cellular Respiration

Bad concept names: Cells, The eukaryotic cell, A plant

Choosing Concepts for a UT

The main concept of a UT is the entity or event that the UT is about.

It is usually the grammatical subject of the UT

UT: All plant cells have a nucleusMain concept: Plant Cell

Note that "All plant cells" is not a good concept, because it is not in singular form.

UT: Photosynthesis produces oxygenMain concept: Photosynthesis

Choosing Concepts for a UT

Some UTs are true in a specific context. In such cases, also choose a contextual concept –contextual concept is OPTIONAL

The contextual concept usually follows a word such as "in", "during", "while", etc.

UT: The phospholipids in a membrane move laterallyMain concept: PhospholipidContextual concept: MembraneNote that "the" or "a" is never part of the concept.

UT: The oxygen produced during photosynthesis is used during cellular respirationMain concept: oxygenContextual concept: Photosynthesis

UT: Cellular respiration in eukaryotes happens in the mitochondriaMain Concept: Cellular RespirationContextual Concept: Eukaryote

Bring your Best, then Move On

We hope you got good understanding of what you’re supposed to do

Give it your best shot, it does NOT need to be perfect

Do not spend too much time on one sentence/UT, move on if it’s too hard

Feel free to work on sentences or UT others started

Feel free to ask us if you’re really confused or not sure

Aura wiki tutorial


Our Solution: Aura Wiki

Contents:A few chapters of Campbell Biology

Without figures and using a simplified layout

PurposeUse the power of crowdsourcing and online collaboration

Let users „translate“ sentences from the textbook into UTs.

Sample wiki page for the textbook

Sample wiki page for a whole section

Sample wiki page for a sentence

Creating UTsThe translation of sentences to UTs is done on the sentence page.

The sentence page can display the paragraph surrounding the sentence (or even more context) if needed for understanding and formulating the UT.

Under the heading Universal Truths you can see all UTs that exist for this sentence and you can add yor own.

Sample UTs presented in the sentence page

Creating UTs – Add new UT

Creating UTs – Input Form

Input form for new UT. First two inputs

are required.

Creating UTs - Autocompletion

Autocompletion is automatically triggered and

suggests existing concepts

Subject Matter Expert

Assignment


The AssignmentNow it is your turn

Go to your User page

There is a list of sentences assigned to you

Once you have added UTs they will be displayed there together with the sentences (after refreshing the page)

At the top of your user page, you will find links to help-texts, some sample UTs and the questionnaire for after the session.

The AssignmentThis link always brings you back

to your user page

Sentences (and UTs)

Help documents and

a list of example UTs

The AssignmentIMPORTANT

This evaluation is conducted in order to assess the system, not to assess your work as a SME. You are supporting the evaluation, but you are not a test subject!

Please work with the system at your own convenience. You do not need to achieve any quantitative or qualitative goal. (This is of course no excuse for being lazy )

You have pencils and paper available – if you encounter any problems or have ideas you think are worth mentioning, feel free to take a short note so you can bring it up during the discussion later

If there are questions please ask them anytime

Subject Matter Expert Review


QuestionnaireGo to your user page

There is a link to the online questionnaire at the top

In the first question, select the username you were assigned today

Fill out the entire questionnaire. Don’t forget to submit at the end

Discussion – System

FunctionalityDid you have trouble following the UT authoring workflow? If so, where and why?

What changes would you apply to the existing functionality and/or workflows to increase productivity and ease of use?

What new functionality could you think of that might have a positive influence on your work?

Discussion – UT Authoring

Did you understand the notion of UT and concept?

Did the training sufficiently explain the meaning of UT and concept?

Do you feel the training you received in the beginning and the help materials on the site were sufficient for executing your task? If not, what would you improve? Which areas need better coverage?

Conclusion


Thank you.Contact: Jesse Wang ([email protected])

William Smith ([email protected] )

mailto:[email protected]

mailto:[email protected]

experiment on knowledge acquisition

Documents

relevant sentence

previous sentence

given sentence

thoughthe sentence

ut authoring portal

correct set of uts

pure sodium

source sentences