Session 1: Plenary

Themes in Discovery Informatics

Science Has a Never-ending Thirst for Technology

Computing as a substrate for science in innovative ways

Ongoing investments in cyberinfrastructure have a tremendous impact in scientific discoveriesShared high end instrumentsHigh performance computingDistributed servicesData managementVirtual organizations

These investments are extremely valuable for science, but do not address many aspects of science

Further Science NeedsEmphasis has been on data and computation, not so

much on models Need to support model formulation and testing is missing Models should be related to data (observed or simulated)

Emphasize insight and understandingFrom correlations to causality and explanation

Developing tools for the full discovery process and using tools for the discovery process

Tools that help you do new things vs tools that help you do things better

Further Science NeedsMany aspects of the scientific process could be

improved Some are not addressed by CI (eg literature search, reasoning

about models) Others could benefit from new approaches (eg capturing

metadata)

Effort is significant Many scientists do not have the resources or inclination to

benefit from CI How do you create a culture in which science stays timely in

its use of CI? Discipline-specific services make it harder to cross bounds Methods and process for being able to work with scientists

Further Science NeedsIntegration is important and far from being a

solved problem Integration across science domains Integration within a domain

Connecting tools and technologies to the practice of scienceMost science is done local, need to respond accordingly

(e.g., how do you support your student, get tenure)How to reduce the impedance mismatch between

cognition and practiceThe “long tail” of science – most of science is not big

science nor big dataCI can transform all elements of the discovery timeline

Further Science NeedsUser-centered design

Usability

Functionality

What are metrics for successAdoption by others?

Characterization of domains and facets that impact discovery informatics is still not understoodYou can’t get this by asking the scientistsWhat are equivalent classes of domains as they pertain

to CI

Need to treat domain scientists, social scientists, and computer scientists on equal footing

Emerging Movement?A movement for scientist-centered system design?

A movement to focus on the “human processor bottleneck”? Human cognitive capacity is flat (or at best getting slightly linearly),

while other dimensions of computing have grown exponentially

A movement for non-centralized science? (“long tail” of science (on multiple dimensions) aka “dark matter” of science; small science vs big; small data vs large)

A movement to improve the use of mundane technology in science practice?

A movement to lower the learning curve in infrastructure? There will be some curve, but it is smaller and the same no matter

what you need to accesseg web infrastructure is a good example

What is Discovery Informatics We should come back to a definition later in the meeting

Some possible defining characteristics: Small data science still has a major role to play

Complements big data science Much of science is largely local

Complements science at larger scales Big data science can be seen as a movement to more centralized science

The “long tail” of scientists are still largely underserved The “long tail” of scientific questions still has rudimentary technology

Spreadsheets are still in widespread use Many valuable datasets are never integrated to address aggregate questions

Discovery is a social endeavor Socio-technical systems to support ad-hoc collaborations Enable routine unexpected or indirect interactions among scientists

eg, unanticipated data sharing

DI: Automating and enhancing scientific processes at all levels?

DI: Empowering individual researchers through local infrastructure?

Do Scientific Discoveries Result from Special Kinds of Scientific

Activities?Perhaps, but we do not need to address this question

if we can agree to consider discoveries in a continuumThe more the scientific processes are improved, the

more the discovery processes are improvedThe more we empower scientists to cope with more

complex models (larger scope, broader coverage), the more the discovery processes are improved

The more we open access of potential contributors to scientific processes, the more the discovery processes are improved

Discovery Informatics: Why Now

Discovery informatics as “multiplicative science”: Investments in this area will have multiplicative gains as they will impact all areas of science and engineeringMultiplicative in the dimension of the “human bottleneck”Could address current redundancy in {bio|geo|eco|…}informatics

Discovery informatics will empower the public: Society is ready to participate in scientific activities and discovery tools can capture scientific practices “Personal data” will give rise to “personal science”

I study my genes, my medical condition, my backyard’s ecosystem

Volunteer donations of funds and time are now commonplace Enable donations of more intellectual contributions and insights

Discovery informatics will enable lifelong learning and training of future workforce in all areas of scienceFocuses on usable tools that encapsulate, automate, and disseminate

important aspects of state-of-the-art scientific practice

Discovery Informatics: Why Now

Scope to include engineering, medicineScience too big to fit in your head all at one time

Need computation to help understand itCurrent process of conducting science in all areas is

utterly broken, often reinventing processes year after yearScience are more willing to adopt and collaborate

Three Major Themes in Discovery Informatics

IN THIS SESSION:

For each theme:1. Why important to

discuss

2. State of the art (where is it published)

3. Topics

Focus is on coming up as a group with topics that each breakout should elaborate Bring up a topic not

yet listed but do not dwell on it

THEME 1: Improving the Experimentation and Discovery

ProcessUnprecedented complexity of scientific enterprise

Is science stymied by the human bottleneck?

Data collection and analysis through integrated robotics

Data sharing through Semantic Web

Cross-disciplinary research through collaborative interfaces

Result understanding through visualization

Managing publications through natural language technologies

Capturing current knowledge through ontologies and models

Multi-step data analysis through computational workflows

Process reproducibility and reuse through provenance

What aspects of the process could be improved, e.g.:

THEME 2: Learning Models from Science Data

Complexity of models and complexity of data analysisData analysis activities placed in a larger context

Using models to drive data collection activities

Preparing data in service of model formation and hypothesis testing

Selecting relevant features for model development

Highlighting interesting behaviors and unusual results

Comprehensive treatment of data to models to hypotheses cycle

THEME 3: Social Computing for Science

Multiplicative gains through broadening participationSome challenges require it, others can

significantly benefit

What scientific tasks could be handled

How can tasks be organized to facilitate contributions

Can reusable infrastructure be developed

Can junior researchers, K-12 students, and the public take more active roles in scientific discoveries

Managing human contributions

Three Major Themes

Improving the Discovery Process:

Why Characterizing what the discovery process is

Current processes are in many ways inefficient / less effectiveManual data analysisReproducibility is too costlyLiterature is vast and unmanageable…

Improving the Discovery Process:

What is the State of the ArtWorkflow systems

Automate many aspects of data analysis, make it reproducible/reusable

Emerging provenance standards (OPM, W3C’s PROV) Augmenting scientific publications with workflows

Creating knowledge bases from publications Ontological annotations of articles including claims and evidence Text mining to extract assertions to create knowledge bases Reasoning with knowledge bases to suggest or check hypotheses

Visualization 3 separate fields: scientific visualization, information visualization,

and visual analytics “design studies” Combining visualizations with other data

Improving the Discovery Process:

What is the State of the ArtWhat is the state of the art of what’s currently used in

science?

Opening data and models

Visualization not just of data, but also models and relationships between models

Improving the Discovery Process:

Discussion Topics (I)Automation of discovery processes

What is possible and unlikely in near/longer termRepresentations are key to discovery, hard to engineer

change of representation in a systemChallenge is to find the right division of labor between

human and computer

User-centered design Automation should come with suitable explanations

Of processes, models, data, etc.

Designing tools for the individual scientist (the “long tail”)

Improving the Discovery Process:

Discussion Topics (II)Workflows

Understand barriers to widespread practiceHave they reached the tipping point of usability vs pain?

Workflow reuse across labs, across workflow systemsAre workflows useful?What can we learn from workflows in non-science

domains?

Text extraction / generation

Annotating publications

Improving the Discovery Process:

Discussion Topics (III)Visualizations could help maximize the bandwidth of what

humans can assimilate

Visualization Do scientists know what they want?

Scientists seem to prefer interaction, ie, control over the visualization, rather than automatic visualizations

Active co-creation of visualization helps scientistsDomain specification / requirements extraction

Centrality of knowledge representations (means to an end) Data Processes Reuse, open access, dynamic Enabling integrated representation, reasoning, and learning Risk of not being pertinent to some areas of science

From Models to Data and Back Again:

Why Need to integrate better data with models and sense-

makingSemantic integration to enable reasoningLinking claims to experimental designs to data Interpreting data is a cognitive social process, aided by

visualizations that integrate context into the data

How do we integrate prior knowledge, formalisms scientists use, how do we update knowledge/formalisms

Generating useful data is a bottleneck, generating lots of models is easy, should leverage this

Need to help scientists to evaluate models

Learning “Models” from Data:

What is the State of the Art Cognitive science studies of discovery and insight

The role of effective problem representations The challenges of programming representation change

Computational discovery

Model-based reasoning

Causality Temporal dependency analysis

Design of quasi-experiments

Spatial and temporal data Variability, multi-scale,

Sensor noise Quality control Sensor noise vs actual phenomena

Learning Models from Data:

Discussion Topics (I) Integrating better models/knowledge and data

Model-guided data collection Collect data based on goals

Observations guiding the revision of models Explaining findings and revising models and knowledge Visualizations that combine models and data

Deriving stuff from data Enable causal connections across diverse data sources Causal relations co-existing with gaps and conflicts stands in the way to more

unified databases

Models / patterns / laws?

Importance of uncertainty, quality, utility

From models to use

Connecting computer simulations and model building from data HPC, simulation, and modeling from data should be connected

Learning Models from Data:

Discussion Topics (II)Learning models that are communicable

Potential for unifying models and associated tools for doing so

ML has a lot of theoretical results that have not yet been made useful more broadlyNeed to be more usable/accessible

Particularly in social sciencesNot always easy to apply to big data

Learning Models from Data:

Discussion Topics (III)Incentivizing digital resource sharing to enable

discoveries

Privacy and security: data being misused or not appropriately credited

The social sciences are a particularly promising area for discovery informatics, and what would facilitate this

Digital resource curation as a social issue

Verification (of models, conclusions, data, explanations, etc.)

Social Computing:

Why Many valuable datasets lack appropriate metadata

Labels, data characteristics and properties, etc.

Human computation has beaten best of breed algorithms

Social agreement accelerates data sharing

Public interest in participating in scientific activity

Community assessment of models, knowledge, etc. Concretizing elements that were mushy in the past

Mixed-initiative processes – humans exceed machine in many areas, so we need to assimilate them for the things that they do better

Harness knowledge about what makes online communities (including, e.g., Wikipedia) work well or poorly

Role of incentives, motivation, in bringing people together to do science

Social Computing:

What is the State of the ArtVery different manifestations:

Collecting data (eg pictures of birds)Labeling data (eg Galaxy Zoo)Computations (eg Foldit)Elaborate human processes (eg theorem proving)Bringing people and computing together in

complementary ways

Social Computing:

Discussion Topics (I)Several names: is there a distinction

Crowdsourcing, citizen science,

Designing the system Roles: peers, senior researchers, automation Incentives Training

Platforms and infrastructure (using clouds right, social web platforms)

Incorporating semantic information and metadata

Expertise finding

New modalities for peer review, scholarly communication

Social Computing:

Discussion Topics (II)Defining workflows with more elaborate processes

that mix human processing with computer processingHumans to do more complex tasksCan facilitate reproducibility

Enticing people to participate while ensuring quality

Some existing systems should be revisited to be designed as social systemsWorkflow libraries and reuse tools Data curation toolsOpen software

Social Computing:

Discussion Topics (III)Systems that enable collaborations that are not

deliberate but ad-hocOpportunistic partnershipsUnexpected uses of data

Systems that support a marketplace of ideas and track creditNew ideas/discoveries are often seen as a threat to the

status quo, how do we facilitate integrationEmpower people to share ideas on a problem while

credited

Incentive structures for new models of scholarly communication, such as blogs