developing a generic process model for longitudinal data ......for longitudinal data collections...

26
3rd Annual European DDI Users Group Meeting (EDDI11) Gothenburg, December 5–6, 2011 Marcel Hebing German Socio-Economic Panel Study (SOEP), DIW Berlin and William Block CISER, Cornell University present joined work with Ingo Barkow, Jay Greenfield, Arofan Gregory, Larry Hoyle, Wolfgang Zenk-Möltgen from DDI Experts Workshop, Dagstuhl 2011 Developing a Generic Process Model for Longitudinal Data Collections

Upload: others

Post on 07-Oct-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

3rd Annual European DDI Users Group Meeting (EDDI11)Gothenburg, December 5–6, 2011

Marcel HebingGerman Socio-Economic Panel Study (SOEP), DIW Berlin

and

William BlockCISER, Cornell University

present joined work with Ingo Barkow, Jay Greenfield,Arofan Gregory, Larry Hoyle, Wolfgang Zenk-Möltgen

from DDI Experts Workshop, Dagstuhl 2011

Developing a Generic Process Modelfor Longitudinal Data Collections

Page 2: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Experts-Workshop, Dagstuhl 2011

3 Working Groups

1. Foundational Metadata

2. How to

3. GLBPM

+ Glossary

Page 3: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Previous Work

Tornado Model (Dagstuhl 2010)

DDI Data Life Cycle

GSBPM

Page 4: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

DDI Data Life Cycle

Page 5: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Tornado Model

Page 6: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

GSBPMhttp://www.unece.org/stats/gsbpm.html

Page 7: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Mapping GLBPM against DDI Data Lifecycle

Page 8: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Generic Longitudinal Business Process Model

Business Processes are structured activities in the context of labor division, to increase

effectivity and efficiency.

A Business Process Model describes Business Processes in a way, that helps to discuss

them and adapt Software.

Generic means, that the model doesn't have to match exactly one case, but is designed to be

adaptable to many cases. The more precisely you get, the less cases will fit.

Longitudinal studies have specific needs, as you run most parts of the underlying business

process in each wave. The reuse of work from previous waves becomes important.

Page 9: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Generic Longitudinal Business Process Model

9 high level categories

broken down into 52 associated sub-elements.

This categorization may be useful to individuals or projects engaged three main activities:

Project Management or Quality Management,Metadata Management, Using External

Standard Metadata (classifications, concepts, questions, variables, etc.)

It's legal to re-arrange the elements and their sub-steps.

Page 10: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 2 3 4 5 6 7 8 9

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

GLBPM: High Level View

Page 11: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 2 3 4 5 6 7 8 9

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

GSBPM:

GLBPM:

Data Lifecycle:

Page 12: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 2 3 4 5 6 7 8 9

1.1Define research

questions, universe & high-level concepts

1.2Evaluate existing

data & publications

1.3Establish outputs

& needed infrastructure

1.4Define specific concepts to be

measured

1.5Planning, time-table & needed infrastructure

1.6Prepare proposal

& get funding

2.5Specify processing / cleaning methods

2.3Design collection

instruments

2.4Specify data

elements

2.6Organize research

team

2.7Design

infrastructure

2.1Identify sources

2.2Design sampling

methods

3.5Finalize production

systems

3.3Validate

instruments

3.4Test production

systems

3.1Develop data

collection instruments

3.2Create or enhance

infrastructure components

4.3Run collection

4.4Finalize collection

4.1Select sample

4.2Set up collection

5.5Construct new

variables and units

5.3Explore, validate &

clean data

5.4Impute missing

data

5.6Calculate weights

5.7Calculate

aggregates

5.1Integrate data

5.2Classify & code

6.3Preserve data /

metadata

6.1Ingest data /

metadata

6.2Enhance metadata

7.5Provide data

citation support

7.3Deploy access control system /

policies

7.4Promote

dissemination products

7.6Enhance data

discovery

7.7Manage user

support

7.1Deploy release infrastructure

7.2Prepare

dissemination products

8.5Prepare research

papers

8.3Prepare data

8.4Analyse data

8.6Manage disclosure

risk

8.7Publish research

8.1Discovery of data

and relevant research

8.2Access data

9.3Conduct

evaluation

9.4Determine future

actions

9.1Establish

evaluation criteria

9.2Gather evaluation

inputs

5.8Anonymize data

5.9Finalize data

outputs

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

Page 13: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 2 3 4 5 6 7 8 9

1.1Define research

questions, universe & high-level concepts

1.2Evaluate existing

data & publications

1.3Establish outputs

& needed infrastructure

1.4Define specific concepts to be

measured

1.5Planning, time-table & needed infrastructure

1.6Prepare proposal

& get funding

2.5Specify processing / cleaning methods

2.3Design collection

instruments

2.4Specify data

elements

2.6Organize research

team

2.7Design

infrastructure

2.1Identify sources

2.2Design sampling

methods

3.5Finalize production

systems

3.3Validate

instruments

3.4Test production

systems

3.1Develop data

collection instruments

3.2Create or enhance

infrastructure components

4.3Run collection

4.4Finalize collection

4.1Select sample

4.2Set up collection

5.5Construct new

variables and units

5.3Explore, validate &

clean data

5.4Impute missing

data

5.6Calculate weights

5.7Calculate

aggregates

5.1Integrate data

5.2Classify & code

6.3Preserve data /

metadata

6.1Ingest data /

metadata

6.2Enhance metadata

7.5Provide data

citation support

7.3Deploy access control system /

policies

7.4Promote

dissemination products

7.6Enhance data

discovery

7.7Manage user

support

7.1Deploy release infrastructure

7.2Prepare

dissemination products

8.5Prepare research

papers

8.3Prepare data

8.4Analyse data

8.6Manage disclosure

risk

8.7Publish research

8.1Discovery of data

and relevant research

8.2Access data

9.3Conduct

evaluation

9.4Determine future

actions

9.1Establish

evaluation criteria

9.2Gather evaluation

inputs

5.8Anonymize data

5.9Finalize data

outputs

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

Page 14: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 3 4 5 6 7 8 9

1.1Define research

questions, universe & high-level concepts

1.2Evaluate existing

data & publications

1.3Establish outputs

& needed infrastructure

1.4Define specific concepts to be

measured

1.5Planning, time-table & needed infrastructure

1.6Prepare proposal

& get funding

2.5Specify processing / cleaning methods

2.3Design collection

instruments

2.4Specify data

elements

2.6Organize research

team

2.7Design

infrastructure

2.1Identify sources

2.2Design sampling

methods

3.5Finalize production

systems

3.3Validate

instruments

3.4Test production

systems

3.1Develop data

collection instruments

3.2Create or enhance

infrastructure components

4.3Run collection

4.4Finalize collection

4.1Select sample

4.2Set up collection

5.5Construct new

variables and units

5.3Explore, validate &

clean data

5.4Impute missing

data

5.6Calculate weights

5.7Calculate

aggregates

5.1Integrate data

5.2Classify & code

6.3Preserve data /

metadata

6.1Ingest data /

metadata

6.2Enhance metadata

7.5Provide data

citation support

7.3Deploy access control system /

policies

7.4Promote

dissemination products

7.6Enhance data

discovery

7.7Manage user

support

7.1Deploy release infrastructure

7.2Prepare

dissemination products

8.5Prepare research

papers

8.3Prepare data

8.4Analyse data

8.6Manage disclosure

risk

8.7Publish research

8.1Discovery of data

and relevant research

8.2Access data

9.3Conduct

evaluation

9.4Determine future

actions

9.1Establish

evaluation criteria

9.2Gather evaluation

inputs

5.8Anonymize data

5.9Finalize data

outputs

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

2

Page 15: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 2 4 5 6 7 8 9

1.1Define research

questions, universe & high-level concepts

1.2Evaluate existing

data & publications

1.3Establish outputs

& needed infrastructure

1.4Define specific concepts to be

measured

1.5Planning, time-table & needed infrastructure

1.6Prepare proposal

& get funding

2.5Specify processing / cleaning methods

2.3Design collection

instruments

2.4Specify data

elements

2.6Organize research

team

2.7Design

infrastructure

2.1Identify sources

2.2Design sampling

methods

3.5Finalize production

systems

3.3Validate

instruments

3.4Test production

systems

3.1Develop data

collection instruments

3.2Create or enhance

infrastructure components

4.3Run collection

4.4Finalize collection

4.1Select sample

4.2Set up collection

5.5Construct new

variables and units

5.3Explore, validate &

clean data

5.4Impute missing

data

5.6Calculate weights

5.7Calculate

aggregates

5.1Integrate data

5.2Classify & code

6.3Preserve data /

metadata

6.1Ingest data /

metadata

6.2Enhance metadata

7.5Provide data

citation support

7.3Deploy access control system /

policies

7.4Promote

dissemination products

7.6Enhance data

discovery

7.7Manage user

support

7.1Deploy release infrastructure

7.2Prepare

dissemination products

8.5Prepare research

papers

8.3Prepare data

8.4Analyse data

8.6Manage disclosure

risk

8.7Publish research

8.1Discovery of data

and relevant research

8.2Access data

9.3Conduct

evaluation

9.4Determine future

actions

9.1Establish

evaluation criteria

9.2Gather evaluation

inputs

5.8Anonymize data

5.9Finalize data

outputs

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

3

Page 16: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 2 3 4 5 6 7 8 9

1.1Define research

questions, universe & high-level concepts

1.2Evaluate existing

data & publications

1.3Establish outputs

& needed infrastructure

1.4Define specific concepts to be

measured

1.5Planning, time-table & needed infrastructure

1.6Prepare proposal

& get funding

2.5Specify processing / cleaning methods

2.3Design collection

instruments

2.4Specify data

elements

2.6Organize research

team

2.7Design

infrastructure

2.1Identify sources

2.2Design sampling

methods

3.5Finalize production

systems

3.3Validate

instruments

3.4Test production

systems

3.1Develop data

collection instruments

3.2Create or enhance

infrastructure components

4.3Run collection

4.4Finalize collection

4.1Select sample

4.2Set up collection

5.5Construct new

variables and units

5.3Explore, validate &

clean data

5.4Impute missing

data

5.6Calculate weights

5.7Calculate

aggregates

5.1Integrate data

5.2Classify & code

6.3Preserve data /

metadata

6.1Ingest data /

metadata

6.2Enhance metadata

7.5Provide data

citation support

7.3Deploy access control system /

policies

7.4Promote

dissemination products

7.6Enhance data

discovery

7.7Manage user

support

7.1Deploy release infrastructure

7.2Prepare

dissemination products

8.5Prepare research

papers

8.3Prepare data

8.4Analyse data

8.6Manage disclosure

risk

8.7Publish research

8.1Discovery of data

and relevant research

8.2Access data

9.3Conduct

evaluation

9.4Determine future

actions

9.1Establish

evaluation criteria

9.2Gather evaluation

inputs

5.8Anonymize data

5.9Finalize data

outputs

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

Page 17: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 2 3 4 5 6 7 8 9

1.1Define research

questions, universe & high-level concepts

1.2Evaluate existing

data & publications

1.3Establish outputs

& needed infrastructure

1.4Define specific concepts to be

measured

1.5Planning, time-table & needed infrastructure

1.6Prepare proposal

& get funding

2.5Specify processing / cleaning methods

2.3Design collection

instruments

2.4Specify data

elements

2.6Organize research

team

2.7Design

infrastructure

2.1Identify sources

2.2Design sampling

methods

3.5Finalize production

systems

3.3Validate

instruments

3.4Test production

systems

3.1Develop data

collection instruments

3.2Create or enhance

infrastructure components

4.3Run collection

4.4Finalize collection

4.1Select sample

4.2Set up collection

5.5Construct new

variables and units

5.3Explore, validate &

clean data

5.4Impute missing

data

5.6Calculate weights

5.7Calculate

aggregates

5.1Integrate data

5.2Classify & code

6.3Preserve data /

metadata

6.1Ingest data /

metadata

6.2Enhance metadata

7.5Provide data

citation support

7.3Deploy access control system /

policies

7.4Promote

dissemination products

7.6Enhance data

discovery

7.7Manage user

support

7.1Deploy release infrastructure

7.2Prepare

dissemination products

8.5Prepare research

papers

8.3Prepare data

8.4Analyse data

8.6Manage disclosure

risk

8.7Publish research

8.1Discovery of data

and relevant research

8.2Access data

9.3Conduct

evaluation

9.4Determine future

actions

9.1Establish

evaluation criteria

9.2Gather evaluation

inputs

5.8Anonymize data

5.9Finalize data

outputs

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

Page 18: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 2 3 4 5 6 7 8 9

1.1Define research

questions, universe & high-level concepts

1.2Evaluate existing

data & publications

1.3Establish outputs

& needed infrastructure

1.4Define specific concepts to be

measured

1.5Planning, time-table & needed infrastructure

1.6Prepare proposal

& get funding

2.5Specify processing / cleaning methods

2.3Design collection

instruments

2.4Specify data

elements

2.6Organize research

team

2.7Design

infrastructure

2.1Identify sources

2.2Design sampling

methods

3.5Finalize production

systems

3.3Validate

instruments

3.4Test production

systems

3.1Develop data

collection instruments

3.2Create or enhance

infrastructure components

4.3Run collection

4.4Finalize collection

4.1Select sample

4.2Set up collection

5.5Construct new

variables and units

5.3Explore, validate &

clean data

5.4Impute missing

data

5.6Calculate weights

5.7Calculate

aggregates

5.1Integrate data

5.2Classify & code

6.3Preserve data /

metadata

6.1Ingest data /

metadata

6.2Enhance metadata

7.5Provide data

citation support

7.3Deploy access control system /

policies

7.4Promote

dissemination products

7.6Enhance data

discovery

7.7Manage user

support

7.1Deploy release infrastructure

7.2Prepare

dissemination products

8.5Prepare research

papers

8.3Prepare data

8.4Analyse data

8.6Manage disclosure

risk

8.7Publish research

8.1Discovery of data

and relevant research

8.2Access data

9.3Conduct

evaluation

9.4Determine future

actions

9.1Establish

evaluation criteria

9.2Gather evaluation

inputs

5.8Anonymize data

5.9Finalize data

outputs

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

Page 19: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 2 3 4 5 6 7 8 9

1.1Define research

questions, universe & high-level concepts

1.2Evaluate existing

data & publications

1.3Establish outputs

& needed infrastructure

1.4Define specific concepts to be

measured

1.5Planning, time-table & needed infrastructure

1.6Prepare proposal

& get funding

2.5Specify processing / cleaning methods

2.3Design collection

instruments

2.4Specify data

elements

2.6Organize research

team

2.7Design

infrastructure

2.1Identify sources

2.2Design sampling

methods

3.5Finalize production

systems

3.3Validate

instruments

3.4Test production

systems

3.1Develop data

collection instruments

3.2Create or enhance

infrastructure components

4.3Run collection

4.4Finalize collection

4.1Select sample

4.2Set up collection

5.5Construct new

variables and units

5.3Explore, validate &

clean data

5.4Impute missing

data

5.6Calculate weights

5.7Calculate

aggregates

5.1Integrate data

5.2Classify & code

6.3Preserve data /

metadata

6.1Ingest data /

metadata

6.2Enhance metadata

7.5Provide data

citation support

7.3Deploy access control system /

policies

7.4Promote

dissemination products

7.6Enhance data

discovery

7.7Manage user

support

7.1Deploy release infrastructure

7.2Prepare

dissemination products

8.5Prepare research

papers

8.3Prepare data

8.4Analyse data

8.6Manage disclosure

risk

8.7Publish research

8.1Discovery of data

and relevant research

8.2Access data

9.3Conduct

evaluation

9.4Determine future

actions

9.1Establish

evaluation criteria

9.2Gather evaluation

inputs

5.8Anonymize data

5.9Finalize data

outputs

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

Page 20: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 2 3 4 5 6 7 8 9

1.1Define research

questions, universe & high-level concepts

1.2Evaluate existing

data & publications

1.3Establish outputs

& needed infrastructure

1.4Define specific concepts to be

measured

1.5Planning, time-table & needed infrastructure

1.6Prepare proposal

& get funding

2.5Specify processing / cleaning methods

2.3Design collection

instruments

2.4Specify data

elements

2.6Organize research

team

2.7Design

infrastructure

2.1Identify sources

2.2Design sampling

methods

3.5Finalize production

systems

3.3Validate

instruments

3.4Test production

systems

3.1Develop data

collection instruments

3.2Create or enhance

infrastructure components

4.3Run collection

4.4Finalize collection

4.1Select sample

4.2Set up collection

5.5Construct new

variables and units

5.3Explore, validate &

clean data

5.4Impute missing

data

5.6Calculate weights

5.7Calculate

aggregates

5.1Integrate data

5.2Classify & code

6.3Preserve data /

metadata

6.1Ingest data /

metadata

6.2Enhance metadata

7.5Provide data

citation support

7.3Deploy access control system /

policies

7.4Promote

dissemination products

7.6Enhance data

discovery

7.7Manage user

support

7.1Deploy release infrastructure

7.2Prepare

dissemination products

8.5Prepare research

papers

8.3Prepare data

8.4Analyse data

8.6Manage disclosure

risk

8.7Publish research

8.1Discovery of data

and relevant research

8.2Access data

9.3Conduct

evaluation

9.4Determine future

actions

9.1Establish

evaluation criteria

9.2Gather evaluation

inputs

5.8Anonymize data

5.9Finalize data

outputs

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

Page 21: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 2 3 4 5 6 7 8 9

1.1Define research

questions, universe & high-level concepts

1.2Evaluate existing

data & publications

1.3Establish outputs

& needed infrastructure

1.4Define specific concepts to be

measured

1.5Planning, time-table & needed infrastructure

1.6Prepare proposal

& get funding

2.5Specify processing / cleaning methods

2.3Design collection

instruments

2.4Specify data

elements

2.6Organize research

team

2.7Design

infrastructure

2.1Identify sources

2.2Design sampling

methods

3.5Finalize production

systems

3.3Validate

instruments

3.4Test production

systems

3.1Develop data

collection instruments

3.2Create or enhance

infrastructure components

4.3Run collection

4.4Finalize collection

4.1Select sample

4.2Set up collection

5.5Construct new

variables and units

5.3Explore, validate &

clean data

5.4Impute missing

data

5.6Calculate weights

5.7Calculate

aggregates

5.1Integrate data

5.2Classify & code

6.3Preserve data /

metadata

6.1Ingest data /

metadata

6.2Enhance metadata

7.5Provide data

citation support

7.3Deploy access control system /

policies

7.4Promote

dissemination products

7.6Enhance data

discovery

7.7Manage user

support

7.1Deploy release infrastructure

7.2Prepare

dissemination products

8.5Prepare research

papers

8.3Prepare data

8.4Analyse data

8.6Manage disclosure

risk

8.7Publish research

8.1Discovery of data

and relevant research

8.2Access data

9.3Conduct

evaluation

9.4Determine future

actions

9.1Establish

evaluation criteria

9.2Gather evaluation

inputs

5.8Anonymize data

5.9Finalize data

outputs

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

Page 22: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 2 3 4 5 6 7 8 9

1.1Define research

questions, universe & high-level concepts

1.2Evaluate existing

data & publications

1.3Establish outputs

& needed infrastructure

1.4Define specific concepts to be

measured

1.5Planning, time-table & needed infrastructure

1.6Prepare proposal

& get funding

2.5Specify processing / cleaning methods

2.3Design collection

instruments

2.4Specify data

elements

2.6Organize research

team

2.7Design

infrastructure

2.1Identify sources

2.2Design sampling

methods

3.5Finalize production

systems

3.3Validate

instruments

3.4Test production

systems

3.1Develop data

collection instruments

3.2Create or enhance

infrastructure components

4.3Run collection

4.4Finalize collection

4.1Select sample

4.2Set up collection

5.5Construct new

variables and units

5.3Explore, validate &

clean data

5.4Impute missing

data

5.6Calculate weights

5.7Calculate

aggregates

5.1Integrate data

5.2Classify & code

6.3Preserve data /

metadata

6.1Ingest data /

metadata

6.2Enhance metadata

7.5Provide data

citation support

7.3Deploy access control system /

policies

7.4Promote

dissemination products

7.6Enhance data

discovery

7.7Manage user

support

7.1Deploy release infrastructure

7.2Prepare

dissemination products

8.5Prepare research

papers

8.3Prepare data

8.4Analyse data

8.6Manage disclosure

risk

8.7Publish research

8.1Discovery of data

and relevant research

8.2Access data

9.3Conduct

evaluation

9.4Determine future

actions

9.1Establish

evaluation criteria

9.2Gather evaluation

inputs

5.8Anonymize data

5.9Finalize data

outputs

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

Page 23: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build/Rebuild

CollectProcess /Analyse

Archive /Preserve /

Curate

DataDissemination /

Discovery

Research /Publish

RetrospectiveEvaluation

1 2 3 4 5 6 7 8 9

1.1Define research

questions, universe & high-level concepts

1.2Evaluate existing

data & publications

1.3Establish outputs

& needed infrastructure

1.4Define specific concepts to be

measured

1.5Planning, time-table & needed infrastructure

1.6Prepare proposal

& get funding

2.5Specify processing / cleaning methods

2.3Design collection

instruments

2.4Specify data

elements

2.6Organize research

team

2.7Design

infrastructure

2.1Identify sources

2.2Design sampling

methods

3.5Finalize production

systems

3.3Validate

instruments

3.4Test production

systems

3.1Develop data

collection instruments

3.2Create or enhance

infrastructure components

4.3Run collection

4.4Finalize collection

4.1Select sample

4.2Set up collection

5.5Construct new

variables and units

5.3Explore, validate &

clean data

5.4Impute missing

data

5.6Calculate weights

5.7Calculate

aggregates

5.1Integrate data

5.2Classify & code

6.3Preserve data /

metadata

6.1Ingest data /

metadata

6.2Enhance metadata

7.5Provide data

citation support

7.3Deploy access control system /

policies

7.4Promote

dissemination products

7.6Enhance data

discovery

7.7Manage user

support

7.1Deploy release infrastructure

7.2Prepare

dissemination products

8.5Prepare research

papers

8.3Prepare data

8.4Analyse data

8.6Manage disclosure

risk

8.7Publish research

8.1Discovery of data

and relevant research

8.2Access data

9.3Conduct

evaluation

9.4Determine future

actions

9.1Establish

evaluation criteria

9.2Gather evaluation

inputs

5.8Anonymize data

5.9Finalize data

outputs

Project Management / Quality Management

Metadata Management

Use of External Standard Metadata (classifications, concepts, questions, variables)

Page 24: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Evaluate /Specify Needs

Design /Redesign

Build /Rebuild

Collect

Process /Analyse

Archive /Preserve /

Curate

DataDissemination/

Discovery

Research/Publish

RetrospectiveEvaluation

1

2

3

4

56

7

8

9

Stu

dy D

esig

n

Methodological

Report

InstrumentDocumentation

Raw

Data

& M

etadata

Pro

cess

ed D

ata

Physical

Data ProductsCitations &Publication

Assessm

ent Report

Circle View

Page 25: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

Outlook

Match DDI elements to sub-steps.

Evaluate for existing longitudinal studies.

Page 26: Developing a Generic Process Model for Longitudinal Data ......for Longitudinal Data Collections Experts-Workshop, Dagstuhl 2011 3 Working Groups 1. Foundational Metadata 2. How to

3rd Annual European DDI Users Group Meeting (EDDI11)Gothenburg, December 5–6, 2011

Marcel HebingGerman Socio-Economic Panel Study (SOEP), DIW Berlin

and

William BlockCISER, Cornell University

present joined work with Ingo Barkow, Jay Greenfield,Arofan Gregory, Larry Hoyle, Wolfgang Zenk-Möltgen

from DDI Experts Workshop, Dagstuhl 2011

Thank you.