AN ALTERNATIVE DESIGN MODEL FOR BUILDING

ELECTRONIC PERFORMANCE SUPPORT SYSTEMS

Kursat Cagiltay

Instructional Systems Technology and Cognitive ScienceIndiana University

July 15, 2002

What is EPSS?• EPSS is

– a computer based (electronic), – user controlled and easy to use system that provides

support at the moment it is needed (right time), – presents relevant (right type) and – context-focused (right amount) information that the

performer needs, – in the real work environment (right place)

• Components of EPSS– Database– Tools– Instructional component– Expert system

Why do we need a D/D model?• EPSSs are significantly different from instructional

systems. – Main focus is performing rather than learning– EPSS is used while doing the actual job, not beforehand– No need to follow a predetermined sequence

• Especially the focus on the "work process" makes the design and development of an EPSS quite different than the traditional models of instructional design. (Witt & Wager, 1994; Cole et al.,1997; Cote, 2000)

• There is a broad consensus that there is a lack of EPSS design and development models (Gustafson, 1993;

Gustafson, 2000; Milheim, 1997; Laffey, 1995; Rosenberg, Coscarelli & Hutchison, 1999)

Research Questions

• Given that the traditional models of instructional development are inappropriate for EPSS development, what are the primary distinctive characteristics of an effective and efficient EPSS design and development process?

• What should an effective and efficient design model incorporate for the development of an effective and efficient EPSS?

Methodology• Multiple case studies• Formative research methodology (Reigeluth & Frick, 1999)

– Naturalistic case study – Cases are not specifically designed according to

the theory – Cases are analyzed to see the failing points and

valuable elements• Two major types of naturalistic formative research

procedures– “In vivo” (during the process) and – “Post facto” (after the process)

• Research processSelect

several cases

collect and

analyze data on the cases

fully develop a

tentative model

Methodology

• Cases– A higher educational setting (in-vivo, post-

facto) – A car manufacturing company (post-facto and

in-vivo) – A telecommunications company (in-vivo)

Methodology• Data collection

– Observations • In-vivo only

– Interviews • D/D team members, end-users, managers

– Document Analysis • Web-sites, reports, e-mail logs, proposals

• Data Analysis– Continuous and iterative – Explored similarities and differences among the cases – Looked for: social, political and technical aspects of the

EPSS design/development, and process related issues

Results: The Underlying Principles of the Model

• Principles related to social issues 1. Identify and develop awareness and need

for EPSS

2. Understand political/cultural characteristics and history of the organization and the stakeholders

3. For active involvement establish good communication strategies

4. Explore and create mechanisms to increase motivation for active involvement

Results: The Underlying Principles of the Model

• Principles related to technical issues 5. Form a team with technically competent

members6. Focus on compatibility, portability and

standardization of the EPSS 7. Make early decisions about the best technology

needs 8. Design a seamless Performer-EPSS-Interaction

environment (Human-Computer-Interaction)9. Manage content and information overflow 10.Get help from the Information Systems discipline

Results: The Underlying Principles of the Model

• Principles related to process issues 11.Follow a dynamic EPSS design/development

process

12.Form a multi-skilled/multi-disciplined team

13.Create shared vision and make sure everybody believes it

14.Forecast risks in advance and develop a risk management mechanism

Two general patterns were observed

1. Cases were developed for complex socio-technical organizational environments, and the D/D processes have shown a dynamic nature.

2. Throughout the entire D/D process, the team members had to make changes and revisions of their plans to overcome problems: adaptive

The proposed EPSS D/D model is based on the 14 principles. It reflects these two patterns and received its name from them:

Adaptive Dynamic EPSS Model: ADEM

• ADEM includes – people/organization, – the technology that supports them, and – the processes that tie them all together with an

emphasis on peopleware (soft) issues• The focus was more on the generic design

process, with the major components that reflect the patterns of design activities carried out by EPSS designers in the field.

• The ADEM is a framework for portraying overall EPSS design and development. Shows more descriptive model characteristics than a step-by-step, prescriptive one

Two Foundations of the ADEM• The development of the ADEM was helped by

Chaos Theory. – Kellert (1993) defines Chaos Theory as, “qualitative

study of unstable aperiodic behavior in deterministic nonlinear dynamical systems” (p. 2)

• Information System (IS) development and EPSS development have many similarities, so if one side of the ADEM stands on Chaos Theory, the other side is based on the IS discipline

Conceptual Elements of Chaos

• Sensitive dependence on initial conditions

• Fractals: Fractal implies recursion, patterns inside of patterns (feedback between parts or levels of the system )

• Strange Attractors (choices of behavior)

• It (and ADEM) originates from nature

• How?

Visual representation of the ADEM inspired from fractal geometry

Mandelbrot-Julia set fractal

Recursion: Patterns inside patterns

Mandelbrot-Julia set fractal

ADEM: General Process Layout

ADEM: Feedback Mechanism

• Classical negative feedback mechanisms makes the system resistant to change

• ADEM has a positive feedback mechanism or the learning loop approach– Such loops make a system open to change – The growth of knowledge during the overall

process makes learning loops or positive feedback possible

Task-1

Task-2

Task-3

Task-4

Task-n

Time

The Process of the ADEM

• Through increments of several iterations

• Increments occur concurrently

• Change is seen to be normal

• Time or schedule-based phases:

time-boxing Adapted form Highsmith (2000)

Time-Boxes• ADEM approaches the D/D process with a

timeline lifecycle policy (time-boxing).

• Major time-boxes are:– Pre-project time-box: Feasibility, speculation – Take-off time-box: Planning & Design– Rising time-box: Development– Destination time-box: Implementation &

maintenance

Pre-project time-box• Feasibility – Mindset study – Speculation

– Methodological decisions– Is EPSS the right solution? Go, no-go.

• Major Outcomes– Vision statement– Risk management plan

• Major issues– Organization type, support, motivation

mechanisms, communication, vision, risks, technology, information resources

Take-off time-box• Analysis & Design

– Functional & structural requirements of the EPSS are determined

– Analysis framework uses Checkland’s SSM CATWOE analysis

– Based on the analysis, activities and time-boxes are determined. Time mgmt. policy!

• Major issues– Organizational environment, iterative process,

participatory design, technical decisions, multiple perspectives, positive feedback

Rising time-box• Development

– Based on the blueprint of the previous stage– “Concurrent component engineering”– Incorporates multiple perspectives (social,

technical, organizational)– Results driving approach (goal: quality products)– Development is not finished - it evolves

• Major issues– Multidisciplinary collaboration, short cycles,

learning from mistakes, active participation

Destination time-box• Implementation – Moving to real-life environment

– Input from more and more users– Analysis and design still continue– “As soon as the design is implemented, its

consequences indicate the need for redesign” Cherns (1976) – Weakest part of the ADEM – No cases reached it

• Major issues– Political and cultural issues, communication, reward

mechanisms

Model SummaryMethodology Adaptive & Dynamic, Socio-Technical

People All of the actively participating stakeholders

Team Multidisciplinary

Environment Complex

Process Product-oriented, concurrent, iterations with increments

Change Continuous, learning-based

Evaluation Parallel to activities, continuous testing, positive feedback

Management Timeboxing, Risk management, Vision-based

Technology Standard, compatible, suitable

Discussion• Is it safe to use the ADEM?

– Answer is Yes: Needs good planning and careful work

• Does it need more work?– Answer is Yes: Needs to be critically evaluated and

improved

• Hard to use it?– Yes, but simple solutions work for simple problems

• The model itself needs to be improved in dynamic and adaptive way

Limitations• Three cases were investigated

• Identifying the underlying detailed steps of the ADEM was beyond the scope of this study. They can be addressed in future studies

• The principles are tentative, guidelines are not widely tested, and the model has not been applied in any real case

Future Research

• Further research on Chaos Theory is needed for EPSS D/D.

• A strong bridge definitely should be established between IS and EPSS research.

• Detailed guidelines need to be developed for the phases and the implementation phase needs to be made clearer

Questions?