A Comprehensive Assessment Approach for the School of Advanced Studies

Rob Olding, Ph.D. Associate Dean for AssessmentUniversity of Phoenix: School of Advanced Studies

Learning Assessment for Doctoral Level Education is …?• Relatively new in current conceptualization• Relatively abstract (compared to Associate and

Bachelor levels)• Higher thresholds for expectations of existing

skills• Research and writing intensive

A Multiple Method Approach Assessment Strategy­ Somewhat in the form of an “Instrumental

Case Study” approach consisting of “data vectors” from various perspectives:­ Rubric based assessment based on IRMA mapping of

curriculum­ External evaluation sources (e.g., accreditor reports,

Council on Graduate Schools, external consultants).­ Appreciative Inquiry approach to assessment

Triangulation of Assessment Data Sources (basic model)

 

SAS Practitioner Doctorate Learning Assessment Data

Sources

Rubric based measurement: Student Learning Outcomes

External 3rd Party Data (Reports)

Appreciative Inquiry (AI) Based Assessment

Deliberative Analysis

Rubric based measurement: Student Learning Outcomes

­ Critical to the assessment process is the development of analytic rubrics to measure performance on signature assignments as described.

­ As documented in multiple sources, analytic rubrics have been shown as effective feedback even at the advanced education level (Groggins-Selke, 2013)

Analytic Rubrics focused upon:

Signature assignments in content and research courses

Key Progression assignments such as dissertation concept paper and proposal

Doctoral Dissertation assessment via multiple reviewers as “capstone” measure of doctoral success

External 3rd Party Data (Reports)

­ Leveraging the data from these different sources to form a clear analytical picture of how our learning outcomes and programs are perceived from external views.

External Consultant Reports as engaged by the University of Phoenix School of Advanced Studies – (e.g., Council of Graduate Schools)

Survey Data (e.g., SEOCS and Alumni Surveys)

Accreditor Reports (e.g., Higher Learning Commission)

Appreciative Inquiry approach to Learning Assessment ­ This approach is argued to be

atypical in assessment of learning, but very appropriate for the Doctoral level which compared to lower level degrees can be diverse, abstract and dynamic in outcomes.

As a method is known to be “transformational” and “engaging” which is important for both faculty and students

Positive in focus: “What were the most important learning outcomes you found in this course”?

Tied directly to a process of envisioning what we do best, what our dream outcomes would be, and how we might move to obtain that level!

Appreciative Inquiry Steps: 4-D Modeladapted from Watkins and Mohr (2001)

Discovery“Best of what is?”

Appreciating

Dream“What might be?”

Envisioning Results

Design“What should be the

ideal?”

Destiny“How to empower, learn, improve?”

Sustaining Change

­ The three approaches to assessment data provide a diverse set of perspectives

­ These perspectives can be left unreconciled, or…

­ Integrated to provide a new level of perspective that informs far beyond any on its own.

­ TRIANGULATION

Integration of Multiple Data Sources using diverse methods

­ Includes diverse perspectives in the discussion

­ Provides for a basis of meaningful engagement for administration, faculty and students

­ Avoids an over reliance on a single paradigm approach that limits perceptions

Advantages:

­ The Triangulation process requires a means of bringing together diverse data sources

­ An ideal approach would extend the process to multiple “stakeholders” that would include students, faculty and administration

­ Such a method would work to ensure “fair consideration” of each data source in relation to optimum learning outcomes

Means of Integration:

The Deliberation ModelAn inclusive approach that involves all stakeholders and represents each of the data sources in the process

Deliberation: A means of integrating diverse perspectives

­ Deliberation is neither a casual discussion or a formal or informal debate. The discussion should focus first on each source individually and then in combination and from a “best solution” view

Maximum engagement – Can be used in face to face, online synchronous and asynchronous formats

Allows for carefully weighing diverse views and bringing light upon points not previously considered and explored

Allows for exploration of facts in relationship to strategic positions as represented by each area data source

External Reports (3rd party)

Rubric based Assessment

Appreciative Inquiry Results

Interpretation and

Integration

Each perspective is sequentially discussed in terms of meaning and both positive and negative implications. Then they are compared and discussed in relationship to integration with each other. Commonalities are of particular note in TRIANGULATING meaning.

­ The strategy is to:­ Maximize engagement of stakeholders­ Optimize the data sources in ensuring that

diverse methods and perspectives are included in consideration

­ Develop an integrated set of insights that can inform strategy and contribute meaningfully to continuous improvement

­ Move assessment and evaluation to a new level for Doctoral Education

Insights and Outcomes:

Thank You!

Questions????

REFERENCES

Borkowski, Nancy, A. (2006). Changing our thinking about assessment at the doctoral level. In Maki,

Peggy L. & Borkowski, Nancy A. (Eds.), The assessment of doctoral education:

Emerging criteria and new models for improving outcomes (pp. 11-51). Stylus Publishing LLC,

Sterling, Virginia.

Bushe, Gervase R. & Kassam, Aniq F. (2005). When is appreciative inquiry transformative?: A

meta- case analysis. The Journal of Applied Behavioral Science, Vol. 41, No. 2, pp. 161-181.

DOI: 10.1177/0021886304270337.

Groggins Selke, Mary J. (2013). Rubric assessment goes to college: Objective, comprehensive

evaluation of student work. Rowman & Littlefield Education, Lanham, Maryland.