1 conclusion. 2 metrics revisited inputs outputs quality and satisfaction interrelated. rely...
Post on 19-Dec-2015
234 views
TRANSCRIPT
![Page 1: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/1.jpg)
1
Conclusion
![Page 2: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/2.jpg)
2
Metrics Revisited
Inputs Outputs Quality and Satisfaction
Interrelated. Rely heavily on subjective responses and self-reporting
OutcomesImpacts on community- Our effects
![Page 3: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/3.jpg)
3
Outcomes Assessment
Measuring change as a direct result of interaction with the library
Most often addressed in academic and school libraries Types of change:
Knowledge Behavior Awareness/ Percpetions Attitudes/ Opinions
![Page 4: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/4.jpg)
4
Assessment Defined
The ongoing process of:
Establishing clear, measurable expected outcomes of student learning
Ensuring that students have sufficient opportunities to achieve those outcomes
Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations
Using the resulting information to understand and improve student learning [at both the course and program] level Suskie (2004, p. 3)
![Page 5: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/5.jpg)
5
NEASC STANDARD SEVEN: Library and Information Resources
1: The institution makes available the library and information resources necessary for the fulfillment of its mission and purposes. These resources support the academic and research program and the intellectual and cultural development of students, faculty, and staff. Library and information resources may include the holdings and necessary services and equipment of libraries, media centers, computer centers, language laboratories, museums, and any other repositories of information or technological systems required for the support of institutional offerings. Clear and disseminated policies govern access, usage, and maintenance of the library, information resources, and services. The institution ensures that students use these resources as an integral part of their education.
2: Through the institution's ownership or guaranteed access, sufficient collections, information technology systems, and services are readily accessible to students wherever programs are located or however they are delivered. These collections, systems, and services are sufficient in quality, level, diversity, quantity, and currency to support and enrich the institution's academic offerings. The institution provides facilities adequate to house the collections and equipment so as to foster an atmosphere conducive to inquiry, study, and learning among students, faculty, and staff. --- Satisfaction?
![Page 6: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/6.jpg)
6
NEASC STANDARD SEVEN (continued) 3: The institution provides sufficient and consistent financial support for the effective
maintenance and improvement of the institution's library, information resources, and services. It makes provision for their proper maintenance, preservation, currency, and security. It allocates resources for scholarly support services compatible with its instructional and research programs and the needs of faculty and students.
4: Professionally qualified and numerically adequate staff administer the institution's library, information resources, and services. The institution provides appropriate orientation and training for use of these resources, as well as instruction in basic information literacy.
5: The institution participates in the exchange of resources and services with other institutions and within networks as necessary to support and supplement its educational programs. It provides appropriate support for distance learning students and faculty, such as on-line reference service and contractual access to relevant off-campus library resources.
6: The institution regularly and systematically evaluates the adequacy and utilization of its library, information resources, and services and uses the results of the data to improve and increase the effectiveness of these services (satisfaction, service quality?)
![Page 7: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/7.jpg)
7
NEASC STANDARD Four:Programs and Instruction
Compliance = outcomes student learning outcomes student outcomes
Undergraduate Degree Programs Graduate Degree Programs Scholarship and Research Instruction Admissions and Retention
http://community.pmc.edu/neasc/standards/Default.htm
![Page 8: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/8.jpg)
8
More on Accreditation
Regional Accreditation- 3 use “information literacy” 3 use “equivalent language” NCC makes the point that today’s students must be prepared to be
“knowledge workers” in the sense that they must not just master certain information, but must be able to comprehend, synthesize and apply that information. More than just technologically literate, these workers will be valued for their “capacity to sift and winnow massive amounts of information in order to discover or create new or better understandings”
Support for collaboration Professional and Discipline-Specific Accreditors also paying
attention to information literacy Important to fields focused on Evidenced-Based Practice
![Page 9: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/9.jpg)
9
Student Outcomes versus Student Learning Outcomes
Student outcomes are aggregate statistics on groups of students (e.g., graduation rates, retention rates, transfer rates, course and program completion rates, and job placement: employment rates for a graduating class)
Such outcomes are institutional outcomes and are used to compare institutional performance
They do not measure changes in students themselves due to their college experience
these outcomes are outputs and reflect what the institution has accomplished; they do not reflect what (or how much) students learned
Student learning outcomes —development of students: demonstrable acquisition of specific knowledge and skills
How well do students Transfer and apply concepts,
principles, ways of knowing, and problem solving across their major program of study?
Integrate their core curriculum, general studies, or liberal studies into their major program or field of study?
Develop understanding, behaviors, attitudes, values, and dispositions that the institution asserts it develops?*
* Maki (2004)
![Page 10: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/10.jpg)
10
Student Learning Outcomes What should students learn?
How does the content of one course relate to another? How well are they learning it?
Across courses What evidence do we gather to answer these
questions? Beyond graded assignments, course evaluations
How do we use that evidence to improve learning Looking a courses as comprising a program
![Page 11: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/11.jpg)
11
Purposes of Assessment
Impacting student learning
Working collaboratively: thinking beyond the set of courses “I” teach
Demonstrating accountability and accreditation—
Meeting institutional mission
Demonstrating institutional effectiveness
![Page 12: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/12.jpg)
12
Institutional Effectiveness Defined as how well an institution achieves its
mission and major institutional goals “Since student learning is the heart of most
institutional missions, the assessment of student learning is a major component of the assessment of institutional effectiveness.”
However, institutional effectiveness does examine other aspects such as scholarship and research, community service, etc.
Suskie, 9-10
![Page 13: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/13.jpg)
13
Student Learning Outcomes
ConceptualLeadership
Critical ThinkingCreative ThinkingProblem Solving
Information LiteracyGlobal Citizen
Values (moral, etc.)Foundations and Skillsfor Lifelong Learning
SkillsOral/written communication
Foreign language communicationTechnological sophisticationQuantitative reasoning ability
Other
Do “satisfaction” and “service quality” impact performance?
![Page 14: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/14.jpg)
14
Direct Methods
Embedded course assessment (performance on assignments, etc.; minute paper)
Portfolio assessment Performance (internships,
practicum, student teaching) Professional jurors or
evaluators Capstone course/experience
Experimental research designs), with pre- and post-testing
Use of standardized tests Think-aloud protocol Directed conversation Videotape/audiotape
evaluation Analysis of
theses/dissertations/ senior papers (content analysis,
interviews, or oral defense)
![Page 15: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/15.jpg)
15
Indirect Methods
Surveys (self-reporting) and self-assessments
Curriculum and syllabus evaluation
Exit interviews Observation
Other
![Page 16: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/16.jpg)
16
A culture of
Assessment
Evidence
Evaluation
![Page 17: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/17.jpg)
17
Have They Learned Anything?
![Page 18: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/18.jpg)
18
So What?
Or Why Evaluation Research?
![Page 19: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/19.jpg)
19
Evaluation Research Addresses Questions Such As What improvements in a program, service, or policy
might result in continuous quality improvement and better accountability?
How well does a program, service, or policy reach its target population and meet the group’s information needs and expectations?
Is the program, service, or policy being implemented in the ways envisioned?
Is it effective?
![Page 20: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/20.jpg)
20
Utility
What are the findings good for? What decision in the real world do the findings
impact? What insights will the study provide? Can the study findings be dismissed with “so
what?”
![Page 21: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/21.jpg)
21
Evaluation (for a Purpose)
Feedback loop (basic systems model)
Decision making
Accountability Summary judgment
(summative evaluation) Continuous [quality]
improvement Benchmarking and Best
Practices
![Page 22: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/22.jpg)
22
Evaluation Research
Is also a political and managerial activity, an input into the complex mosaic from which we can make policy decisions and allocations for the planning, design, implementation, and continuance of programs to better the human
In this sense evaluation research is part of social policy and public administration
![Page 23: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/23.jpg)
23
Evaluation Research--Steps
Reflective inquiry
Procedures
Conduct studyAnalyze data
Prepare/deliver report
Reliability
Validity
Communication skills
Setting up action plan
![Page 24: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/24.jpg)
24
What are important areas for “evaluation research”?
______________ ______________ ______________
etc.
![Page 25: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/25.jpg)
25
Evaluation of Information Services
Systems Analysis Other Research Courses*
*Strongly recommended for anyone planning a career in academic libraries, especially university libraries
Courses in Individual Concentrations
![Page 26: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/26.jpg)
26
Scholarly CommunicationKey Issues Time lag between authorship, peer review,
publication, and dissemination Information overload
![Page 27: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/27.jpg)
27
Future of PublishingOur Core Interests Competitive market Easy distribution and reuse Innovative applications of technology Quality assurance Permanent preservation (“archiving”)
![Page 28: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/28.jpg)
28
Electronic Scholarly Publishing Some Models Digital library model Electronic book model Electronic collection model Retrospective model Preprint model Peer review lite model Commercial publishing model Academic server model
![Page 29: 1 Conclusion. 2 Metrics Revisited Inputs Outputs Quality and Satisfaction Interrelated. Rely heavily on subjective responses and self-reporting Outcomes](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d385503460f94a110ed/html5/thumbnails/29.jpg)
29
Electronic Scholarly Publishing Some Models (continued) Prestigious publishing model University publishing cooperative Public domain model Government server model