eit 2016-17 continuous improvement reportualr.edu/eit/files/2010/08/eit_annual_report_-bs... · eit...

EIT

2016-17

Continuous Improvement Report

for the

BS Geology [program name]

Program

Table of Contents

PROGRAM EDUCATIONAL OBJECTIVES .......................................... 25

STUDENT OUTCOMES.......................................................................... 31

CONTINUOUS IMPROVEMENT ........................................................... 34

25

CRITERION 2. PROGRAM EDUCATIONAL OBJECTIVES

A. Mission Statement

Provide the institutional mission statement.

University of Arkansas System and UALR institutional mission, objectives, and role and scope statements are published in the 2015-16 Undergraduate Catalog (p. 322-323). The institutional mission statement is:

UALR Mission The mission of the University of Arkansas at Little Rock is to develop the intellect of students; to discover and disseminate knowledge; to serve and strengthen society by enhancing awareness in scientific, technical, and cultural arenas; and to promote humane sensitivities and understanding of interdependence. Within this broad mission are the responsibilities to use quality instruction to instill in students a lifelong desire to learn; to use knowledge in ways that will contribute to society; and to apply the resources and research skills of the University community to the service of the city, the state, the nation, and the world in ways that will benefit humanity. (Adopted by the UALR Faculty Senate, 1988)

B. Program Educational Objectives List the program educational objectives and state where these can be found by the general public.

Program educational objectives are broad statements that describe what graduates are expected to attain within a few years after graduation. Program educational objectives are based on the needs of the program’s constituencies. The educational objectives of the Geology Program are to produce graduates who:

• [PEO 1] Are pursuing productive careers as geologists engaged in continuous

professional growth along their chosen career path, or are pursuing graduate or professional education in geology or related fields.

• [PEO 2] Are able to become Geologist in Training (GIT) and are able to become licensed

Professional Geologists (PG) after gaining the required professional experience and the requisite knowledge to pass the licensing exams.

• [PEO 3] Engage in lifelong learning, through on-the-job training, participation in

professional societies, additional formal education, continuing education and professional development, research, and self-study, in order to use state-of-the-art knowledge to solve geologic problems and/or provide high quality service to the general public, employers, clients, and other professionals.

26

The program educational objectives are published in the 2015-16 Undergraduate Catalog (p. 243-244). The program educational objectives are also available via a link on the Department of Earth Sciences website:

http://ualr.edu/earthsciences/

C. Consistency of the Program Educational Objectives with the Mission of the Institution

Describe how the program educational objectives are consistent with the mission of the institution.

The mission of the University of Arkansas at Little Rock is to develop the intellect of students; to discover and disseminate knowledge; to serve and strengthen society by enhancing awareness in scientific, technical, and cultural arenas; and to promote humane sensitivities and understanding of interdependence. Within this broad mission are the responsibilities to use quality instruction to instill in students a lifelong desire to learn; to use knowledge in ways that will contribute to society; and to apply the resources and research skills of the University community to the service of the city, the state, the nation, and the world in ways that will benefit humanity. The geology program develops the intellect of students by exposing them to a broad set of academic disciplines, including not only science, technology, engineering, and mathematics (STEM) but also fine arts, humanities, and social sciences. The program curriculum provides graduates the geologic knowledge to pass licensing exams (PEO2). The geology program discovers and disseminates knowledge and enhances awareness in scientific and technical arenas by teaching courses in all of the major areas of geology and by conducting geological research. For example, courses are offered in mineralogy, petrology, sedimentary geology, structural geology, field geology, hydrogeology, environmental geology and geospatial technology. Several research projects are ongoing, involving undergraduate students, graduate students, and geology faculty members. Students and faculty involved in these projects discover new knowledge and communicate that knowledge to others in professional and academic forums (PEO 1 and PEO 3). The general education component of the geology program enhances awareness in cultural arenas and promotes humane sensitivities and understanding of interdependence. For example, geology students complete courses from fine arts, humanities, social science, and speech communication as a part of their undergraduate curriculum (PEO 3). The geology program also instills a lifelong desire to learn. Pursuing productive careers engaged in continuous professional growth (PEO 1), developing the skills to pass licensing examinations (PEO 2), or pursuing additional education in graduate or

27

professional school (PEO 1, PEO 3) all instill a desire for continual growth of professional skills. Using knowledge that will contribute to society and to the service of the city, the state, the nation, and the world in ways that will benefit humanity is explicitly consistent with PEO 3 which includes “to use state-of-the-art knowledge to solve geologic problems and/or provide high quality service to the general public, employers, clients, and other professionals.”

D. Program Constituencies List the program constituencies. Describe how the program educational objectives meet the needs of these constituencies.

The constituencies of the geology program include:

• Potential employers o Geological Consulting firms o Environmental Consulting firms o Engineering firms o Energy companies o Natural Resource companies o Non-profit organizations o Government agencies o K-12 Education

• Faculty members o UALR Geology Program o Graduate School o Professional School

• Professional societies • Alumni

The program educational objectives encourage the creation of an educated workforce with the knowledge, skills, and abilities to contribute to the economic growth and quality of life in Arkansas and nationally. The program educational objectives encourage students and graduates to pursue licensure, certification, lifelong learning, and continued involvement with professional societies. Representatives of these constituencies formally approved the program education objectives for the geology program on September 23, 2015, at the fall meeting of the Advisory Council.

28

E. Process for Review of the Program Educational Objectives Describe the process that periodically reviews the program educational objectives including how the program’s various constituencies are involved in this process. Describe how this process is systematically utilized to ensure that the program’s educational objectives remain consistent with the institutional mission, the program constituents’ needs and these criteria.

The Advisory Council comprises representatives from the various constituencies of the geology program. The Advisory Council meets at least twice a year to keep up with issues impacting the geology program, to share their needs and concerns, and to provide advice to program faculty.

Proposed program goals, educational objectives and student outcomes were shared with the Advisory Council at the May 15, 2015 meeting. Revised goals, objectives and outcomes, as well as progress toward accreditation, were shared and discussed with the Advisory Council at the September 23, 2015 Advisory Council meeting. At this meeting, he Advisory Board voted unanimously in affirmation that these objectives are consistent with our mission as the program’s constituencies. Please refer to the letter below dated October 29, 2015, from Tim Kresse, Chair of the Advisory Council.

Program assessment progress and results will be presented to the Advisory Council on an annual basis. Members of the Advisory Council will be invited to share any changes in their needs and to propose changes to the program educational objectives. The program educational objectives will be reviewed formally at least every three years at a meeting of the Advisory Council.

31

CRITERION 3. STUDENT OUTCOMES

A. Process for the Establishment and Revision of the Student Outcomes Describe the process used for establishing and revising student outcomes.

In preparing to seek initial ABET accreditation, the geology program faculty adopted ABET student outcomes b, d, f, g, i, j, and k verbatim and a, c, e, and h with slight modifications (see Table 3-1 for relationship between program student outcomes and ABET student outcomes). Review of student outcomes will be conducted every year by the geology program faculty in consultation with its Advisory Council.

B. Student Outcomes List the student outcomes for the program and describe their relationship to those in Criterion 3 of the general criteria and any applicable program criteria. Display this information in Table 3-1. Indicate where the student outcomes are documented.

The geology program will produce graduates who have:

a) an ability to apply knowledge of mathematics and applied and natural sciences to areas relevant to geology.

b) an ability to design and conduct experiments, or test hypotheses, as well as to analyze and interpret data.

c) an ability to formulate or design a system, process, or program to meet desired needs. d) an ability to function on multidisciplinary teams. e) an ability to identify and solve geological problems. f) an understanding of professional and ethical responsibility. g) an ability to communicate effectively. h) the broad education necessary to understand the impact of geological solutions in a

global economic, environmental and societal context. i) a recognition of the need for, and an ability to engage in life-long learning. j) a knowledge of contemporary issues. k) an ability to use the techniques, skills, and modern scientific and technical tools

necessary for professional practice. The student outcomes are published in the 2015-16 Undergraduate Catalog (p. 243-244). The student outcomes are also available via a link on the geology program website: http://ualr.edu/earthsciences/

32

Table 3-1. Relationship Between Program Student Outcomes and Criterion 3

Student Outcomes/Program Criteria

Geology

Criterion 3 Student Outcomes Program Student Outcomes a b c d e f g h i j k

a) an ability to apply knowledge of mathematics and applied and natural sciences to areas relevant to geology.

x

b) an ability to design and conduct experiments, or test hypotheses, as well as to analyze and interpret data. x

c) an ability to formulate or design a system, process, or program to meet desired needs x

d) an ability to function on multidisciplinary teams x e) an ability to identify and solve geological problems x f) an understanding of professional and ethical responsibility x

g) an ability to communicate effectively x h) the broad education necessary to understand the impact of geological solutions in a global economic, environmental and societal context

x

i) a recognition of the need for, and an ability to engage in life-long learning x

j) a knowledge of contemporary issues x k) an ability to use the techniques, skills, and modern scientific and technical tools necessary for professional practice

x

33

C. Relationship of Student Outcomes to Program Educational Objectives Describe how the student outcomes prepare graduates to attain the program educational objectives.

The matrix below shows how the attainment of the program student outcomes contributes to the attainment of program educational objectives. All of the student outcomes support PEO 1. In order to be pursuing productive careers as professional geologists or attending graduate or professional school, students must have the content knowledge, skills and abilities, and professional habits described by each of the program student outcomes. All of the student outcomes support PEO 2. Student outcomes a, b, c, e and k prepare students for taking the ASBOG FG exam, which is the first step in becoming a Geologist in Training (GIT). Student outcomes d, f, g, h, i, and j prepare students to become licensed Professional Geologists (PG) in that they are necessary for helping students to gain the required professional experience and knowledge to pass the licensing exam. All of the student outcomes also support PEO 3. Engaging in life-long learning is a cumulative process that depends on content knowledge, fundamental skills, professional experience and responsibilities, and societal context.

Program Educational Objectives

Program Student Outcomes PEO 1 PEO 2 PEO 3 a) an ability to apply knowledge of mathematics and applied and natural sciences to areas relevant to geology.

x x x

b) an ability to design and conduct experiments, or test hypotheses, as well as to analyze and interpret data.

x x x

c) an ability to formulate or design a system, process, or program to meet desired needs.

x x x

d) an ability to function on multidisciplinary teams x x x e) an ability to identify and solve geological problems x x x f) an understanding of professional and ethical responsibility x x x g) an ability to communicate effectively x x x h) the broad education necessary to understand the impact of geological solutions in a global economic, environmental and societal context

x x x

i) a recognition of the need for, and an ability to engage in life-long learning

x x x

j) a knowledge of contemporary issues x x x k) an ability to use the techniques, skills, and modern scientific and technical tools necessary for professional practice

x x x

34

CRITERION 4. CONTINUOUS IMPROVEMENT This section of your Self-Study Report should document your processes for regularly assessing and evaluating the extent to which the student outcomes are being attained. This section should also document the extent to which the student outcomes are being attained. It should also describe how the results of these processes are utilized to affect continuous improvement of the program.

Assessment is defined as one or more processes that identify, collect, and prepare the data necessary for evaluation. Evaluation is defined as one or more processes for interpreting the data acquired though the assessment processes in order to determine how well the student outcomes are being attained. Although the program can report its processes as it chooses, the following is presented as a guide to help you organize your Self-Study Report.

A. Student Outcomes It is recommended that this section include (a table may be used to present this information):

1. A listing and description of the assessment processes used to gather the data upon which the evaluation of each student outcome is based. Examples of data collection processes may include, but are not limited to, specific exam questions, student portfolios, internally developed assessment exams, senior project presentations, nationally-normed exams, oral exams, focus groups, industrial advisory committee meetings, or other processes that are relevant and appropriate to the program.

2. The frequency with which these assessment processes are carried out 3. The expected level of attainment for each of the student outcomes 4. Summaries of the results of the evaluation process and an analysis illustrating the extent

to which each of the student outcomes is being attained 5. How the results are documented and maintained

A.1 Assessment Process

Assessment Methodology 1. The eleven program student outcomes map directly to the ABET (a) – (k) student

outcomes.

2. The attainment of each of the student outcomes is to be demonstrated by at least one direct measure and at least one indirect measure.

3. All components of each student outcome are assessed.

4. Assessment measurements may be based on student work such as assignments, exams, projects, presentations, laboratory experiments, etc.

35

5. Samples of student work supporting assessment of student outcomes will be retained electronically and placed in assessment binders maintained in the department office.

Student outcomes to be assessed:

The geology program will produce graduates who have: a) an ability to apply knowledge of mathematics and applied and natural sciences to

areas relevant to geology. b) an ability to design and conduct experiments, or test hypotheses, as well as to analyze and

interpret data. c) an ability to formulate or design a system, process, or program to meet desired needs. d) an ability to function on multidisciplinary teams. e) an ability to identify and solve geological problems. f) an understanding of professional and ethical responsibility. g) an ability to communicate effectively. h) the broad education necessary to understand the impact of geological solutions in a

global, economic, environmental and societal context. i) a recognition of the need for, and an ability to engage in life-long learning j) a knowledge of contemporary issues. k) an ability to use the techniques, skills, and modern scientific and technical tools

necessary for professional practice.

A.2 Listing of Assessment Instruments and Frequency of Use The table below contains a list of the assessment instruments used in the geology program assessment process. Collection frequency and specific student outcome assessed by each instrument are also indicated.

Assessment Instrument

Collection Frequency

Student Outcomes Assessed (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k)

ACAT Exam Annual x ASBOG FG Exam Annual x x PSA Annual x x x x x Senior Seminar Pres. Annual x x Field II Project Annual x x x x x x Outcomes Survey Annual x x x x x x x x x x x Exit Survey Annual x

A.3 Descriptions of Assessment Instruments and Mapping of Performance Indicators to Student Outcomes

ACAT Exam

The Area Concentration Achievement Test (ACAT) exam is a learning outcomes assessment instrument developed by PACAT, Inc. for specific disciplines. It is used primarily by public and private baccalaureate degree granting institutions in the United States. The ACAT provides

36

nationally referenced tests from items contributed by the faculty using them. It can be delivered online or as paper booklets and answer sheets. Beginning Spring 2016, taking the ACAT exam became a graduation requirement of the geology program. Students will be required to take the ACAT during the final semester before their projected graduation date. The ACAT does not have assigned administration dates or windows and can be administered by the UALR testing services center or proctored by faculty in the geology computer classroom. Initial ACAT data come from students graduating in spring 2016. ACAT provides individual and average results, by subject, for those taking the exam. The following table displays how ACAT exam scores in various subjects are mapped to the student outcomes.

ACAT Exam Subjects Student Outcomes Assessed (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k)

1. Geomorphology x 2. Historical Geology x 3. Paleontology x 4. Petrology x 5. Physical Geology x 6. Stratigraphy x 7. Structural Geology x 8. Oceanography x

Because the ACAT is the only nationally-normed assessment exam in the field of geology, we chose to use this exam provide data to assess Student Outcome (a) despite some potential flaws in the exam. One potential issue is content validity. Of the eight content areas tested, four areas (Geomorphology, Paleontology, Stratigraphy, and Oceanography) are not required courses for students graduating from the geology program. Also, an important required course, Sedimentology, is not tested. Since administering the exam, we have learned that the exam questions have not been updated since at least 1996 indicating additional issues with content validity. Because of the lack of useful assessment data provided, we will phase out the use of this assessment instrument in favor of an internally developed assessment exam beginning with spring 2017 graduating seniors.

ASBOG Fundamentals of Geology (FG) Examination

The National Association of State Boards of Geology (ASBOG) Fundamentals of Geology (FG) examination is a licensing exam administered by the ASBOG State Member Boards simultaneously typically in early October and the middle of March each year. The exams are administered via paper-based testing at designated sites. Passing the FG exam is one of the requirements for registration as a Geologist-in-Training (GIT), and eventual licensure as a Professional Geologist (P.G.) in the state of Arkansas. Currently, thirty states and the territory

37

of Puerto Rico register/license Professional Geologists. In Arkansas, all government entities that require geological services must contract with a registered professional geologist or an Arkansas firm with a registered professional geologist. To date, not all UALR geology majors/graduates have taken the ASBOG. Information regarding professional registration and ASBOG exams is incorporated into ERSC 4190 Senior Seminar. The department strongly encourages all geology program students to take the exam and pays for them to do so. Taking the exam, however, is currently not a requirement for graduation. We have not made it a requirement for two reasons: 1) the PG is not universally required in order to practice geology and some students will pursue employment that does not require them to be a PG; and 2) the exam is designed to be taken after graduating from college or during the final semester in college. Many of our students have not yet completed the required geology courses that will be tested on the exam by the early to mid-semester examination date. Thus, some students prefer to take the exam after they graduate. ASBOG provides results, by subject, for geology majors at UALR who take the FG exam. ASBOG redacts names from the score reports so we cannot tie results directly to individual students. We have chosen five of the exam categories to use for assessment of geology content and three of the exam categories to use for assessment of professional practice. The following tables display how FG exam scores in geology sub-categories are mapped to the student outcomes.

FG ASBOG Exam Subjects Student Outcomes Assessed (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k)

General, Field x Mineralogy, Petrology, Geochemistry x Sedimentology, Stratigraphy,

Paleontology x

Geomorphology, Surficial Processes, Quaternary Geology x

Structural Geology, Tectonics, Seismology x

Hydrogeology x Engineering Geology x Economic Geology, Energy Resources x

Professional Skills Assessment (PSA) We have modified the Engineering Professional Skills Assessment (EPSA) method and rubric to assess professional skills in geology. The EPSA was developed at Washington State University’s Center for Teaching, Learning, and Technology in 2006. It has been the subject of a four year study (2012-2015) by the National Science Foundation to determine its validity as a direct method of assessing the professional skills represented by ABET student outcomes f, g, h, i, and j.

38

Beginning with the Fall 2015 semester, the PSA became a required exercise in the required capstone course ERSC 4190 Senior Seminar. Prior to a specified class meeting, students are provided with a case study scenario and a set of guidelines for discussing this scenario. The discussion takes place in class and is recorded. Multiple faculty members will assess the discussion using the PSA scoring rubric, some in real time and some after viewing to the recorded event. The following table displays how the PSA scores are mapped to the student outcomes.

PSA Performance Indicators Student Outcomes Assessed (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k)

Stakeholder Perspective x Problem Identification x Ethical Considerations x Meeting Professional Responsibilities x Group Interaction x Group Self-Regulation x Impact/Context x Sources/References x Discern Fact/Opinion x Knowledge Status x Presumptions x Technical Issues x Non-Technical Issues x

Senior Seminar Oral Presentation Assessment Beginning with the Fall 2015 semester, the Senior Seminar Oral Presentation is used for assessing student outcomes. The Senior Seminar Oral Presentation has been a required exercise in the ERSC 4190 Senior Seminar for over ten years. The Senior Seminar Oral Presentation project provides an opportunity for summative assessment of communication skills and an understanding of professional and ethical responsibilities. Oral presentation skills will be evaluated in six areas, using an Oral Presentation Evaluation Form. An understanding of professional and ethical responsibilities will be assessed in two areas. The following table displays how the various oral presentation scores are mapped to the student outcomes.

Senior Seminar Oral Presentation

Performance Indicators Student Outcomes Assessed

(a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) Oral delivery x Presentation organization x Visual aids x Presentation content x Image of presenter x Appropriate citation of sources x

39

Field Geology II Project Assessments The Field Geology II project has been a required project in the required capstone course ERSC 4320 Field Geology II since the courses inception (Summer 2009). Initial assessment data was collected from 5 students from the Summer 2015 Field Geology II course. The Field Geology II provides an opportunity for summative assessment of several student outcomes including communication, teamwork, and geological problem-solving skills. These skills are evaluated using a Field Geology II Project Evaluation Form. Teamwork is evaluated using a Team Evaluation Form (the Team Evaluation form will be introduced initially in Summer 2016). The following table displays how the various project scores are mapped to the student outcomes.

Field Geology II Project Performance Indicators


Map: Title, Explanation, Drafting x Map: Geologic Data x Map: Contacts x Map: Structure x Cross-section: Topography, Scales x Cross-section: Title, Drafting x Cross-section: Pattern matches map x Cross-section: Structure x Cross-section: Projections x Stereonet x Report: Format x Report: Clarity and Style of writing x Report: Geologic history x Team members contributed equally x Team member collegiality x

40

Student Outcomes Surveys The Student Outcomes Survey Form is used to conduct indirect assessment of student outcomes, broken down into specific performance indicators. After completion of all course requirements, graduating seniors assess the level of attainment of the performance indicators for the geology student body as a whole. Initial assessment comes from Fall 2015 and Spring 2016 graduates. The following table displays how the various student outcome survey scores are mapped to the student outcomes.

Student Outcomes Survey Performance Indicators


1. Apply knowledge of mathematics x 2. Apply knowledge of basic sciences x 3. Apply knowledge of applied science x 4. Ability to design an experiment x 5. Ability to conduct an experiment x 6. Ability to analyze data x 7. Ability to interpret data x 8. Ability to design a system, component, or process to meet a desired need x 9. Ability to function on teams x 10. Ability to identify geological problem x 11. Ability to formulate geological

problem x 12. Ability to solve geological problem x 13. Understands professional

responsibility x 14. Understands ethical responsibility x 15. Ability to communicate orally x 16. Ability to communicate in writing x 17. Understand global context x 18. Understand economic context x 19. Understand environmental context x 20. Understand societal context x 21. Recognition of need for lifelong

learning x 22. Ability to engage in lifelong learning x 23. Knowledge of contemporary issues x 24. Ability to use geology techniques x 25. Ability to use geology skills x 26. Ability to use modern geological tools x

41

Graduating Senior Exit Surveys The Graduating Senior Exit Survey Form is used to collect information about the program that could be used to improve the program. Much of this information is unrelated to the ABET student outcomes. However, the form does include several questions about professional plans, such as pursuing additional formal education, professional development, licensure, and certifications. The answers to these questions indirectly indicate whether the students recognize the need for lifelong learning. Initial assessment comes from Fall 2015 and Spring 2016 graduates. The following table displays how the various questions are mapped to the student outcomes.

Exit Survey Performance Indicators Student Outcomes Assessed (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k)

11. Plan to attend graduate school x 13. Plan to pursue the PG registration x 15. Plan become active in professional

society x

A.4 Performance Indicators and Targets The expected level of attainment on student outcome assessment measures varies depending on the assessment tool. The target level of attainment was selected such that primary areas of weakness for each assessment instrument were highlighted. Most results for performance indicators are reported on a uniform 5.0 scale, where 0.0 is the lowest level of attainment and 5.0 represents the highest level of attainment. Performance indicators based on results that are reported as percentages are converted to a 5-point scale, e.g., a value of 60% is reported as 3.00 (60% of 5.00). Other performance indicators were evaluated using questionnaires with some form of a 5-point Likert scale. The ACAT scores are reported as stanine scores, a method of scaling test scores on a nine-point standard scale (1-9) with a mean of five and a standard deviation of two. The faculty will reevaluate the targets levels of student attainment as additional data is collected.

Assessment Instrument Target Level of Attainment ACAT Exam 5 ASBOG FG Exam 2.75 Professional Skills Assessment 2.50 Senior Seminar Oral Presentation 3.75 Field Geology II Project 3.00 Student Outcomes Survey 3.50 Student Exit Survey 3.50

42

The results for each performance indicator, organized both by assessment instrument and by student outcome, are summarized in the following tables. Target and actual levels of attainment for each performance indicator are provided. Performance indicators that did not meet target values are highlighted in red. The raw data to support these results will be available upon request during the onsite review. A.5 Summary Results Organized by Assessment Instrument

ACAT Exam

Outcome Performance Indicator Target Level of Attainment

Actual Level of Attainment

(a)

Geomorphology 5 5 Historical Geology 5 5 Paleontology 5 5 Petrology 5 6 Physical Geology 5 5 Stratigraphy 5 3 Structural Geology 5 6 Oceanography 5 5

ASBOG Exam



(b)

A. General, Field 2.75 2.80 B. Mineralogy, Petrology, Geochemistry 2.75 2.80 C. Sedimentology, Stratigraphy, Paleontology 2.75 3.25 D. Geomorphology, Surficial Processes,

Quaternary Geology 2.75 2.70

E. Structural Geology, Tectonics, Seismology 2.75 3.05

(e) F. Hydrogeology 2.75 2.55 G. Engineering Geology 2.75 3.05 H. Economic Geology and Energy Resources 2.75 2.80

43

Professional Skills Assessment (PSA)



(f) Stakeholder Perspective 2.50 2.33 Problem Identification 2.50 3.00 Ethical Considerations 2.50 2.67

(g) Group Interaction 2.50 3.00 Group Self-Regulation 2.50 2.17

(h) Impact/Context 2.50 2.67

(i) Scrutinize Information; Discern Fact/Opinion 2.50 1.50 Knowledge Status 2.50 2.00

(j) Non-Technical Issues 2.50 3.00 Technical Issues 2.50 2.50

Senior Seminar Oral Presentation



(f) Image of presenter 3.75 * Appropriate citation of sources 3.75 *

(g)

Oral delivery 3.75 3.9 Presentation organization 3.75 3.9 Visual aids 3.75 4.1 Presentation content 3.75 3.7

*Initial data to be collected Fall 2016.

Field Geology II Project*



(a) Cross-section: Topography, Scales 3.00 4.4 Cross-section: Projections 3.00 2.4

(b) Map: contacts 3.00 2.7 Cross-section: pattern matches map 3.00 4.3 Plot field and stereonet data 3.00 2.1

(c) Map: structure 3.00 2.3 Cross-section: structure 3.00 2.0 Report: geologic history 3.00 2.8

(d) Team members contributed equally 3.00 5.0 Team member collegiality 3.00 4.8

(g)

Map: Title, explanation, drafting 3.00 4.0 Cross-section: Title, drafting 3.00 4.8 Report: Format 3.00 4.2 Report: Clarity, style of writing 3.00 3.7

(k) Map: Geologic Data 3.00 3.6 *Updated after Summer 2016

44

Student Outcomes Survey



(a) 1. Apply knowledge of mathematics 3.50 3.67 2. Apply knowledge of basic sciences 3.50 4.00 3. Apply knowledge of applied science 3.50 4.67

(b)

4. Ability to design an experiment 3.50 4.33 5. Ability to conduct an experiment 3.50 4.33 6. Ability to analyze data 3.50 4.00 7. Ability to interpret data 3.50 4.00

(c) 8. Ability to design a system, component, or process to meet a desired need 3.50 3.83

(d) 9. Ability to function on teams 3.50 4.00

(e) 10. Ability to identify geological problem 3.50 4.17 11. Ability to formulate geological problem 3.50 3.83 12. Ability to solve geological problem 3.50 3.83

(f) 13. Understands professional responsibility 3.50 3.83 14. Understands ethical responsibility 3.50 4.00

(g) 15. Ability to communicate orally 3.50 4.33 16. Ability to communicate in writing 3.50 4.17

(h)

17. Understand global context 3.50 4.33 18. Understand economic context 3.50 3.00 19. Understand environmental context 3.50 4.50 20. Understand societal context 3.50 4.00

(i) 21. Recognition of need for lifelong learning 3.50 4.33 22. Ability to engage in lifelong learning 3.50 4.50

(j) 23. Knowledge of contemporary issues 3.50 4.33

(k) 24. Ability to use geology techniques 3.50 3.83 25. Ability to use geology skills 3.50 3.83 26. Ability to use modern geological tools 3.50 4.17

Graduating Senior Exit Survey



(i) 11. Plan to attend graduate school 3.50 5.00 13. Plan become active in professional society 3.50 5.00 15. Plan to pursue the PG registration 3.50 5.00

45

A.6 Summary of Results Organized by Student Outcome

Student Outcome (a) An ability to apply knowledge of mathematics, science, and geology

Assessment Instrument Performance Indicator Target Level

of Attainment Actual Level

of Attainment

ACAT Exam

1. Geomorphology 5 5 2. Historical Geology 5 5 3. Paleontology 5 5 4. Mineralogy 5 6 5. Petrology 5 5 6. Physical Geology 5 5 7. Stratigraphy 5 3 8. Structural Geology 5 6 9. Oceanography 5 5

Field Geology II Project

Cross-section: Topography, Scales 3.00 4.00

Cross-section: Projections 3.00 2.30

Outcomes Survey

1. Apply knowledge of mathematics 3.50 3.67 2. Apply knowledge of basic sciences 3.50 4.00 3. Apply knowledge of applied science 3.50 4.67

Student Outcome (b) An ability to design and conduct experiments, as well as to analyze and interpret data



of Attainment

ASBOG FG Exam

A. General & Field Geology 2.75 2.80 B. Mineralogy, Petrology & Geochemistry 2.75 2.80 C. Sedimentology, Stratigraphy, Paleontology 2.75 3.25 D. Geomorphology, Surficial Processes &

Quaternary Geology 2.75 2.70

E. Structural Geology, Tectonics, Seismology 2.75 3.05 Field Geology II Project

Map: contacts 3.00 2.90 Cross-section: pattern matches map 3.00 4.20 Plot field and stereonet data 3.00 2.90

Outcomes Survey

4. Ability to design an experiment 3.50 4.33 5. Ability to conduct an experiment 3.50 4.33 6. Ability to analyze data 3.50 4.00 7. Ability to interpret data 3.50 4.00

46

Student Outcome (c)

An ability to design a system, component, or process to meet desired needs



of Attainment Field Geology II Project

Map: structure 3.00 3.30 Cross-section: structure 3.00 3.10 Report: geologic history 3.00 3.70

Outcomes Survey

8. Ability to design a system, component, or process to meet a desired need 3.50 3.83

Student Outcome (d) An ability to function on multidisciplinary teams




Team members contributed equally 3.00 *

Team member collegiality 3.00 *

Outcomes Survey 9. Ability to function on a team 3.50 4.00

* Initial data to be collected July 2016.

Student Outcome (e) An ability to identify and solve geological problems



of Attainment

ASBOG FG Exam

F. Hydrogeology 2.75 2.55 G. Engineering Geology 2.75 3.05 H. Economic and Resources Geology 2.75 2.80

Outcomes Survey

10. Ability to identify geological problem 3.50 4.17 11. Ability to formulate geological problem 3.50 3.83 12. Ability to solve geological problem 3.50 3.83

47

Student Outcome (f) An understanding of professional and ethical responsibility



of Attainment Senior Seminar Pres.

Image of presenter 3.75 *

Appropriate citation of sources 3.75 *

PSA Stakeholder Perspective 2.50 2.33 Problem Identification 2.50 3.00 Ethical Considerations 2.50 2.67

Outcomes Survey

13. Understands professional responsibility 3.50 3.83 14. Understands ethical responsibility 3.50 4.00

*Initial data to be collected fall 2016.

Student Outcome (g) An ability to communicate effectively



of Attainment

PSA Group Interaction 2.50 3.00 Group Self-Regulation 2.50 2.17

Field Geology II Project

Map: Title, explanation, drafting 3.00 4.30 Cross-section: Title, drafting 3.00 4.60 Report: Format 3.00 4.20 Report: Clarity, style of writing 3.00 3.70

Senior Seminar Present.

Oral delivery 3.75 3.90 Presentation organization 3.75 3.90 Visual aids 3.75 4.10 Presentation content 3.75 3.70

Outcomes Survey

15. Ability to communicate orally 3.50 4.33 16. Ability to communicate in writing 3.50 4.17

Student Outcome (h) The broad education necessary to understand the impact of geological solutions in a global,

economic, environmental, and societal context Assessment Instrument Performance Indicator Target Level


of Attainment PSA Impact/Context 2.50 2.67

Outcomes Survey

17. Understand global context 3.50 4.33 18. Understand economic context 3.50 3.00 19. Understand environmental context 3.50 4.50 20. Understand societal context 3.50 4.00

48

Student Outcome (i) A recognition of the need for, and an ability to engage in life-long learning



of Attainment

PSA Scrutinize Information; Discern Fact/Opinion 2.50 1.50 Knowledge Status 2.50 2.00

Outcomes Survey

21. Recognition of need for lifelong learning 3.50 4.33 22. Ability to engage in lifelong learning 3.50 4.50

Exit Survey 11. Plan to attend graduate school 3.50 5.00 13. Plan become active in professional society 3.50 5.00 15. Plan to pursue the PG registration 3.50 5.00

Student Outcome (j) A knowledge of contemporary issues



of Attainment

PSA Non-Technical Issues 2.50 3.00 Technical Issues 2.50 2.50

Outcomes Survey 23. Knowledge of contemporary issues 3.50 4.33

Student Outcome (k) An ability to use the techniques, skills, and modern scientific and technical tools necessary for

professional practice. Assessment Instrument Performance Indicator Target Level



Map: Geologic Data 3.00 3.8

Outcomes Survey

24. Ability to use geology techniques 3.50 3.83 25. Ability to use geology skills 3.50 3.83 26. Ability to use modern geological tools 3.50 4.17

A.7 Documentation and Maintenance of Results Starting in 2015-16, the results of the assessment process will be documented in an annual assessment report that each program will submit to the EIT college assessment and accreditation coordinator.

49

B. Continuous Improvement Describe how the results of evaluation processes for the student outcomes and any other available information have been systematically used as input in the continuous improvement of the program. Describe the results of any changes (whether or not effective) in those cases where re-assessment of the results has been completed. Indicate any significant future program improvement plans based upon recent evaluations. Provide a brief rationale for each of these planned changes.

The assessment results collected to date reveal that the students did not reach the target level of attainment for 12 of 77 performance indicators. These 12 performance indicators were distributed over 7 of the 11 student outcomes (a, b, e, f, g, h and i) rather than concentrated in just one or two. Thus, for every student outcome, a majority of its performance indicator scores met or exceeded the target level of attainment. The assessment results were evaluated by program faculty and resulted in several changes to the geology program. The intent of these changes is to improve the student level of attainment in all areas, but specifically address the 12 below-target performance indicators identified in the previous section. The following table summarizes changes that have been or are being made.

50

Student Outcome (a) Performance Indicator Change / Status of Change

ACAT Exam: Stratigraphy

The average score of the Spring 2016 graduates (stanine 3) did not reach the target level of attainment (stanine 5) indicating a deficiency in subject matter knowledge in this area.

Change: Stratigraphy is taught primarily in two courses in the program curriculum: Historical Geology / Historical Geology Lab (ERSC 2303 / 2103) and Sedimentology and Stratigraphy (ERSC 3440). A curricular weakness in stratigraphy was recognized two years ago during faculty discussion of departmental curriculum. Based on this discussion, we decided to significantly revise the Historical Geology / Lab course including elevating it from a 1000-level general education core course with no prerequisites (ERSC 1303/1103) to a 2000-level core course with a introductory geology course prerequisite (ERSC 2303/2103). This change allows for more in depth coverage of stratigraphy (and other topics). The new version of Historical Geology first appeared in the 2015-16 Undergraduate Catalog and in was first taught in the fall of 2015. Course content in Sedimentology and Stratigraphy (ERSC 3440) has focused more on sedimentology than stratigraphy likely contributing to this deficiency. The course content will be revised beginning this fall 2016 semester to include more stratigraphy content and exercises.

Status: This change in the Historical Geology course was first implemented in the fall of 2015. The change in the Sedimentology and Stratigraphy course will be implemented in fall of 2016. It will take a few years before the results from these changes can be fully evaluated.

Field Geology II Project: Cross Sections – Projections

The average score of the summer 2015 students (2.3) did not reach the target level of attainment (3.0). Students need to improve their skill in applying mathematical functions to correctly project geologic cross sections.

Change: The cross-section projection techniques used in this field course are introduced in the junior-level course Structural Geology (ERSC 3430). Because of this identified deficiency, additional exercises incorporating cross-section projection techniques will be incorporated into structural geology laboratories.

Status: The change in the Structural Geology course will be implemented in spring 2017 when this course is next taught. We expect improvement as soon as summer 2017 for those Field Geology II students who complete Structural Geology in spring 2017.

51

Student Outcome (b) Performance Indicator Change / Status of Change

ASBOG FG Exam: Geomorphology, Surficial Processes & Quaternary Geology

The average score of fall 2015 and spring 2016 seniors (2.70) did not meet the target (2.75) for this ASBOG content domain.

Change: The field of geomorphology, and the closely allied fields of surficial processes and Quaternary geology, are not part of the required curriculum for the UALR geology program - nor are they required courses at most geology programs. Geomorphology (ERSC 4419) is offered as an elective ay UALR. Aspects of these three fields are taught in the introductory Physical Geology (ERSC 1302/1102) and Earth and the Environment (ERSC 1304/1104) courses but not at the level of understanding required for the FG exam. Because of credit hour restraints and the difficulty of offering this course every year, we do not currently plan to make Geomorphology a required course. To improve student performance in this (and other) ASBOG content domains, we will begin offering a short course each semester to better prepare students for this exam. Part of this course will focus on the aspects of geomorphology, surficial processes, and quaternary geology.

Status: This change will be implemented in the spring of 2017, thus improvement is expected in the results of the spring 2017 offering of the ASBOG exam.

Field Geology II Project: Map Contacts

The average score of the summer 2015 students (2.9) did not reach the target level of attainment (3.0). Students need to improve their skill in correctly locating geologic contacts consistent with topography.

Change: Mapping geologic contacts, including the use of topography to assist with this mapping, is introduced during an introductory mapping project in Field Geology I (ERSC 3320) but also relies on geological reasoning progressively developed in all geology courses. The primary means to become more proficient in this skill is to practice mapping in the field and plotting contacts on maps. To address this deficiency, a specific exercise on mapping geologic contacts will be added to the Field Geology I course. The effect of topography on contacts is covered in Structural Geology (ERSC 3430). An additional lab exercise will be added to this course to provide students additional practice with this skill.

Status: The exercise on mapping geologic contacts will be added to Field Geology I at it next offering in spring 2017. The additional structure and topography lab exercise will be added to Structural Geology when it is next offered in the spring of 2017. Improvement is expected for students in the summer 2017 Field Geology II course.

52

Field Geology II: Plot field and stereonet data

The average score of the summer 2015 students (2.9) did not reach the target level of attainment (3.0). Students need to improve their skill in correctly plotting field and stereonet data that is consistent with map and cross sections.

Change: Initial instruction on collecting, plotting and interpreting stereographic data occurs in Structural Geology (ERSC 3340). To provide students with additional experience plotting and interpreting field and stereographic data, a field project requiring these skills will be added to Structural Geology (ERSC 3430).

Status: The field project will be added to Structural Geology when it is next offered in the spring of 2017. Improvement is expected for students in the summer 2017 Field Geology II course.

Student Outcome (e) Performance Indicator Change / Status of Change

ASBOG FG Exam: Hydrogeology

The average score of fall 2015 and spring 2016 seniors (2.55) did not meet the target (2.75).

Change: Hydrogeology (ERSC 4473) is not a required course in the Geology Program (but is required for the Environmental Geology concentration). This course is not required at most traditional geology programs but is required at most applied geology programs. Discussion between geology faculty led to the conclusion that, in place of one of three electives, Hydrogeology should be added as required course.

Status: Curriculum change forms to add Hydrogeology as a required course in the geology curriculum has been completed and will be submitted at the beginning of the fall 2016 semester.

Student Outcome (f) Performance Indicator Change / Status of Change

Professional Skills Assessment: Stakeholder Perspective

The average score of fall 2015 seniors (2.33) did not meet the target (2.5).

Change: The Professional Skills Assessment (PSA) was included in Senior Seminar (ERSC 4190) for the first time in fall 2015. Lower than anticipated performance on the “Stakeholder Perspective” part of the PSA has led faculty to conclude that relevant material must be covered more directly throughout the curriculum prior to assessment during Senior Seminar. Therefore, required upper-level ERSC courses will be modified to include activities and discussions that relate to application of geology content knowledge to specific issues scenarios. Additionally, the Senior Seminar course must be modified to include more directed information related to student understanding of stakeholders and the stakeholder perspective.

53

Status: Beginning in fall 2016, required upper-level ERSC courses will be modified to include additional course content involving real-world application of content-specific geological concepts. For example, Mineralogy (ERSC 3410) will now include a section of content related to collection, distribution, and use of mineral resources, and a service-learning activity to engage directly with the public (i.e. potential stakeholders). Additional courses will be modified in a similar manner in future semesters. Senior Seminar will also be modified to include a pre-PSA activity to review components of discussion as they relate to professional skills.

Student Outcome (g) Performance Indicator Change / Status of Change

Professional Skills Assessment: Group Self-Regulation

The average score of the 2015 seniors (2.17) did not reach the target level of attainment (2.50).

Change: The Professional Skills Assessment (PSA) was included in Senior Seminar (ERSC 4190) for the first time in fall 2015. Lower than anticipated performance on the “Group Self-Regulation” part of the PSA has led faculty to conclude that relevant material must be covered more directly throughout the curriculum prior to assessment during Senior Seminar. Therefore, several courses will be modified to include additional discussions that relate to application of geology content knowledge to specific issues scenarios. Additionally, the Senior Seminar course will be modified to include more directed information related to student understanding of group self-regulation.

Status: Beginning in fall 2016, group discussions will be expanded in all major courses, including examining and reflection on methods for maintaining constructive discussion. Several different class discussion techniques (e.g. Gallery Walk, Philosophical Chairs, Pinwheel Discussion, etc. – see www.cultofpedagogy.com/speaking-listening-techniques for a longer list of discussion techniques) will be implemented. We also intend to bring in representatives from UALR Communication Center to work with students in Senior Seminar to learn effective group self-regulation techniques.

Senior Seminar Presentation: Presentation Content


Change: Respondents to surveys collected during Senior Seminar (ERSC 4190) student presentations indicated that the presentation content was slightly lower than expected. Therefore, better and more specific faculty directives and feedback will be needed to ensure higher quality content is included in presentations.

54

Status: Beginning in fall 2016, student presenters will be required to include and disclose a faculty-determined number of peer-reviewed journal articles as primary source content for presentation. Practice presentations, which are already a part of the Senior Seminar curriculum, will be more rigorously reviewed for content issues.

Student Outcome (h) Performance Indicator Change / Status of Change

Outcomes Survey: understand economic context

The average score of the 2015 seniors (3) did not reach the target level of attainment (3.5).

Change: The field of economic geology is not part of the required curriculum for the UALR geology program - nor is it a required courses at most geology programs. We plan to incorporate economic context into new exercises, or exercises that already exist in our upper level required geology courses.

Status: Beginning in fall 2016, required upper-level ERSC courses will be modified to include additional course content involving real-world application of the economic content of geological concepts and materials. Mineralogy (ERSC 3410) will now include a section of content related to collection, distribution, and use of mineral resources. Additional courses will be modified in a similar manner in future semesters.

Student Outcome (i) Performance Indicator Change / Status of Change

Professional Skills Assessment: Scrutinize Information; Discern Fact from Opinion


Change: The Professional Skills Assessment (PSA) was included in Senior Seminar (ERSC 4190) for the first time in fall 2015. Lower than anticipated performance on the “Scrutinize Information; Discern Fact from Opinion” part of the PSA has led faculty to conclude that more specific directives must be made to students regarding the importance of scrutinizing various sources of information for legitimacy, and to be more specific in their determination of facts, opinions, and information that may not be able to be separated into fact or opinion.

Status: Beginning in fall 2016, PSA instructions will be modified to direct students to more rigorous and direct scrutiny of source information to determine if the information is opinion, fact, or some combination of both fact and opinion. They will be asked to directly score the source information on a subjective reliability scale from pure opinion (0) to pure fact (10). The students will be directed to share these scores with other group members during the PSA

55

activity.

Professional Skills Assessment: Knowledge Status


Change: The Professional Skills Assessment (PSA) was included in Senior Seminar (ERSC 4190) for the first time in fall 2015. Lower than anticipated performance in the “Knowledge Status” part of the PSA has led faculty to conclude that addition of a pre-PSA preparation activity is required to ensure sufficient accumulation and reflection on collected knowledge content related to the geology scenario presented by the PSA.

Status: Beginning in fall 2016, students will be directed to prepare one page, individual summaries of content knowledge information they have collected and their professional opinion regarding the scenario in question. The students will be directed to read aloud these individual summaries at the beginning of the group PSA activity. These individual summaries will also be collected by faculty and used to grade preparation for the PSA activity.

eit 2016-17 continuous improvement reportualr.edu/eit/files/2010/08/eit_annual_report_-bs... · eit...

Documents