Academic Profile Examination Results

Fall 1992 through Spring 2006

Revised byLaura B. Taylor

Previous Reports Prepared byShauna L. StewartAnn M. Nakamura

Tony Y. LamDavid W. Murphy

Center for Assessment and Instructional SupportMissouri State University

Martha Kirker, Ph. D.Director

Report Number: LOCSUR07-01

TABLE OF CONTENTS

Abstract...............................................................................................................................1Highlights for Fiscal Year 2006...............................................................................2

Highlights for Fall 1992 Through Spring 2006........................................................6

Introduction........................................................................................................................8

Data and Methods..............................................................................................................9Instrument................................................................................................................9Participants...............................................................................................................9Data........................................................................................................................11

Results...............................................................................................................................13Norm-Referenced Data..........................................................................................13 Mean Total Scores for Baccalaureate Students..............................................13 College-Level Reading Scores........................................................................17 College-Level Writing Scores.........................................................................17 Critical Thinking Scores.................................................................................18 College-Level Mathematics Scores................................................................18 College-Level Humanities Scores...................................................................19 College-Level Social Science Scores..............................................................19 College-Level Natural Science Scores........................................................... 20Student-Reported Data……………………………………………………….…..21 Mean Total Score by Student-Reported Class Level.....................................21 Mean Total Score by Student-Reported GPA.................................................21 Mean Total Score by Student-Reported Portion of General Education/Core Curriculum Requirements Successfully Completed........22 Mean Total Score by Student-Reported Hours Worked.................................23 Mean Total Score by Student-Reported Number of Credits Transferred.................................................................................................24Criterion-Referenced Data.....................................................................................25 Writing Scores................................................................................................26

Fall 2001 – Spring 2006 Writing Summary...............................................27................................................................................................................................................

Mathematics Scores........................................................................................29Fall 2001 – Spring 2006 Mathematics Summary......................................30

Reading/Critical Thinking Scores...................................................................32Fall 2001 - Spring 2006 Reading Summary..............................................33

Concluding Discussion.....................................................................................................35Issues that Pertain to Norm-Referenced Data........................................................35Issues that Pertain to Total Scores for Subgroups of Student-Reported Data....................................................................................36Issues that Pertain to Criterion-Referenced Data...................................................36Conclusion.............................................................................................................38

Appendix A: Specific Materials Covered in the Humanities,Social Sciences, and Natural Sciences Areas........................................................40

Appendix B.1: Proficiency Level Definitions for Writing................................................41Appendix B.2: Proficiency Level Definitions for Mathematics........................................42Appendix B.3: Proficiency Level Definitions for Reading/Critical Thinking...................43Appendix B.4: Proficiency Level Classifications for Reading/Critical Thinking,

Writing, and Mathematics......................................................................................44Appendix C.1: Colleges and Universities Included in Comparison Group for

Summer 2003 through Spring 2006.......................................................................45

This report summarizes the results of the Academic Profile examination administered to seniors and juniors at Southwest Missouri State University (now Missouri State University) between fall 1992 and spring 2006. The purpose of this examination process was to evaluate the general academic knowledge and skills of the students receiving a baccalaureate degree, to assess the outcomes of the general education program, and to improve the quality of instruction and learning. Future testing will apply the Measure of Academic Proficiency and Progress (MAPP).

For the past fourteen years this campus has consistently assessed baccalaureate graduates. Over 98% of baccalaureate graduates participated with the exception of the initial two years when fiscal year 1995 included 95% and fiscal year 1994 included 90% participation. Seniors represented 88% of the 32,856 students taking the exit exam between fall 1992 and spring 2006.

This report provides an overview of the data collected over the past 14 years. The most recent data from the 2006 fiscal year is highlighted briefly and included in the aggregate summary. Normative data is illustrated by semester ranging from fall 1992 through spring 2006. Student-reported data is based on questions and response options at the time of examination. Due to revisions of the test by Educational Testing Service (ETS), some response options were adjusted. Criterion-referenced data is illustrated to include data from the past 14 years and national comparison scores. In 2001, ETS recalculated criterion for proficiency definitions and comparisons. Years were categorized based on ETS criterion for proficiency definitions and include: fall 1992-summer 1997; fall 1997-summer 2001; fall 2001-spring 2006; and the Educational Testing Service national comparison scores established in the 2001-2002 academic year.

Since fall 1992, this University annual mean total score has been at or above the national comparison for every fiscal year.

The report concludes the administration of the Academic Profile edition of the general education examination produced by the Educational Testing Service. This edition is being discontinued and replaced by the Measure of Academic Proficiency and Progress (MAPP). According to the Educational Testing Service, this new general education examination will produce scores that are comparable to the scores produced by the Academic Profile

1

Abstract

Norm-Referenced Data

Missouri State University mean scores for the 2006 fiscal year were above the Master’s (Comprehensive) Colleges and Universities I and II-Senior comparative means for all seven of the norm-referenced test dimensions: Reading, Writing, Critical Thinking, Math, Humanities, Social Science, and Natural Science. The subscores are reported on a scale that extends from 100 to 130. Although the subscores are all reported on the same scale, they are not comparable between dimensions.

Norm-Referenced TestMissouri State

University FY06 Senior Mean

National Comparative Mean

Reading 120.79 119.88Writing 116.06 115.42

Critical Thinking 112.81 112.16Mathematics 115.66 114.60Humanities 116.42 115.78

Social Science 115.12 114.60Natural Science 116.69 116.04

Criterion-Referenced Data

For the summer 2005 through spring 2006 semesters, 78% of our seniors were proficient at Level 1-writing, the lowest level.

Writing Average Percentage of Seniors (Fiscal Year 2006)

Level Not Proficient Marginal ProficientLevel 1- lowest 4.7% 17.0% 78.3%Level 2 29.0% 46.0% 24.3%Level 3- highest 50.7% 38.7% 10.7%

For the summer 2005 through spring 2006 semesters, 67% of the seniors were proficient at Level 1-mathematics, the lowest level.

Mathematics Average Percentage of Seniors (Fiscal Year 2006)Level Not Proficient Marginal ProficientLevel 1- lowest 7.3% 25.7% 67.0%Level 2 30.0% 30.3% 39.7%Level 3- highest 67.0% 24.3% 8.7%

2

Highlights for Fiscal Year 2006

For the summer 2005 through spring 2006 semesters, 77% of the seniors were proficient at Level 1-reading, the lowest level.

Reading Average Percentage of Seniors (Fiscal Year 2006)Level Not Proficient Marginal ProficientLevel 1- lowest 7.3% 16.0% 77.0%Level 2 31.0% 20.3% 48.3%Level 3- Critical Thinking 73.7% 19.3% 6.7%

Student-Reported Data The total scores are reported on a scale that extends from 400 to 500.

Mean total score for Missouri State University baccalaureate graduates (451.00)

was above the national comparative mean total score (448.0).

Average total score was higher for seniors (452.40) than for juniors (446.37).

3

Mean total scores increased relative to increases in self-reported grade point averages.

Students who took the examination after successfully completing 100% of the general education/core curriculum distribution requirements had a higher mean total score (452.31) than those students who had completed about 75% of the requirements (445.22) before taking the examination.

4

Those students who reported working 16 to 30 hours per week had the highest mean total score (452.94).

Students who reported that they had transferred one to 15 credits to Missouri State University received a higher mean total score (453.73) than students in the other three categories.

5

During the 14 years, the University virtually met the goal set for baccalaureate students: One hundred percent of baccalaureate graduates will be assessed by a standardized test of general education and the resulting institutional mean will meet or exceed the national comparative mean.

For each fiscal year, the University average score for graduating seniors consistently met or exceeded the national comparative mean.

General education scores for students near graduation followed the national trend of declining several points over the fourteen year time period.

Although students generally met or exceeded the comparative mean for reading, writing, mathematics, and critical thinking, the number of students rising above the minimal proficiency level relative to specified criteria have room for considerable improvement when compared to specific competencies.

Scores in the humanities, social sciences, and natural sciences generally exceeded slightly or met the national mean, and followed the direction of the national trend.

Seniors consistently obtained higher scores than juniors during the fourteen year time period. The average senior scores ranged from 449 to 456 and the average junior scores ranged from 437 to 456.

General education scores increased relative to increases in student reported grade point averages.

Students who took the examination after completing their required general education courses had higher total scores than those students who reported they had completed less than 75% of the general education requirements.

Writing: Recent Senior Proficiency

In the most recent five years, 78% of seniors indicated proficiency at the basic level of writing; they could recognize agreement among basic grammatical elements and incorrect word choice, ordered sentences in a paragraph and elements in an outline.

Twenty-five percent of seniors were able to incorporate new material in a passage, recognize agreement among basic grammatical elements when complicated by intervening words, and combine simple clauses into single, more complex combinations.

Twelve percent of seniors met the highest level of writing and in addition to basic competencies, discriminated between appropriate and inappropriate uses of parallelism, and between appropriate and inappropriate uses of idiomatic language.

6

Highlights for Fall 1992 Through Spring 2006

Conversely, five percent of the seniors did not meet the most basic level of writing on the examination.

Mathematics: Recent Senior Proficiency

In the most recent five years, 68% of seniors indicated proficiency at the basic level. They could work problems that would most likely be solved by arithmetic and did not involve conversion of units or proportionality, but did involve the informal properties of numbers and operations, often involving the number line, including positive and negative numbers, whole numbers and fractions and square roots and squares of numbers.

Thirty-nine percent of seniors could also solve problems with some complications, such as complex wording, maximizing or minimizing, and embedded ratios. Those included algebra problems that can be solved by arithmetic. Seniors were able to simplify algebraic expressions, perform basic translation, and draw conclusions from algebraic equations and inequalities. They could interpret a trend represented in a graph or choose a graph that reflects a trend. These seniors also solved problems involving sets with numeric answer choices.

Nine percent of seniors met the highest level of mathematics in addition to basic competencies. They could solve word problems unlikely to be solved by arithmetic. These problems involved difficult arithmetic concepts such as exponents and roots other than squares and percent of increase or decrease. Seniors were able to generalize about numbers, solve problems requiring an understanding of the properties of integers, rational numbers, and interpret a graph in which trends are to be expressed algebraically or require insight or logical reasoning.

Eight percent of seniors did not meet the most basic level of mathematics.

Reading: Recent Senior Proficiency

In the most recent five years, 77% of seniors indicated proficiency at the basic level; recognized factual material explicitly presented in a reading passage and understood the meaning of words or phrases in the context of a reading passage.

Forty-seven percent of seniors could also synthesize material from different sections of a passage. They could recognize valid inferences derived from material in the passage, identify accurate summaries of a passage, understand and interpret figurative language and discern the main idea, purpose, or focus of a passage or a significant portion of a passage.

Critical Thinking: Recent Senior Proficiency

Almost seven percent of seniors were proficient at the highest level of reading which was critical thinking. They could evaluate competing casual explanations, and hypotheses for

7

consistency with known facts, as well as determine the relevance of information for evaluating an argument or conclusion. Seniors could determine whether an artistic interpretation is supported by evidence, and recognize the salient features or themes in a work of art. This level of proficiency also required seniors to evaluate the appropriateness of procedures for investigating a question of causation, evaluate data for inconsistency with known facts, hypotheses, or methods, and recognize flaws and inconsistencies in an argument.

The primary purpose of this report is to provide an aggregate overview of the data collected over the past 14 years. This report provides the results of the Academic Profile examination, which was given to undergraduate students before graduation. The report provides information collected by the Center for Assessment and Instructional Support from the fall semester of 1992 through the spring semester of 2006.

The University philosophy of developing educated persons places the student at the center of its efforts. The general education program is the academic foundation that provides the shared knowledge and intellectual tools with which students explore their disciplines. Administering a standardized test of general education provides a means of assessing the general academic knowledge and skills of the students in order to evaluate the outcomes of the general education near graduation. By assessing student learning in this way, steps can be taken to improve the quality of instruction and learning.

The College Board and Educational Testing Service developed the Academic Profile. These organizations define the examination as “…a test of general academic knowledge and skills. It is intended for use by colleges and universities in assessing the outcomes of their general education programs to improve the quality of instruction and learning. The test focuses on the academic skills developed through general education courses rather than on knowledge acquired about the subjects taught in these courses. It does this by testing college-level reading, college-level writing, critical thinking, and mathematics in the context of humanities, social sciences, and natural sciences.”

During a pilot assessment project in the fall semester of 1992, the Academic Profile was an elective general education exit examination given to baccalaureate degree seeking students prior to graduation. The examination continued to be elective until the fall semester of 1993. At that time, a standardized test of general education became a requirement to be fulfilled prior to graduation, according to the undergraduate catalog. For those students who entered a degree program prior to fall 1993, the examination remained elective. However, students who have entered a degree program since the

8

Introduction

initiation of the requirement are required to take the examination to receive a baccalaureate degree.

Students are given the option of taking the examination during their senior year or junior year, if leaving campus, prior to graduation. Students should complete their general education requirements before taking the examination. There are two reasons for this. First, this examination is administered to assess the general education program. To obtain an accurate reflection of the general education program, students need to have completed the general education requirements. Second, as stated by The College Board and Educational Testing Service, “a student who has taken general education courses in humanities, social sciences, and natural sciences will be better able to read the materials in the test and answer the questions than a student who has not taken such courses.” Missouri State University considers students’ experiences during the entire educational career while enrolled as contributing to the general education of graduates.

The examination results are used for a number of purposes. The combined results from the examination aid in the evaluation of the effectiveness of the general education curriculum at the University. The information gained provides a basis for improving the quality of instruction and student learning. The results also represent the performance of students to the public and are incorporated in legislative funding decisions, which influence the general budget of the University. Finally, students can use the examination results as an assessment of their individual abilities relative to the graduating class for that semester and include the information in resumes or portfolios.

Instru ment

This University adopted the short form of the Academic Profile examination for assessment of general education resulting from a four-year baccalaureate degree program. The Academic Profile consists of 36 multiple-choice questions to be answered in 40 minutes. Students are allowed to use standard, non-graphing calculators to complete the examination. The materials included in the Academic Profile address issues, themes, and ideas from the humanities, social sciences, and natural sciences. Detailed descriptions of these three academic areas are in Appendix A. The questions on the Academic Profile examination assess college-level reading, college-level writing, critical thinking and mathematics in the context of humanities, social sciences, and natural sciences. Mathematical data are presented not only within the context of the three academic areas, but in a numerical form as well.

9

Data and Methods

Par ticipants

A total of 32,856 undergraduates have taken the Academic Profile from fall 1992 through spring 2006. Seniors represented 88% of the total. The following table shows the number of students who took the Academic Profile during each of the 41 semesters as well as their self-reported class standing.

10

Number of Missouri State Students Who Took the General Education Exam During Each Semester

Semester Juniors Seniors Total*Fall 1992 8 278 288Spring 1993 96 1003 1119Summer 1993 25 151 179Fall 1993 183 1054 1249Spring 1994 173 1219 1401Summer 1994 24 160 190Fall 1994 114 824 948Spring 1995 169 1124 1300Summer 1995 18 138 156Fall 1995 111 709 823Spring 1996 143 1087 1238Summer 1996 31 141 174Fall 1996 125 752 883Spring 1997 141 988 1142Summer 1997 23 140 164Fall 1997 107 772 882Spring 1998 135 1000 1145Summer 1998 21 125 149Fall 1998 115 675 798Spring 1999 133 990 1138Summer 1999 21 156 176Fall 1999 91 749 844Spring 2000 143 990 1141Summer 2000 29 110 141Fall 2000 118 696 821Spring 2001 140 1108 1261Summer 2001 26 167 195Fall 2001 72 842 922Spring 2002 102 1351 1469Summer 2002 7 166 177Fall 2002 60 860 922Spring 2003 105 1408 1520Summer 2003 16 154 173Fall 2003 81 811 897Spring 2004 100 1380 1491Summer 2004 9 154 165Fall 2004 86 840 928Spring 2005 117 1420 1544Summer 2005 18 139 160Fall 2005 87 865 959Spring 2006 138 1430 1584TOTAL 3,461 29,126 32,856

*The data presented in this table shows that the majority of the students who took the examination reported being of junior or senior class standing. However, a total of 130 students reported being of sophomore standing and 38 reported being of freshmen standing during the 14 years. The remaining 101 students did not choose to report class standing.

Da ta

Educational Testing Service (ETS) reports the scores for the Academic Profile in two ways. Scores are criterion-referenced and norm-referenced. Norm-referenced scores have meaning only in comparison to other scores on the same test; for example, scores of other students, scores at different time periods, or scores from other colleges and universities. The SAT and ACT are examples of other norm-referenced tests. The eight norm-referenced scores represent humanities, social sciences, natural sciences, college-level reading, college-level writing, critical thinking, mathematics, and a total score.

In the fourteen years of using The Academic Profile, Missouri State University transitioned through two national comparative groupings established by Educational Testing Service. ETS provided comparative means for groups of colleges and universities at a national level based on Carnegie Classifications. The transition occurred when Carnegie Classifications were redefined. The national comparative mean for Comprehensive Colleges and Universities-Upperclassmen was used through spring 2003. In summer 2003, ETS and Missouri State University began using the national comparative mean for Masters (Comprehensive) Colleges and Universities I and II-Senior.

According to The Academic Profile User’s Guide published in 1990, the comparative mean for Comprehensive Colleges and Universities-Upperclassmen was 451.0. In 1994, the comparative mean for Comprehensive Colleges and Universities-Upperclassmen was reduced to 450.0 using data from 1991 through 1993 administrations. In 1998, ETS used student data from 1994 through 1997 administrations of the Academic Profile to determine the national comparative mean of 448.6. In 2000, that national comparative mean was maintained at 448.6 when Educational Testing Service and The College Board used 1995 through 1999 administrations of the examination to recalculate the mean. In 2003, ETS and The College Board calculated a new mean using 2001 through 2002 administrations of the exam. The new mean for Master’s (Comprehensive) Colleges and Universities I and II- Senior was 448.0, and was used as the national comparison mean through spring 2006. Figure 1 provides a summary of the ETS publications and the national comparative means obtained.

Figure 1: National Comparative Mean TimelineETS Publication

YearNational Data

Obtained Comparison Group Classification National Comparative Mean

1990 Comprehensive Colleges and Universities-Upperclassmen 451.0

11

1994 1991-1993 Comprehensive Colleges and Universities-Upperclassmen 450.0

1998 1994-1997 Comprehensive Colleges and Universities-Upperclassmen 448.6

2000 1995-1999 Comprehensive Colleges and Universities-Upperclassmen 448.6

2003 2001-2002 Master’s (Comprehensive) Colleges and Universities I and II- Senior 448.0

Criterion-referenced scores depend on specific competencies expected of students. The Educational Testing Service adopted criterion-referenced scores for the Academic Profile that determine three levels of competency in writing, in mathematics, and in reading/critical thinking: not proficient, marginal, and proficient. Students who do not meet basic competencies in the dimensions are classified as “not proficient”. Students whose test results do not provide enough evidence to categorize them as not proficient or proficient are classified as “marginal”. Students who meet the competency requirements within a specific skill dimension are classified as “proficient. The three hierarchical levels within each skill dimension indicate that students who are proficient at the higher levels are also proficient at the lower levels.

The competencies necessary for students to perform at each level for the criterion-referenced writing, mathematics, and reading or critical thinking dimensions can be found in Appendices B.1, B.2, and B.3 respectively. Educational Testing Service and The College Board developed the specific competencies of each level. The results were summarized as the percent of students meeting the criteria for each of the levels.

12

This section will focus on each of the previously mentioned highlights in considerable detail. It also includes other notable findings based on the results of the examination from the fall semester of 1992 through the spring semester of 2006. The Educational Testing Service revised the Academic Profile and made the new version available for use in the fall semester of 2001. The new Academic Profile provides scores that are designed to be comparable to the scores on the previous form of the examination. Therefore, scores from both versions of the examination can be used as an indicator of trends.

The results section will be divided into three subsections. The first subsection presents the results for the norm-referenced data. The second subsection examines the average scores of students in subgroups according to student-reported information. The third subsection presents the criterion-referenced scores.

Norm-Referenced Data

Norm-referenced data have meaning only in comparison to other scores on the same test; for example, scores of other students, scores at different time periods, or scores from other colleges and universities. Educational Testing Service provided comparative means for groups of colleges at a national level based on Carnegie Classifications. From 1993 through 2002, the most appropriate comparison for Missouri State University norm-referenced scores was the Comprehensive Colleges and Universities- Upperclassmen mean scores. For the semesters from summer 2003 to spring 2006, Master’s (Comprehensive) Colleges and Universities I and II- Senior mean scores were used as comparison data. The more recent comparison with senior data holds students to a higher standard than the previous comparison with upperclassman.

Between the fall 1992 and summer 2001 semesters, ETS did not distinguish seniors from other students when reporting norm-referenced data. In fall 2001 ETS began providing summary data for seniors. These distinctions are reflected in this report with data between fall 1992 and summer 2001 reported for “students” and data between fall 2001 and spring 2006 reported for “seniors.”

Mean Total Scores for Baccalaureate Stud ents

13

Results

The Academic Profile mean consistently met or exceeded the national comparative mean for each of the fiscal years beginning with FY 1994. The percentage of baccalaureate graduates assessed by the standardized general education examination has been above 99 percent since FY 1998. National comparative means were recalculated in 1998 and 2003.

Year Missouri State Mean Comparative Mean Percentage Assessed

FY06 451 448 99.52FY05 451 448 99.80FY04 452 448 99.29FY03 452 448 99.26FY02 453 449 99.48FY01 452 449 99.51FY00 452 449 99.71FY99 451 449 99.81FY98 451 450 99.17FY97 452 450 98.61FY96 452 450 98.63FY95 453 450 94.62FY94 454 450 89.65

The mean total scores for baccalaureate graduates from the fall semester of 1997 through the spring semester of 2006 were above the national comparative mean for Master’s (Comprehensive) Colleges and Universities I and II- Senior during every fiscal year and all semesters, but one. The following table displays the means and standard deviations of baccalaureate graduate scores as well as the comparative means and standard deviations for all semesters.

14

15

Missouri State University Baccalaureate Graduate Mean Scores and Standard Deviations and Comparative Means and Standard Deviations for Fall 1997 through Spring 2006

Semester Missouri State Mean

Missouri State Standard Deviation

Comparative Mean

Comparative Standard Deviation

Fall 1992 454 16.2 451 10.0

Spring 1993 455 17.3 451 10.0

Summer 1993 455 16.5 451 10.0

Fall 1993 453 16.3 450 6.8

Spring 1994 453 17.3 450 6.8

Summer 1994 452 15.7 450 6.8

Fall 1994 453 17.0 450 6.8

Spring 1995 453 17.6 450 6.8

Summer 1995 451 17.5 450 6.8

Fall 1995 453 16.9 450 6.8

Spring 1996 451 17.5 450 6.8

Summer 1996 452 17.3 450 6.8

Fall 1996 452 16.5 450 6.8

Spring 1997 453 17.2 450 6.8

Summer 1997 450 16.3 450 6.8

Fall 1997 452 16.3 450 6.8

Spring 1998 452 17.3 450 6.8

Summer 1998** 451 17.6 448.6 18.2

Fall 1998 451 17.3 448.6 18.2

Spring 1999 452 17.2 448.6 18.2

Summer 1999 451 17.0 448.6 18.2

Fall 1999 452 17.5 448.6 18.2

Spring 2000 453 17.1 448.6 18.3

Summer 2000 450 16.9 448.6 18.3

Fall 2000 451 17.4 448.6 18.3

Spring 2001 453 17.5 448.6 18.3

Summer 2001 450 17.2 448.6 18.3

Fall 2001 453 18.0 448.6 18.3

Spring 2002 454 18.3 448.6 18.3

Summer 2002 448 18.4 448.6 18.3

Fall 2002 451 18.3 448.6 18.3

Spring 2003 453 18.0 448.6 18.3

Summer 2003* 449 18.2 448.0 19.0

Fall 2003 451 18.2 448.0 19.0

Spring 2004 453 18.4 448.0 19.0

Summer 2004 448 17.7 448.0 19.0

Fall 2004 451 17.8 448.0 19.0

Spring 2005 453 18.0 448.0 19.0

Summer 2005 449 18.6 448.0 19.0

Fall 2005 451 17.2 448.0 19.0

Spring 2006 453 17.9 448.0 19.0

*Beginning in Summer 2003, the Educational Testing Service recalculated the comparison mean for Senior- Master’s (Comprehensive) Colleges and Universities I and II to 448.0. This value has been corrected from previous reports that used the comparison mean for All Institution Types- Master’s (Comprehensive) Colleges and Universities I and II of 448.7.

**Prior to Summer 1998, the Educational Testing Service provided institutional comparative means and standard deviations. Individual comparative means and standard deviations have been provided and reported since Summer 1998.

The following graph shows the mean scores for baccalaureate graduates for semester of graduation and the national comparative mean for each semester. Prior to summer semester 1998, national institutional comparative means were used. Since summer 1998, national individual means have been provided by ETS and used as comparisons. For the past 41 semesters, the Missouri State University mean has been at or above the comparative mean in all but one semester. The summer semester of 2002 fell below the national comparative mean by six-tenths of a point.

16

College-Level Reading Scores

Mean scores of students for the college-level reading scale were above the national comparative mean for Comprehensive Colleges and Universities- Upperclassmen in all but two of the 41 semesters. During the summer 2000 and 2002 semesters, the means dropped to 119.60 and 119.70 respectively. The mean scores for seniors from the summer semester of 2003 to the spring semester of 2006 were at or above the Master’s Comprehensive) Colleges and Universities I and II- Senior national comparative mean of 119.88.

College-Level Writing Scores

Mean scores for students on the college-level writing scale were at or above the national comparative mean for Comprehensive Colleges and Universities-Upperclassmen between fall 1997 and summer 2001. Prior to that, scores fell below the comparison mean only twice in summer 1994 and summer 1997. Fall 2001 through spring 2003 mean scores were lower than the national comparative mean of 116.60. The summer 2003 through spring 2006 scores for seniors were all above the national comparative mean of 115.42 for Master’s (Comprehensive) Colleges and Universities I and II- Senior.

17

College-Level Critical Thinking Scores

Mean scores for students on the norm-referenced critical thinking scale were above the comparative mean for Comprehensive Colleges and Universities-Upperclassmen for all of the 32 semesters from fall 1992 through spring 2003, except summer 2002. Scores from summer 2003 through spring 2006 were all above the comparative mean for Master’s (Comprehensive) Colleges and Universities I and II-Senior of 112.16.

College-Level Mathematics Scores

Mean scores of students for the norm-referenced college mathematics scale were consistently at or above the national comparative mean for Comprehensive Colleges and Universities- Upperclassmen from fall 1992 to spring 2003. The following semesters through spring 2006 were all above the comparative mean of 114.60 for Master’s (Comprehensive) Colleges and Universities I and II- Senior.

College-Level Humanities Scores

18

Mean scores for students on the norm-referenced humanities scale were above the national comparative mean for Comprehensive Colleges and Universities-Upperclassmen for most of the 41 semesters of fall 1992 to spring 2006. In summer 1997 scores barely fell below the mean of 116.10. In summer 2000, summer 2002, and in fall 2002, the mean scores for students were below the comparative mean. Both of the following summer semesters of 2003 and 2004 were below the comparative mean of 115.78 for Master’s (Comprehensive) Colleges and Universities I and II- Senior.

College-Level Social Science Scores

Mean scores for students on the norm-referenced social science scale met or exceeded the national comparative mean for Comprehensive Colleges and Universities- Upperclassmen for all 15 semesters from fall 1992 through summer 1997. The mean scores for the semesters of summer 1998, summer 2002 and fall 2002 all fell below this comparative mean. The mean scores for seniors exceeded the comparative mean for Master’s (Comprehensive) Colleges and Universities I and II- Senior in all semesters from the summer of 2003 through the spring of 2006 except summer and fall of 2004. The comparative mean score was 114.60.

College-Level Natural Science Scores

19

Norm-referenced natural science mean scores for students were above the comparative mean for Comprehensive Colleges and Universities- Upperclassmen for 27 consecutive semesters from fall of 1992 through summer 2001. All semesters from fall 2001 through spring 2003 were below the comparative mean of 117.50. Senior scores from summer 2003 to spring 2006 consistently exceeded the mean for Master’s (Comprehensive) Colleges and Universities I and II- Senior of 116.04.

Student-Reported Data

20

Student-reported data allows trend comparisons in reference to particular subgroups of students. Student-reported data is based on questions and response options at the time of examination. Due to revisions of the test by Educational Testing Service (ETS), some response options were adjusted. This section examines the average scores of students in subgroups according to student-reported information.

Mean Total Score by Student-Repor ted Class Level

Students had the opportunity to report their class level. The majority of students indicated that they were either of junior or senior class standing. In general, seniors had a higher total score than juniors on the examination. The graph presents the scores obtained by juniors and seniors for each semester beginning with spring 1993. The mean total score of seniors in the 2006 fiscal year was 452.40 and 446.37 for juniors. It is interesting to note that the total score for seniors, from semester to semester, tended to be more consistent than the total score for juniors, from semester to semester. Total mean scores for seniors ranged from 449 to 456, while total scores for juniors ranged from 437 to 456.

Mean Total Score by Student- Reported GPA

Students reported their grade point average based on one of five choices ranging from 1.0 to 4.0. Mean total scores increased relative to increases in student-reported grade point average for all forty-one semesters from fall 1992 to spring 2006. Semesters were separated to allow comparisons between the first five years of the Academic Profile to the following nine years. A single total mean score was calculated for each of the four GPA groups and the results are presented on the graph.

For those students with a self-reported GPA of 3.5 to 4.0 the mean total score was 463.13 for semesters from fall 1992 through spring 1997 and 460.83 for semesters from fall 1997 through spring 2006. The mean total score for students with a self-reported GPA of 3.0 to 3.49 was 453.47 for fall 1992 through spring 1997 and 451.60 for semesters falling between fall 1997 and spring 2006. Students with a self-reported GPA of 2.5 to 2.99 had

21

a mean total score of 447.87 between the semesters of fall 1992 through spring 1997, and 445.30 between the semesters of fall 1997 through spring 2006. Students with a self-reported GPA of 2.0 to 2.49 had a mean total score of 444.40 for the semesters between fall 1992 and spring 1997, and a mean total score of 443.36 for the semesters following through spring 2006.

Mean Total Score by Student-Reported Portion of General Education/Core Curriculum Distribution Requirements Successfully Completed

Students who took the examination after completing the general education requirements tended to have higher total scores than those students who completed only approximately 75% of the general education requirements. For the 2006 fiscal year, the mean total score for those students reporting having completed the general education requirements was 452.31, while the mean of those who had completed 75% was 445.22.

The graph illustrates mean total scores for each of the 41 semesters for those students who reported completing the general education requirements prior to taking the examination. It also shows those students who reported having completed about 75% of the requirements.

22

Mean Total Score b y Student-Reported Hours Worked

Many students worked in jobs for pay in addition to attending classes. It is interesting to compare responses from the first five years of this test to responses from the last three years. From 2003-2006 the highest mean total score was recorded by students reporting working one to 15 hours a week, whereas from 1992-1997 the highest mean total score came from students reporting working one to five hours a week. This could be due to the regrouping; however, fiscal year 2006 data suggest movement to higher number of hours worked in recent years.

In the earliest form of the exit exam students were asked to report the number of hours worked based on a scale indicating none, one to five hours a week, six to 10 hours a week, 11 to 20 hours a week, 21 to 30 hours a week, 31 to 40 hours a week, and more than 40 hours a week. For the semesters falling between fall 1992 and spring 1997, students who reported working one to five hours per week had the highest mean score of 454.89 and students who reported working 31 to 40 hours had the lowest mean score of 451.80. Students who reported working six to 10 hours per week had the second highest mean score of 454.09.

Beginning in the summer semester of 2003, this scale was changed to none, one to 15 hours a week, 16 to 30 hours a week, and more than 30 hours a week. Students who reported working 30 hours or more a week had the lowest mean total score of 449.98. Those students who worked one to 15 hours a week had the highest mean total score of 452.17. The mean total score of those who reported working 16 to 30 hours was 451.55, and that of those reporting not working had a mean total score of 451.23.

23

Mean T otal Score by Student-Reported Number of Credits Transferred

Students were asked to report whether or not they transferred credits to Missouri State University from another college or university. If credits had been transferred, then students could report the number of credits successfully transferred based on one of three choices.

Between 1992 and 2006, students who reported not transferring to Missouri State University received a mean total score of 451.88. Students who chose the category for transferring one to 15 credits had the highest mean total score of 452.88. Students who chose the category for transferring 16 to 30 credits had a mean total score of 451.29. Students who chose the category for transferring more than 30 credits had a mean total score of 450.19.

24

Criterion-Referenced Data

Criterion-referenced scores depend on specific competencies expected of students. The Educational Testing Service adopted criterion-referenced scores for the Academic Profile that determine three levels of competency in writing, in mathematics, and in reading/critical thinking: not proficient, marginal, and proficient. Students who do not meet basic competencies in the dimensions are classified as “not proficient”. Students whose test results do not provide enough evidence to categorize them as not proficient or proficient are classified as “marginal”. Students who meet the competency requirements within a specific skill dimension are classified as “proficient. The three hierarchical levels within each skill dimension indicate that students who are proficient at the higher levels are also proficient at the lower levels.

Criterion-referenced data is illustrated to include data from the past 14 years and national comparison scores. In 2001, ETS recalculated criterion for proficiency definitions and comparisons. Between the fall 1992 and summer 2001 semesters, ETS did not distinguish seniors from other students when reporting criterion-referenced data. In fall 2001 ETS began providing summary data for seniors. These distinctions are reflected in this report with data between fall 1992 and summer 2001 indicated for “students” and data between fall 2001 and spring 2006 indicated for “seniors.”

Graphs are provided to illustrate proficiencies in four groups: fall 1992-summer 1997; fall 1997-summer 2001; fall 2001-spring 2006; and national comparison scores. National comparison data was obtained through the Educational Testing Service website. The data was gathered during the academic year 2001-2002, and represents seniors under the Master’s (Comprehensive) Colleges and Universities I and II classification. Additional graphs show the results of the criterion-referenced scores from fall 2001 through spring 2006 to illustrate trends by semester for the past five years. A detailed description of each level of the criterion-referenced writing dimension is in Appendix B.1. The results were summarized as the percent of Missouri State University students meeting the criteria for each level.

25

Writing Scores

Between the fall 1992 and summer 1997 semesters the highest percentage of students (42%) performed proficiently at Level 1, the lowest level of writing. When the percentages for each year are averaged, 32% of the students performed at Level 2, and 16% of the students performed proficiently at Level 3, the highest level. Approximately 11% of the students did not meet the requirements of the most basic level.

From fall 1997 through summer 2001, similar trends were noticed. Proficiency at Level 1 showed the highest percentage of students (42%). Approximately 30% of students performed at Level 2, and 16% at Level 3. Around 13% of the students did not reach proficiency at Level 1.

The most recent five years, between fall 2001 and spring 2006 semesters, 78% of the seniors tested indicated proficiency at Level 1. For these fourteen semesters, 25% of the seniors performed proficiently at Level 2. Approximately 12% of the seniors were proficient at Level 3, and five percent of seniors who took the examination did not meet the specific components for the most basic level in writing.

The national comparison data provided a summary of proficiency classifications for writing. The percentage of seniors showing proficiency at Level 1 was 73%, for Level 2, 22%, and Level 3, 10%. Seven percent of seniors were not proficient at the most basic level.

26

Fall 2001-Spring 2006 Writing Summary

LEVEL 1-WRITING

At level 1, a student demonstrates the following writing skills:• Recognize agreement among basic grammatical elements (e.g., nouns, verbs, pronouns, conjunctions)• Recognize appropriate transition words• Recognize incorrect word choice• Order sentences in a paragraph• Order elements in an outline

LEVEL 2-WRITING

At level 2, a student demonstrates the following writing skills:• Incorporate new material into a passage• Recognize agreement among basic grammatical elements (e.g., nouns, verbs,pronouns, conjunctions) when these elements are complicated by intervening words or phrases

• Combine simple clauses into single, more complex combinations• Recast existing sentences into new syntactic combinations

27

LEVEL 3-WRITING

At level 3, a student demonstrates the following writing skills:• Discriminate between appropriate and inappropriate uses of parallelism• Discriminate between appropriate and inappropriate uses of idiomatic language• Recognize redundancy• Discriminate between proper and improper constructions• Recognize the most effective revision of a sentence

28

Mathematics S cores

During the semesters between fall 1992 and summer 1997, the percentage of students who met proficiencies at Level 1 and at Level 2 were equivalent. When the percentages for each year are averaged, 31% of the students met the requirements for each of these two levels. Of the remaining students, 25% performed at Level 3. Thirteen percent of the students did not meet the requirements of the most basic level.

From fall 1997 through summer 2001, the highest percentage of students (33%) performed proficiently at Level 1. Approximately 29% of students performed at Level 2. Of the remaining students, 23% performed at Level 3. Roughly 15% of the students did not meet the requirements of the most basic level.

Approximately 68% of the seniors tested during the fall 2001 through spring 2006 semesters were proficient at Level 1 of the criterion-referenced mathematics dimension. Thirty-nine percent of seniors were proficient at Level 2 mathematics. Almost nine percent of the seniors met the proficiency requirements for Level 3. Eight percent did not meet the specific requirements in the most basic level of mathematics.

The national comparison data provided a summary of proficiency classifications for mathematics. The percentage of seniors showing proficiency at Level 1 was 60%, for Level 2, 31%, and Level 3, 8%. Fourteen percent of seniors were not proficient at the most basic level.

29

Fall 2001-Spring 2006 Mathematics Summary

LEVEL I-MATHEMATICS

At level 1, a student is able to perform the following mathematical tasks:• Solve word problems that would most likely be solved by arithmetic and do not involve conversion of units or proportionality (these problems can be multi-step if the steps are repeated rather than embedded)• Solve problems involving the informal properties of numbers and operations, often involving the Number Line, including positive and negative numbers, whole numbers and fractions (including conversions of common fractions to percent, such as converting 1/4 to 25%)• Solve problems requiring a general understanding of square roots and the squares of numbers• Solve a simple equation or substitute numbers into an algebraic expression.• Find information from a graph (this task may involve finding a specified piece of information in a graph that also contains other information)

LEVEL 2-MATHEMATICS

30

At level 2, a student is able to perform the following mathematical tasks:• Solve arithmetic problems with some complications, such as complex wording, maximizing or minimizing, and embedded ratios (these problems include algebra problems that can be solved by arithmetic: the answer choices are numeric)• Simplify algebraic expressions, perform basic translations, and draw conclusions from algebraic equations and inequalities (these tasks are more complicated than solving a simple equation, though they may be approached arithmetically by substituting numbers)• Interpret a trend represented in a graph, or choose a graph that reflects a trend• Solve problems involving sets (the problems would have numeric answer choices)

LEVEL 3-MATHEMATICS

At level 3, a student is able to perform the following mathematical tasks:• Solve word problems that would be unlikely to be solved by arithmetic; the answer choices are either algebraic expressions or

are numbers that do not lend themselves to back-solving • Solve problems involving difficult arithmetic concepts such as exponents and roots other than squares and square roots and percent of increase or decrease • Generalize about numbers, e.g., identify the values of (x) for which an expression increases as (x) increases• Solve problems requiring an understanding of the properties of integers, rational numbers, etc. • Interpret a graph in which the trends are to be expressed algebraically or in which one of the following is involved: exponents and roots other than squares and square roots, percent of increase or decrease• Solve problems requiring insight or logical reasoning

Reading and Criti cal Thinking Scores

31

During the semesters between fall 1992 and summer 1997, 29% of the students met proficiencies at Level 1. When the percentages for each year were are averaged, the largest percentage of students (46%) performed the skills necessary to meet the requirements of Level 2. Approximately 19% of the students were able to demonstrate the skills necessary for Level 3, which is the critical thinking scale. Seven percent of students did not meet the requirements of the most basic level.

For the semesters of fall 1997 through summer 2001, 30% of the students performed at Level 1. Forty-four percent performed the skills necessary to meet the requirements of Level 2. Roughly 18% of the students were able to demonstrate the skills necessary for the critical thinking scale, and 8% of students did not meet the requirements of the most basic level.

From fall 2001 through spring 2006, 77% of the tested seniors were proficient at Level 1 for the criterion-referenced reading and critical thinking dimension. Forty-seven percent were proficient at Level 2. Almost seven percent of seniors tested in fall 2001 through spring 2006 semesters were proficient at Level 3, critical thinking, and 8% did not meet the specific components of the most basic level of reading.

The national comparison data provided a summary of proficiency classifications for reading/critical thinking. The percentage of seniors showing proficiency at Level 1 was 73%, for Level 2, 40%, and Level 3, 6%. Ten percent of seniors were not proficient at the most basic level.

Fall 2001-Spring 2006 Reading Summary

32

LEVEL 1-READING

At level 1, a student is able to perform the following tasks:• Recognize factual material explicitly presented in a reading passage• Understand the meaning of words or phrases in the context of a reading passage

LEVEL 2-READING

At level 2, a student is able to perform the following reading tasks:• Synthesize material from different sections of a passage• Recognize valid inferences derived from material in the passage• Identify accurate summaries of a passage or of significant sections of the passage• Understand and interpret figurative language• Discern main idea, purpose, focus of a passage, or a significant portion of a passage

33

LEVEL 3-CRITICAL THINKING

At level 3, a student can perform the following critical thinking tasks:• Evaluate competing casual explanations• Evaluate hypotheses for consistency with known facts• Determine the relevance of information for evaluating an argument or conclusion• Determine whether an artistic interpretation is supported by evidence contained in a work• Recognize the salient features or themes in a work of art• Evaluate the appropriateness of procedures for investigating a question of causation• Evaluate data for consistency with known facts, hypotheses, or methods• Recognize flaws and inconsistencies in an argument

34

The Academic Profile uses three types of data to examine the general education of Missouri State University students: norm-referenced, student-reported, and criterion-referenced. Norm-referenced data is meaningful in comparison to other scores on the same test. Such comparisons can be made across Missouri State University students, as well as other institutions. Student-reported data allows trend comparisons in reference to particular subgroups of students. These subgroups include class level, grade point average, approximate percentage of fulfilled general education requirements, work hours, and credit hours. Criterion-referenced data depends on specific competencies expected of students. Specific proficiencies are defined and students are assessed to determine their level of proficiency. Issues pertaining to each type of data are examined below.

Issues that P ertain to Norm-Referenced Data

Norm-referenced scores have meaning only in comparison to other scores on the same test. Scores of other students, scores at different time periods, or scores from other colleges and universities are all examples of norm-referenced scores. The Educational Testing Service and The College Board provided norm-referenced comparative scores for comparison with institutional means. The most appropriate set of comparative scores for Missouri State University students was the set for Master’s (Comprehensive) Colleges and Universities I and II-Senior.

The mean total score for graduating Missouri State University students who took the Academic Profile were above the national comparative mean for Master’s (Comprehensive) Colleges and Universities I and II-Senior for all but one of the nine semesters within the 2003 through 2006 fiscal years. With the exception of the summer semester of 2004, this result is consistent with the goal set forth for the University in the Performance Measures found in the Missouri State University Long-Range Plan. Even though the mean for summer 2004 graduates was slightly below the national comparative mean, the Missouri State University mean for the fiscal year met the goal. The goal states that the “institutional mean will meet or exceed the national comparative mean.”

The norm-referenced college mathematics, critical thinking, reading, writing and natural sciences mean scores for seniors were at or above the national comparative for Master’s (Comprehensive) Colleges and Universities I and II-Senior for all nine semesters from summer 2003 through spring 2006. Of the remaining norm-referenced scores, seniors’ scores on norm-referenced social science and humanities scores each fell below the national comparative mean score for two of the nine semesters.

35

Concluding Discussion

Issues that Pertain to Mean Total Scores for Subgroups of Student-Reported Data

Over the past fourteen years, the average score for students who reported being seniors was consistently higher than the average score for students who reported being juniors. Between fall 1992 and spring 2006, seniors had a mean total score of 452.42 and juniors had a mean total score of 446.75. In addition, the total score for seniors was more stable from semester to semester than the score for juniors. Mean scores for seniors ranged from 449 to 456. Mean scores for juniors ranged from 437 to 456.

Mean total scores increased relative to increases in self-reported grade point averages during the fourteen year span of the Academic Profile. Between 1997 and 2006, the average total score for students who reported having a GPA of 3.5 to 4.0 was 460.83. The mean total score for students with a GPA of 3.0 to 3.49 was 451.60. Students with a GPA of 2.5 to 2.99 had a mean total score of 445.30. Students with a GPA of 2.0 to 2.49 had a mean total score of 443.36, and the mean total score of students reporting a GPA of 1.0 to 1.99 was 425.50.

Between fall 1992 and summer 2001, student-reported numbers of hours worked were selected based on the options of: none, one to five hours, six to 10 hours, 11 to 20 hours, 21 to 30 hours, 31 to 40 hours, and more than 40 hours. Mean scores for these were 449.45, 453.59, 451.55, 451.09, 450.32, 452.24, and 453.05 respectively. Beginning in summer 2003, the categories for reporting number of hours worked were changed to include options of: none, one to five hours, 16 to 30 hours, and more than 30 hours. Students who worked one to 15 hours per week had the highest mean total score of 452.17, while students who reported working 30 plus hours per week had the lowest mean total score of 449.98. Students who did not work had a mean total score of 451.23, and students who reported having worked 16 to 30 hours per week had a mean total score of 451.55.

Between fall 1992 and spring 2006, students who transferred one to 15 credits to Missouri State University received a higher mean total score than students who did not transfer credits or who transferred more than 15 credits. Students who transferred one to 15 credits had a mean total score of 452.88. Students who did not transfer had a mean total score of 451.88. Students who transferred 16 to 30 credits had a mean total score of 451.29. Students who transferred 30 or more credits had the lowest mean total score of 450.19.

Issues that Pertain to Criterion-Referenced Data

Criterion-referenced scores depend on specific competencies expected of students. Student scores on the three dimensions (writing, mathematics, and reading/critical thinking) are grouped into three categories: not proficient, marginal, and proficient. Students who do not meet basic competencies in the dimensions are classified as “not proficient”. Students whose test results do not provide enough evidence to categorize

36

them as not proficient or proficient are classified as “marginal”. Students who meet the competency requirements within a specific skill dimension are classified as “proficient. The three hierarchical levels within each skill dimension indicate that students who are proficient at the higher levels are also proficient at the lower levels. The national comparison data was obtained through the Educational Testing Service website. The data was gathered during the academic year 2001-2002, and represents seniors under the Master’s (Comprehensive) Colleges and Universities I and II classification. This comparison data provided a summary of proficiency classifications for writing, mathematics, and reading/critical thinking.

Writing Scores

Criterion-referenced writing scores indicated that during the semesters between fall 1992 and summer 1997, an average of 32% of the students met proficiencies at Level 2. The largest average percentage of students (42%) performed the skills necessary to meet the requirements of Level 1. An average of 16% of students were able to demonstrate the skills necessary for Level 3. An average of 11% of Missouri State University students did not meet the requirements of the most basic level.

For the semesters of fall 1997 through summer 2001, an average of 30% of the students performed at Level 2. The largest average percentage of students (42%) performed the skills necessary to meet the requirements of Level 1. An average of 16% met the requirements of Level 3. An average of 13% did not meet the requirements of the most basic level.

From fall 2001 through spring 2006, an average of 78% of the tested seniors were proficient at Level 1 for the criterion-referenced writing dimension and 25% of seniors were proficient at Level 2. Twelve percent of seniors tested in fall 2001 through spring 2006 semesters were proficient at Level 3. An average of 5% of the seniors tested did not meet the specific components of the most basic level of writing. Definitions of each level for the criterion-referenced writing scores are in Appendix B.1.

Mathematics Scores

Criterion-referenced mathematics scores indicated that during the semesters between fall 1992 and summer 1997, the average percentage of students who met proficiencies at Level 1 and at Level 2 were equivalent. An average of 31% of the seniors met the requirements for each of these two levels. Of the remaining seniors, an average of 25% performed at Level 3. On average, 13% did not meet the requirements of the most basic level.

From fall 1997 through summer 2001, the highest percentage of students (33%) performed proficiently at Level 1. An average of 29% performed at Level 2. Of the remaining students, an average of 23% performed at Level 3. An average of 15% did not meet the requirements of the most basic level.

37

An average of 68% of the seniors tested during the fall 2001 through spring 2006 semesters were proficient at Level 1 of the criterion-referenced mathematics dimension. An average of 39% of seniors was proficient at Level 2 mathematics. An average of 9% of the seniors met the proficiency requirements for Level 3. Eight percent of the seniors did not meet the specific requirements in the most basic level of mathematics. Descriptions of the criteria for each level of the mathematics scale are in Appendix B.2.

Reading and Critical Thinking Scores

Criterion-reference reading/critical thinking scores indicated that during the semesters between fall 1992 and summer 1997, an average of 29% of the students met proficiencies at Level 1. The largest average percentage of students (46%) performed the skills necessary to meet the requirements of Level 2. An average of 19% of the students were able to demonstrate the skills necessary for Level 3, which is the critical thinking scale. An average of 7% of the students did not meet the requirements of the most basic level.

For the semesters of fall 1997 through summer 2001, an average of 30% of the students performed at Level 1. The largest average percentage of students (44%) performed the skills necessary to meet the requirements of Level 2. An average of 18% of the students was able to demonstrate the skills necessary for the critical thinking scale. An average of 8% of the students did not meet the requirements of the most basic level.

From fall 2001 through spring 2006, an average of 77% of the tested seniors were proficient at Level 1 for the criterion-referenced reading and critical thinking dimension. An average of 47% of seniors were proficient at Level 2. Seven percent of seniors tested in fall 2001 through spring 2006 semesters were proficient at Level 3- critical thinking. An average of 8% of the seniors tested did not meet the specific components of the most basic level of reading. Descriptions of the criteria for each level of the reading or critical thinking scale are in Appendix B.3.

Conclusion

This report summarized the results of the Academic Profile examination given to juniors and seniors at Missouri State University between fall 1992 and spring 2006. The purpose of giving the Academic Profile to students seeking baccalaureate degrees was to test the general academic knowledge and skills of the students in order to assess the outcomes of the general education program and, in turn, to improve the quality of instruction and learning. A goal for the University was to test 100% of the graduating students for each semester. The percentage of students who took the Academic Profile was 99.29% in the fiscal year of 2004, 99.80% in the fiscal year of 2005, and 99.52% in the fiscal year of 2006.

A total of 32,856 Missouri State University students have taken the examination since fall 1992. Of these students, 29,126 (99%) reported being of either junior or senior class standing. Of the 32,856 students who took the examination, 29,574 (90%) reported that they had completed all general education requirements, 2,775 students (8%) reported that

38

they had completed 75% of the general education requirements, and 108 students (0.5%) reported that they had completed 50% or less of the general education requirements prior to taking the examination. The remaining students did not indicate the percentage of the general education requirements completed prior to taking the examination. The remaining students did not indicate the percentage of the general education requirements completed prior to taking the examination. The data indicates that most of the students who took the examination had completed the general education requirements prior to taking the examination. A large portion of the remaining students took a substantial percentage of the required courses before taking the Academic Profile.

This report presented the results of the Academic Profile for the most recent fiscal year and summarized the data collected over the last fourteen years. The report concludes the administration of this edition of the general education examination produced by the Educational Testing Service. The Academic Profile is being discontinued and replaced by the Measure of Academic Proficiency and Progress (MAPP). This new general education examination will produce scores that are comparable to the scores produced by the Academic Profile according to the Educational Testing Service.

39

Appendix ASpecific Materials Covered in the Humanities, Social Sciences, and

Natural Sciences AreasHUMANITIES

The humanities materials include selections (poetry, and fiction and nonfiction prose) from classical literature and from American and British literature. The materials also include discussions of key terms in the humanities. Visual elements are also presented as well as discussions of painting, music, philosophical issues, and materials by and about women and people of color.

SOCIAL SCIENCES

The social sciences materials present topics and issues that might be discussed in a general education course in history, economics, political science, psychology, anthropol-ogy, sociology, or indeed in any general education social science course. The materials are often interdisciplinary in their approach to a topic; that is, they draw on more than one of the social science fields in their presentation of a topic. Social science topics of particular relevance to women and people of color are represented in the materials, which include reading selections and graphs and/or tables.

NATURAL SCIENCES

The natural sciences materials are often interdisciplinary in nature; that is, they present topics and issues that might be discussed in a general education course in biology, chemistry, or physics. In addition, the materials may present experimental findings in ways that are familiar to a student who has taken any kind of general education science course. Where materials are not interdisciplinary, topics and issues in biology will receive more emphasis than those in chemistry and physics. The materials include reading selections and graphs and/or charts.

40

Appendix B.1Proficiency Level Definitions for Writingi

LEVEL 1-WRITING

A student at Level 1 recognizes agreement among basic elements (nouns, verbs, pronouns) in the same clause or phrase. This student avoids gross errors in short or simple structures and can logically select and order main ideas or divisions in a sustained paragraph using appropriate transition words. Students at this level demonstrate a basic understanding of appropriate writing.

LEVEL 2-WRITING

In addition to performing successfully at Level 1, a student who is proficient at Level 2 also recognizes appropriate agreement among basic elements when they are compli-cated by intervening words or phrases, avoids errors in relatively long and complicated constructions, and is able to recast several simple clauses using a single, more complex combination. Students performing at this intermediate level can recognize and use the elements of good writing.

LEVEL 3-WRITING

In addition to performing Level 1 and Level 2 skills successfully, a student at Level 3 also can identify logical statements and comparisons and is able to solve difficult or subtle writing problems such as appropriate use of parallelism. These students can make fine distinctions among closely related root words and grammatical structures characteristic of a mature writing style.

iThe writing proficiency level definitions were scanned from “The Academic Profile Faculty Information Booklet" (p. 22) downloaded at www.ETS.org (Copyright 2002).

41

Appendix B.2Proficiency Level Definitions for Mathematicsii

LEVEL I-MATHEMATICS

A student at Level 1 demonstrates basic number sense and skills in arithmetic operations and relationships and in elementary geometry and measurement. A student at this level can read and interpret information from simple graphs or charts, solve simple equations or evaluate expressions, and solve simple and routine word problems.

LEVEL 2-MATHEMATICS

In addition to performing successfully at Level 1, a student who is proficient at Level 2 also understands number systems, including order, magnitude, and relationship of integers, fractions, and decimals. A student at this level can solve moderately difficult equations and inequalities, evaluate complex formulas, compare and apply information from more complex charts and graphs, and apply reasoning, geometry, and measurement skills in solving moderately complex problems including word problems.

LEVEL 3-MATHEMATICS

In addition to performing Level 1 and Level 2 skills successfully, a student at Level 3 also can generalize and apply mathematical knowledge and skills in non-routine situations, and demonstrates real comprehension of exponents, variables, geometry, and measurements. A student at this level can solve multi-step and non-routine problems involving a range of reasoning skills.

ii The mathematics proficiency level definitions were scanned from “The Academic Profile Faculty Information Booklet" (pp. 22-23) downloaded at www.ETS.org (Copyright 2002).

42

Appendix B.3Proficiency Level Definitions for Reading/Critical Thinkingiii

LEVEL 1-READING

At Level 1 a student recognizes and comprehends discrete pieces of information, (e.g., a single detail, information presented in a single sentence) as well as relationships or connections explicitly stated in a passage and understands words and phrases in context.

LEVEL 2-READING

In addition to performing successfully at Level 1, students who are proficient at Level 2 can also gather information from different sections of a passage and recombine it. These students recognize relationships that can be inferred but are not explicit: they can recognize summaries and alterative ways of stating information, interpret figurative language, and recognize the point or purpose of a passage as a whole or significant portion of a passage.

LEVEL 3-CRITICAL THINKING

In addition to performing Level 1 and Level 2 skills successfully, students at Level 3 can also evaluate and analyze arguments and, within an academic field, handle interpretation, inductive generalizations, or causal explanations.

Level 3 skills are differentiated within those areas as follows:

Humanities: Evaluate views and interpretations Social Sciences: Evaluate claims, disputes, and inductive generalizations

iii The reading/critical thinking proficiency level definitions were scanned from “The Academic Profile Faculty Information Booklet" (p. 21) downloaded at www.ETS.org (Copyright 2002).

43

Natural Sciences: Evaluate explanatory hypotheses and draw conclusions

44

Appendix B.4.Proficiency Level Classifications for Reading/Critical Thinking, Writing, and

Mathematicsiv

PROFICIENCY LEVELS FOR READING/CRITICAL THINKING

LEVEL 1At level 1, a student is able to perform the following tasks:• Recognize factual material explicitly presented in a reading passage• Understand the meaning of words or phrases in the context of a reading passageLEVEL 2At level 2, a student is able to perform the following reading tasks:• Synthesize material from different sections of a passage• Recognize valid inferences derived from material in the passage• Identify accurate summaries of a passage or of significant sections of the passage• Understand and interpret figurative language• Discern main idea, purpose, or focus of a passage or a significant portion of a passageLEVEL 3At level 3, a student can perform the following critical thinking tasks:• Evaluate competing casual explanations• Evaluate hypotheses for consistency with known facts• Determine the relevance of information for evaluating an argument or conclusion• Determine whether an artistic interpretation is supported by evidence contained in a work• Recognize the salient features or themes in a work of art• Evaluate the appropriateness of procedures for investigating a question of causation• Evaluate data for consistency with known facts, hypotheses, or methods• Recognize flaws and inconsistencies in an argument

PROFICIENCY LEVELS FOR WRITING SKILLS

LEVEL 1At level 1, a student demonstrates the following writing skills:• Recognize agreement among basic grammatical elements (e.g., nouns, verbs, pronouns, conjunctions)• Recognize appropriate transition words• Recognize incorrect word choice• Order sentences in a paragraph• Order elements in an outlineLEVEL 2At level 2, a student demonstrates the following writing skills:• Incorporate new material into a passage• Recognize agreement among basic grammatical elements (e.g., nouns, verbs, pronouns, conjunctions) when these elements are complicated by intervening words or phrases• Combine simple clauses into single, more complex combinations• Recast existing sentences into new syntactic combinationsLEVEL 3At level 3, a student demonstrates the following writing skills:

• Discriminate between appropriate and inappropriate uses of parallelism• Discriminate between appropriate and inappropriate uses ofidiomatic language• Recognize redundancy• Discriminate between proper and improper constructions• Recognize the most effective revision of a sentence

PROFICIENCY LEVELS FOR MATHEMATICAL SKILLS

LEVEL I - MATHEMATICSAt level 1, a student is able to perform the following mathematical tasks:• Solve word problems that would most likely be solved by arithmetic and do not involve conversion of units or proportionality These problems can be multi-step if the steps are repeated rather than embedded. • Solve problems involving the informal properties ofnumbers and operations, often involving the Number Line, including positive and negative numbers, whole numbers and fractions (including conversions of common fractions to percent, such as converting 1/4 to 25%).• Solve problems requiring a general understanding of square roots and the squares of numbers. • Solve a simple equation or substitute numbers into an algebraic expression.• Find information from a graph. This task may involve finding a specified piece of information in a graph that also contains other information.LEVEL 2 - MATHEMATICSAt level 2, a student is able to perform the following mathematical tasks:• Solve arithmetic problems with some complications, such as complex wording, maximizing or minimizing, and embedded ratios. These problems include algebra problems that can be solved by arithmetic (the answer choices are numeric). • Simplify algebraic expressions, perform basic translations, and draw conclusions from algebraic equations and inequalities. These tasks are more complicated than solving a simple equation, though they may be approached arithmetically by substituting numbers. • Interpret a trend represented in a graph, or choose a graph that reflects a trend.• Solve problems involving sets; the problems would have numeric answer choices.LEVEL 3 - MATHEMATICSAt level 3, a student is able to perform the following mathematical tasks:• Solve word problems that would be unlikely to be solved byarithmetic; the answer choices are either algebraic expressions orare numbers that do not lend themselves to back-solving. • Solve problems involving difficult arithmetic concepts such as exponents and roots other than squares and square roots and percent ofincrease or decrease. • Generalize about numbers, e.g., identify the values of (x) for which an expression increases as (x) increases.

iv Scanned from “The Academic Profile Faculty Information Booklet” (pp. 21-23) downloaded at www.ETS.org (Copyright 2002).

45

• Solve problems requiring an understanding of the properties of integers, rational numbers, etc. • Interpret a graph in which the trends are to be expressed algebraically or in which one of the following is involved:

exponents and roots other than squares and square roots, percent of increase or decrease. • Solve problems requiring insight or logical reasoning

Appendix C.1Colleges and Universities Included in Comparison Group for Summer 2003 Through

Spring 2006v

The comparative means for Comprehensive Colleges and Universities-Senior are based on data from 2001 and 2002 administrations of the Academic Profile at the institutions listed below. Only those institutions testing 30 or more students in a college class were included in the analyses for that college class

Institutions Included in this ReportAngelo State University, TX

Armstrong Atlantic State University, GA

Auburn University- Montgomery, AL

Austin Peay State University, TN

Baldwin-Wallace College, OH

Bemidji State University, MN

Benedictine College, KS

Bloomsburg University- Pennsylvania PN

Campbell University, NC

Cardinal Stritch University, WI

Central Missouri State University, PA

Chadron State College, NE

Charleston Southern University, PA

College of St. Scholastica, MN

East Stroudsburg University, PA

Edgewood College, WI

Ferris State University, MI

Friends University, KS

Governors State University, IL

Grambling State University, LA

Harding University, AR

Houston Baptist University, TX

Indiana Wesleyan University, IN

Kean College of New Jersey, NJ

Lake Superior State University, MI

Le Tourneau University, TX

Lincoln University at Missouri, MO

Long Island University, NY

Lynchburg College, VA

Marshall University, WV

Mary Baldwin College, VA

Maryville University - St. Louis, MO

Mississippi University for Women, MS

Morehead State University, KY

Nicholls State University, LA

North Carolina Central University, NC

Northwest Missouri State University, MO

Parker College of Chiropractic, TX

Pfeiffer University, NC

Rivier College, NH

Rockford College, IL

Saint Leo University, FL

Shenandoah Unviersity, VA

Missouri State University, MO

The University of Tennessee at Martin, TN

Trevecca Nazarene University, TN

Troy State University, AL

Tusculum College, TN

University of Southern Colorado, CO

University of Rio Grande, OH

University of Colorado-Colorado Springs, CO

University of Mary, ND

University of Mobile – AL

University of Northern Iowa, IA

46

University of Saint Francis at Indiana, IN

University of Southern Indiana, IN

University of Wisconsin Oshkosh, WI

Walla Walla College, WA

47

v

? Scanned from “The Comparative Data Guide: Introduction” downloaded at www.ETS.org (Copyright 2002).

48