indicators of post-secondary - kansas...

Indicators of Post-Secondary

…and Beyond… Success

John Poggio and Susan Gillmor

School of Education

University of Kansas

January 30, 2014

Poggio & Gillmor, Kansas State Legislative Presentation 2

While ability has been shown to have a positive, direct

relationship with school performance, ability has low,

that is, poor (and overrated and over assumed!)

predictive power for indicators of life success.

The pioneers of intelligence testing cautioned that

ability cannot explain much of the important

particulars associated with success in school and

beyond:

“…to succeed in his studies, one must have qualities which

depend on attention, will, and character; for example a certain

docility, a regularity of habits, and especially continuity of effort.

A child, even if intelligent, will learn little in class if he never

listens, if he spends his time in playing …, in giggling, in being

truant” (Binet, 1916, p.254).

Later, Wechsler, who is largely responsible for

popularizing the movement in intelligence testing,

cautioned that ability measures must be used in

conjunction with non-cognitive indicators to properly

and thoroughly predict of success:

“When our scales measure the non-intellective as well as the

intellectual factors in intelligence, they will more nearly

measure what in actual life corresponds to intelligent behavior.

Under these circumstances they… should do a much better job

in selecting those destined to succeed in life” (Wechsler, 1943,

p. 103).

3 Poggio & Gillmor, Kansas State Legislative Presentation

What’s it mean? To illustrate:

3/24/2014 4

+

C

O

L

L

E

G

E

G

P

A

_

- HS TEST SCORES +

A picture…A thousand words…A summary

On the chart each marker is an actual person

Looking to the left and right are Lower (-) and

Higher (+) high school scores, respectively

Moving up and down are Lower (-) and Higher

(+) college GPA scores, respectively

The swarm (or Plot) of points demonstrates the

maximum relationship (i.e., as good as it gets!)

between HS test scores (e.g., ACT scores)

and college GPA (i.e., r = .50)

What we find is that accepting individuals

based largely on high (+) HS test performance

or GPA denies many who would be successful

in postsecondary schooling…

3/24/2014 5

Grit – Defined as perseverance (i.e., determination,

resolute, doggedness, tenacity, dedication, drive,

persistence, stubbornness, and on and on…) and

passion for long-term goals (Duckworth, 2013).

“Grit, passion and perseverance for very long term goals. Grit is

having stamina. Grit is sticking with your future, day-in day-

out, not just for the week, not just for the month, but for years.

And working really hard to make that future a reality. Grit is

living life like it’s a marathon, not a sprint” (Duckworth, 2013).

Grit has been documented to explain and account for

school and life success above and beyond what can

be explained by intelligence alone. Study after study

find that school success not only requires a modicum

of ability, but also the sustained and focused

application of that ability over time – Grit!


We currently have a serious grit issue in our institutions

of higher education. Nationally, only 55% of the

students who go to college leave with a degree. By

comparison, among our peer industrial nations, the

United States ranks ninth in post-secondary enrollment,

but last in college graduation rates.

This very low, if not appalling, “retention to graduation”

rate is not at all surprising when admissions decisions

are largely, if not exclusively in some situations, based

on cognitive indicators as tests scores and high school

GPA. College admission tests, other traditional academic

indicators as GPA plus state assessment scores, only

explain 10-30% of the variance in first-year college GPA

(Linn, 1990). The Education literature has identified a

group of variables that can more accurately and fully

predict college retention and success.


The Other KEY Predictors of College GPA

Grit/Persistence –ability to stick with challenging tasks to

achieve long-term goals

Willingness to study –students’ commitment to individual

learning outside the classroom

Interest in school

Time spent on outside school/academic related activities

–i.e. leadership, community service, commitment

Organization skills –ability to manage the many and

diverse demands of college life in a structured way

Self-worth and Self-efficacy –the feeling of ones own

deservedness and ability

Goal setting

Expectancy for success

Academic self-confidence

Achievement motivation –the student’s internal desire to

do well in her/his academic pursuits/interests 8

Poggio & Gillmor, Kansas State Legislative Presentation

Adaptability –ability to learn and change in order to thrive

in the college environment.

Stress Management – ability to self-regulate and cope with

the pressures of college and life

Intrapersonal/interpersonal skills –ability to maintain

positive relationships with self and others in the college

environment

Academic self-confidence –sureness in one’s own academic

capability

Academic goal setting –ability to create and track

achievable objectives for oneself

Institutional commitment –the level of satisfaction and

belongingness students feel while at the college

Social support and involvement –feeling of support from

peers when participating in activities in- and outside the

academic setting

9

Predictors of Retention and Success


Existing Measures

Some trait measures presented on the previous slides

have been evaluated in the literature using the tools

listed below. Due to the high-stakes associated with

college admissions, more advanced tools still need to

be developed:

10

Instrument

Study Management and Academic Results Tests (SMART)

Test of Reaction and Adaptation to College (TRAC)

Non-cognitive Questionnaire (Tracey & Sedlacek, 1984)

Motivated Strategies for Learning Questionnaire (Pintrich, 1989)

Emotional Quotient Inventory (EQ-i:Short; Bar-On, 2002)


How much more/improved can

postsecondary success be predicted?

• Duckworth (2007) finds that “Grit” accounts for an average

of 4% of the variance in college success outcomes including

educational test scores and GPA.

• A measure of college students’ ability to adapt to college life

called the Test of Reaction and Adaptation to College

(TRAC) has been found to account for 21% of the variance

in college GPA and is effective for identifying students “at-

risk” (Falardeau, Larose, & Roy, 1989).

• Naumann, Bandalos & Gutkin (2003) find that expectancy

for success among college students accounts for

approximately 34% of the variance in college GPA

compared to 16% explained by ACT scores.

• Ting (1998) finds improvement as a result of the opportunity

for community service and successful leadership experience

which explains an additional 30% of the variance in GPA

for college students.

3/24/2014 11

When we make college admission decisions based

only on cognitive indicators of ability and

achievement, who are we excluding?

The Dean of the University of Kansas in the 1950’s,

George Baxter Smith, conducted a study of KU

graduates who would not have had the opportunity to

attend and complete KU programs had the

university screened them out based on their

entrance exam scores.


“208 graduates of the Class of 1955 at the University of

Kansas would not have been admitted as freshmen if the

“cutting score” of the fiftieth percentile had been in

operation. The loss to the state of Kansas and nation

would have been forty teachers, twenty-two engineers,

five journalists, seven lawyers, seven doctors, seven

pharmacists, and ninety-six graduates from the College of

Liberal Arts and Sciences and the School of Business

who majored in areas where the supply of trained

manpower is in equally short supply” (Smith, 1956, p.28).

The point being, selection based primarily on

cognitive indicators excludes Kansans who have the

potential to not only succeed in college but transition

well into society and become contributing members

to our society and of our economy.


To improve our graduation rates and also the

strength of our workforce, we need to be resolute to

inviting those who are likely to succeed into our

post-secondary programs. As we have seen, it is not

those with the highest test scores or GPAs, but those

also with the grit, passion and the motivation to

apply learned skills to achieve their academic and

career goals.

Not all elected officials in this room were “straight

A” students…. (and I can assure you, neither was

I!). We each have a personal story … Thank

goodness someone was caring, attentive, supportive

and listening. And, that as individuals we were

persistent, determined and cared for what we do…


So, what can we do?

The non-cognitive, i.e., non-academic, indicators of

success are real, reliable, dependable, and trustworthy.

We need to be monitoring and studying these factors,

and evaluating how we can ensure they are considered

in postsecondary admissions and systematically

throughout the postsecondary matriculation process.

Next steps:

1.We need to survey and confirm with employers to

identify the essential and key traits and behaviors that

make graduates desirable in the job market.

2.Evaluate ways and which non-cognitive traits should

be included and purposively streamed lined (e.g., those

previously identified) in the college admissions

process.


Other ideas to explore may include more emphasis

on the personal statement, extracurricular resume,

and recommendations from outside persons for the

applicant speaking to the factors mentioned earlier.

Active, scholarly consideration must also be given to

the idea of developing a Kansas non-cognitive

measure that has strong predictive validity for

college retention and post-graduate employment.

This measure would likely include questions related

to ability, achievement, grit and the affective domain.


Final Thoughts

Not only must we use what we know about the value and

importance of “grit” in post-secondary admissions, but we

must teach …in fact, nurture… it! There is currently a

woefully inadequate presence of scholarly work not only on

how to measure persistence, but how to inform and guide our

students to be more gritty!

The final point of our agenda is to stress to you that knowing

about these non-cognitive indicators of success is not enough:

we need to know it, see it, and study and evaluate the best

methods of transmitting these habits of the mind and intellect

which then will help to breed success for our children, youth

and adults young and old. Policy makers and institutions of

higher education must recognize and value these traits, and we

need to find the best way to explicitly include them in our K-

12, college and post grad curricula and experiences.


17

References

Ackerman, P. L. (1987). Individual differences in skill learning: An integration of psychometric and information

processing perspectives. Psychological Bulletin, 102, 3–27.

Annis, L. F. (1985). Student-generated paragraph summaries and the information-processing theory of prose

learning. Journal of Experimental Education, 54, 4–10.

Binet, A. , & Simon, T. ( 1916). Th e development of intelligence in children (The Binet-Simon Scale) .

Baltimore, MD : Williams & Wilkins.

Baker, C. R. (2012, December 11). Experts say building skills like grit and perseverance will help increase

college readiness in the U.S. Deseret News. Retrieved from

http://www.deseretnews.com/article/865568540/Experts-say-building-skills-like-grit-and-perseverance-

will-help-increase-college-readiness-in-the.html?pg=all

Bar-On, R. (2002). Bar-On Emotional Quotient Short form (EQ-i:Short): Technical manual. Toronto: Multi-

Health Systems.

Dickenson, D. J., & O-Connell, D. Q., (1990). The effect of quality and quantity of study on student grades.

Journal of Educational Research, 83, 227-231.

Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: perseverance and passion for long-

term goals. Journal of personality and social psychology, 92(6), 1087.

Duckworth, A. L. (2013, April). Angela Lee Duckworth: The key to success? Grit [Video file]. Retrieved from

http://www.ted.com/talks/angela_lee_duckworth_the_key_to_success_grit.html

Duckworth, A. L., & Carlson, S. M. (2013). Self-regulation and school success. Self-Regulation and Autonomy,

40, 208.

Falardeau, I., Larose, S., & Roy, R. (1989). Test de Réactions et d’Adaptation au Collégial (TRAC). Sainte-Foy:

Cégep de Sainte-Foy.

Gerdes, H., & Mallinckrodt, B. (1994). Emotional, social, and academic adjustment of college students: A

longitudinal study of retention. Journal of Counseling and Development, 72, 281–288.

Linn, R. L. (1990). Admission testing: Recommended uses, validity, differential prediction and coaching. Applied

Measurement in Education, 3(4), 297-318.

Lotkowski, V. A., Robbins, S. B., & Noeth, R. J. (2004). The role of academic and non-academic factors in

improving college retention. ACT Policy Report, 1.

Naumann, W. C., Bandalos, D., & Gutkin, T. B. (2003). Identifying Variables that Predict College Success for

First-Generation College Students. Journal of College Admission, 181, 4-9.

Poggio & Gillmor, Kansas State Legislative Presentation 18

References, cont.

Pancer, S. M., Hunsberger, B., Pratt, M. W., & Alisat, S. (2000). Cognitive complexity of expectations and

adjustment to university in the first year. Journal of Adolescent Research, 15, 38–57.

Parker, J. D., Summerfeldt, L. J., Hogan, M. J., & Majeski, S. A. (2004). Emotional intelligence and academic

success: Examining the transition from high school to university. Personality and individual differences,

36(1), 163-172.

Parker, J. D. A., Duffy, J., Wood, L. M., Bond, B. J., & Hogan, M. J. (2005). Academic achievement and

emotional intelligence: predicting the successful transition from high school to university. Journal of

First-Year Experience and Students in Transition, 17, 67–78.

Pintrich, P. R. (1989). The dynamic interplay of student motivation and cognition in the classroom. In C. Ames &

M. Maehr (Eds.), Advances in motivation and achievement: Vol. 6. Motivation enhancing environments.

(pp. 117-160). Greenwich, CT: JAI Press.

Pintrich, P. R. (1995). Understanding self-regulated learning. In P.R. Pintrich (Ed.), Understanding self-regulated

learning. San Francisco: Jossey-Bass Publishers.

Ransdell, S. (2001). Predicting college success: The importance of ability and non-cognitive variables.

International Journal of Educational Research, 35(4), 357-364.

Ryan, M. P. (1984). Monitoring text comprehension: Individual differences in epistemological standards. Journal

of Educational Psychology, 76, 248–258.

Ting, S. M. (1998). Predicting first-year grades and academic progress of college students of first-generation and

low-income families. Journal of College Admission, 158, 14-23.

Tracey, T. J., & Sedlacek, W. E. (1984). Non-cognitive variables in predicting academic success by race.

Measurement and Evaluation in Guidance, 16(4), 171-178.

Wechsler , D. ( 1943 ). Non-intellective factors in general intelligence. Journal of Abnormal & Social

Psychology, 38, 101– 103.

Williams, J. E., & Hellman, C. M. (1998). Investigating self-regulated learning among first-generation

community college students. Journal of Applied Research in the Community College, 5(2), 83-87.

Zajacova, A., Lynch, S. M., & Espenshade, T. J. (2005). Self-efficacy, stress, and academic success in college.

Research in Higher Education, 46(6), 677-706.


19

indicators of post-secondary - kansas...

Documents