2015 admissions and student diversity affairs professional development conference€¦ · ·...

Learn

Serve

Lead

2015 Admissions and Student Diversity Affairs Professional Development Conference

June 18, 2015

Association ofAmerican Medical Colleges

The New MCAT Exam: A Better Test For Tomorrow's Doctors

Association of American Medical Colleges

1

2015 AAMC Admissions and Student Diversity AffairsProfessional Development Conference

Session #1

Keeping Up with the Times: Building the New MCAT Exam

2015 AAMC Admissions and Student Diversity AffairsProfessional Development Conference2

2


AAMC Admissions Hubwww.aamc.org/admissions

3

Resources on Exam Content

• Using MCAT Data in 2016 Medical Student Selection

• Interactive Score Report (Facts sheets)

• Video on the new MCAT exam

• Academic Medicine collection of research on the new MCAT exam

• Academic Medicine Last Page comparing the new and old exams


Learning objectivesBy the end of this session, you will:

• Know what you want to tell to your committee about the concepts tested by the new exam

• Know what you want to tell your committee about the possible value of scores from the Psychological, Social, and Biological Foundations of Behavior section

• Develop a plan to familiarize your committee members with the concepts tested by the new exam

4

3


Refer to information in this guide during the session

5


19916

4



Time for change

• Periodic reviews are best practice

• Measures the most important competencies in the best ways

• Advances in medicine and scientific research

• Advances in education

1977

1928

1946

1991

2015

2030

1928

8

5


MR5 committee

9


6


MR5 committee members• Steven G. Gabbe, M.D. (Chair)

• Ronald D. Franks, M.D. (Vice Chair)

• Lisa T. Alty, Ph.D.

• Dwight Davis, M.D.

• J. Kevin Dorsey, M.D.

• Robert Hilborn, Ph.D.

• Barry A. Hong, Ph.D., M.DIV.

• Richard Lewis, Ph.D.

• Maria F. Lima, Ph.D.

• Catherine R. Lucey, M.D.

• Alicia Monroe, M.D.

• Saundra H. Oyewole, Ph.D.

• Erin A. Quinn, Ph.D., M.Ed.

• Richard K. Riegelman, M.D., Ph.D.

• Gary C. Rosenfeld, Ph.D.

• Wayne M. Samuelson, M.D.

• Richard M. Schwartzstein, M.D.

• Maureen Shandling, M.D.

• Catherine (Katie) Spina

• Ricci Sylla, M.D.

11


Blueprints are evidence-based• More than 90 outreach events

• More than 2,700 surveys

Roadmap to Diversity: Integrating Holistic Review Practices

Behavioral and Social Science Foundations for Future Physicians

Scientific Foundations for Future Physicians Report

12

7


Committee gathered input• Pre-health advisors

and baccalaureate faculty

• Medical school administrators and faculty

• Medical students

• Members of disciplinary societies and higher education associations

13


Which concepts are important for students to know at entry to medical school?

14

8


Survey participation was high• 143 US and Canadian medical schools were

invited to participate

• Collected data from over 1100 basic science faculty, clinical science faculty, residents and medical students

15


Are those concepts widely covered in undergraduate courses?

16

9


Survey participation was high• Sampled 290 undergraduate institutions, which

included: 169 schools not classified as minority-serving 121 schools classified as minority-serving

– 60 HBCUs– 59 Hispanic-serving– 2 Tribal-serving

• Over 1,000 undergraduate faculty participated in the survey.

17


Blueprints bring together medical school and undergraduate dataNew exam tests concepts that were:

• Rated as important by medical schools

• Covered by the majority of undergraduate institutions

18

10


Blueprints mirror expert’s recommendations

19

Behavioral and Social Science Foundations for Future Physicians

Scientific Foundations for Future Physicians Report

Biological and Biochemical Foundations of Living Systems Section

Chemical and Physical Foundations of Biological

Systems Section

Concept #1 Concept #2 Concept #3 Concept #4 Concept #5

Biomolecules have unique properties that determine how they contribute to the structure and function of cells, and how they participate in the processes necessary to maintain life.

Highly organized assemblies of molecules, cells, and organs interact to carry out the functions of living organisms.

Complex systems of tissues and organs sense the internal and external environments of multicellular organisms, and through integrated functioning, maintain a stable internal environment within an ever-changing external environment.

Complex living organisms transport materials, sense their environment, process signals, and respond to changes using processes that can be understood in terms of physical principles.

The principles that govern chemical interactions and reactions form the basis for a broader understanding of the molecular dynamics of living systems.

How these Competencies Lay the Foundation for Learning in Medical School

Building Block #1 Building Block #2 Building Block #3 Building Block #4 Building Block #5

With these building blocks, medical students will be able to learn how the major biochemical, genetic, and molecular functions of the cell support health and lead to disease.

With these building blocks, medical students will be able to learn how cells grow and integrate to form tissues and organs that carry out essential biochemical and physiological functions.

With these building blocks, medical students will be able to learn how the body responds to internal and external stimuli to support homeostasis and the ability to reproduce.

With these building blocks, medical students will be able to utilize core principles of physics to learn about the physiological functions of the respiratory, cardiovascular, and neurological systems in health and disease.

With these building blocks, medical students will be able to utilize core principles of human chemistry to learn about molecular and cellular functions in health and disease.

820

11


New exam tests

Concepts key to success

Use knowledge to solve problems

Think like physicians

Role of behavioral and socio-cultural factors

21


What Does the Exam Test?

22

12


Structure of the new MCAT exam

23


Old Content Covered

• Biology• Chemistry• Physics• Verbal reasoning

24

13


Old Content Covered

• Biology• Chemistry• Physics• Verbal reasoning

New Emphases• Application of

knowledge• Behavioral and

sociocultural aspects of health

• Latest science on information processing

New• Biochemistry• Psychology• Sociology

Broader preparation & ability to apply knowledge!

25


How is the new exam

organized?

26

14


9

27


10

28

15


11

29


8

30

16


Research methods and statistics concepts

• Basic probability

• Measures of central

tendency

• Measures of variability

• Confidence intervals

• Statistical significance

levels

• Graphical presentation of data

• Research ethics

• Hypothesis formation

• Independent and dependent variables

• Hypothesis testing

• Reporting research results

30


Putting it together

32

Foundational Concept 8

Psychological, sociocultural, and biological factors influence the way we think about ourselves and others, as well as how we interact with others.

Scientific Inquiry & Reasoning Skill 3

Reasoning and problem solving

Content Category 8B

Attributional Processes

35

17


13

33


Activity 1.1

34

18


Understanding How Scores from the Psychological, Social, and Biological

Foundations of Behavior Section Add Value

35


19


Committee members

• Sylvia Bereknyei, Ph.D., M.S.

• Steve Case, Ph.D.

• Jerry Clark, Ph.D., M.B.A.

• Jorge A. Girotti, Ph.D., M.A.

• Joshua Hanson, M.D., M.P.H.

• Loretta Jackson-Williams, M.D.

• David Jones, Ph.D.

• Dale Krams

• Jonathan Kibble, Ph.D.

• Susan Masters, Ph.D.

• Stephanie C. McClure, M.D., FACP

• Janet McHugh

• Wanda Parsons, M.D., FCFP

• Boyd F. Richards, Ph.D.

• Jason Satterfield, Ph.D.

• Aubrie Swan Sein, Ph.D., Ed.M.

• Robert A. Witzburg, M.D.

37


Early research• 2,000 medical students from these 11 schools

volunteered to take prototype of Psychological, Social, and Biological Foundations of Behavior section

• Mostly first- and second-year students

• Schools are collecting students’ grade and test scores from courses related to the behavioral and social sciences

• Correlate section scores with students’ course and clerkship grades

38

20


Sample of courses and clerkships included in the study

39

Pre-clerkship Courses in the Following Fields

Clerkships*

• Psychiatric Medicine• Mind, Brain, and Behavior• Brain and Behavior• Clinical Epidemiology• Population Health, Disease

Prevention, & Health Promotion

• Psychiatry• Neuroscience• Internal Medicine• Family Medicine• OB-GYN• “Rural”

*Only knowledge-based aspects of performance in clerkships, not behavioral aspects of performance


Scores correlate with academic performance in three types of medical school courses

40

26

21


27

Scores provide different information about Foundations of Psychiatric Medicine outcomes than scores from the old exam

41


27

Scores provide different information about Epidemiology and Public Health outcomes than scores from the old exam

42

22


Median Correlation

27

Scores provide different information about Neuroscience outcomes than scores from the old exam

43


ConclusionsScores from the Psychological, Social, and Biological Foundations of Behavior section:

• Correlate with students’ academic performance in medical school behavioral and social sciences courses

• Provide different information than scores from the old MCAT exam about students’ readiness to learn in medial school behavioral and social sciences courses

44

23


Next steps• This study will continue through 2016

• These results are from the first two (of five) panels of data we’ll collect

• Future data collections will include:

• Clerkship grades

• Step 1 and 2CK Behavioral and Social Sciences subscores

• NBME subject test scores

45


Activity 1.2

46

24


Activity 1.3

47


What are your colleagues thinking?

48

Dwight Davis

Worksheet: Changing the MCAT Exam

1

Session 1

List of resources on exam content

Using MCAT Data in 2016 Medical Student Selection

Interactive Score Report (Facts sheets)

Video on the new MCAT exam

Academic Medicine collection of research on the new MCAT exam

Academic Medicine Last Page comparing the new and old exams


2


3

Activity 1.1

Describe the Content of the New MCAT Exam

Directions:

Take a moment to make some notes about what you want to tell your committee (or others at your institution)

about the content tested by the new MCAT exam. Then, note how the new content is relevant to your

institutional mission and goals.

Exam Section New or Changed Content Relevance to Institutional Mission/Goals

Biological and

Biochemical Foundations

of Living Systems

Chemical and Physical

Foundations of Biological

Systems

Psychological, Social, and

Biological Foundations of

Behavior

Critical Analysis and

Reasoning


4


5

Activity 1.2:

Discuss the Psychological, Social, and Biological Foundations of Behavior Section

Directions:

Take a moment to make some notes about what you want to tell your committee (or others at your institution)

about how the scores from the Psychological, Social, and Biological Foundations of Behavior section may add

value to student selection at your school.

Exam Section Added Value to Admissions Process

Psychological, Social, and

Biological Foundations of

Behavior


6


7

Activity 1.3:

Develop a Plan to Familiarize Your Admissions Committee (or Others at your Institution)

with the Content of the New MCAT Exam

Directions:

Use the information from activities 1.1 and 1.2 to develop a plan for familiarizing your committee members with

the concepts tested by the next exam. Then, share your plan with a partner. Take note of any ideas that you

might want to incorporate into your plan.

Topic Message Related Resources


8

1


Session #2

Fairness and the New MCAT Exam



2

Resources on Fairness

• FACTS data on MCAT Scores (aamc.org/facts)


• “Do Racial and Ethnic Group Differences in Performance on the MCAT Exam Reflect Test Bias” (Academic Medicine, May 2013)

• Khan Academy collection and other free and low cost test preparation resources

2



• Know how fairness was key to the design and development of the new exam

• Vet ideas for helping your committee members consider applicants with a wider range of MCAT scores

• Learn about local efforts to help students prepare for the new exam

3


Refer to information in this guide during the session

4

3


Fairness played a critical role in blueprint development

5


Fairness played a critical role in blueprint development• Balance academic competencies from the

natural, behavioral, and social sciences

• Test psychology and sociology concepts like discrimination, stereotype threat, and socio-economic inequalities

• Increase attention to population health, studies of diverse cultures, ethics, philosophy

• Test concepts covered widely in undergraduate science courses

6

4


Fairness played a critical role in blueprint development• Longer sections that allow medical schools to

consider applicants’ strengths and weaknesses in relation to their curricula

• More working time per question, allowing test takers to work through the information presented in the passage, think about the question, and decide on the best answer

7


Fairness played a critical role in developing test questions

8

5


We used tried-and-true procedures to develop the new questions, and some new ones as well• Editorial review

• Technical review

• “Summits” with experts

• Bias and sensitivity review

• “Field testing” – try out questions to see how they work

9


Addressing Fairness when Developing Questions for the Psychological,

Social, and Biological Foundations of Behavior Section

10

6


Potentially sensitive contentSome content in the Psychological, Social, and Biological Foundations of Behavior Section is potentially sensitive

• Prejudice and Bias

• Discrimination

• Health Disparities

• Healthcare Disparities

11


Gathering feedback on potentially sensitive content

12

Does potentially sensitive Psychological, Social, and Biological Foundations of Behavior section content interfere with the performance of examinees from underrepresented minority groups by distracting them or slowing them down?

7


Methods of gathering feedback

Focus Group with Medical Students and MCAT Exam

Editors

Review of Responses to

PSBB Study Post-administration

Survey

Interviews with 2013

MCAT Examinees

13


Results• Very few individuals identified the content as

distracting

• Individuals underrepresented in medicine were no more likely to identify the content as distracting than others

• In general, participants noted that topics such as health disparities are relevant to medical school and important for future medical students to know

14

8


Monitoring Fairness Data During Field Testing

15


New test questions were tried out on examinees in 2013 and 2014• Examinees volunteered to take a 32-item “trial

section” at the end of the test

• Volunteers who showed a “good faith effort” received $30 and feedback on the new questions

• Data from the trial section were used to:

• Identify questions with good statistical properties

• Highlight any opportunities for improved item development

16

9


We monitored the field test data for fairness • Attended a minority-serving institution or an

institution with no pre-health advisor

• Received fee assistance from the AAMC

• Males vs. females

• Race/ethnicity (white, Black, Latino)

17


Results• More than 113k students tried out new questions

• Participation rates were high, and the vast majority of examinees showed a good faith effort

• The participants were similar demographically to the population of examinees

• The high data quality helped build a large item bank for the new MCAT exam

• Differences in average performance were comparable to the differences on the old MCAT exam

18

10


After Lunch, You’ll See Early Data from the April and May 2015 Examinations

19


Fairness played a critical role in developing the score scales for the new MCAT exam

20

11


The new MCAT scores

21


Developing the Score Scales with Fairness in Mind

22

12


Many data sources influenced the design of the new score scales• Surveys of admissions officers on the use of

MCAT scores and other applicant data in admissions

• Acceptance rates for applicants with different ranges of MCAT scores and undergraduate GPAs

• Graduation rates for medical students with different ranges of MCAT scores and undergraduate GPAs

23

Medical schools use academic metrics, experiences, and attributes holistically

24

Mean Importance Ratings

Academic Metrics Experiences Demographics2

Highest Importance Ratings (>= 3.0)

GPA: Science/math

MCAT total score

Upward or downward grade trend

GPA: Total

Performance in a post‐baccalaureate program

Selectivity of undergrad.

institution (Priv.)5

Healthcare experience

Community service/volunteer experience

Experience with underserved populations

Navigated through cultural barriers or challenges

Leadership experience

U.S. citizenship/permanent residency (Pub.)5

State residency (Pub.)5

Medium Importance Ratings (>=2.5 and <3.0)

On schedule to meet pre‐medical coursework

GPA: Cumulative non‐science/math

Research experience (Priv.)5

Experience with populations unlike the applicant

Lack of access to optimal educational resources

Special family obligations or other circumstances

Work or athletic scholarship obligations while in school

Rural or urban background

(Pub.)5

First‐generation college student

U.S. citizenship/permanent residency (Priv.)5

Race/ethnicity

Socioeconomic status (SES)

Lowest Importance Ratings (<2.5)

Completion of challenging non‐science courses

Selectivity of undergrad.

institution (Public)5

Experience with prejudice

Research experience (Pub.)5

Rural or urban background (Priv.)5

Multilingual

Legacy

Gender 3

13

GPA Total 5-14 15-17 18-20 21-23 24-26 27-29 30-32 33-35 36-38 39-45 All3.80-4.00 4% 2% 11% 21% 38% 61% 76% 84% 88% 91% 69%

3/80 4/162 57/524 317/1,526 1,363/3,554 4,232/6,978 7,110/9,361 6,298/7,504 3,688/4,176 1,385/1,519 24,457/35,384

3.60-3.79 0% 2% 8% 16% 27% 45% 64% 76% 80% 85% 52%0/177 8/367 83/1,024 371/2,332 1,332/4,866 3,725/8,284 5,997/9,359 4,513/5,973 1,782/2,235 435/514 18,246/35,131

3.40-3.59 0% 2% 5% 12% 22% 32% 48% 62% 70% 75% 36%1/336 13/553 67/1,278 314/2,607 1,010/4,691 2,307/7,151 3,600/7,455 2,382/3,854 819/1,176 176/234 10,689/29,335

3.20-3.39 0% 1% 4% 11% 18% 23% 35% 47% 58% 65% 25%0/370 5/561 41/1,168 249/2,262 604/3,344 1,012/4,369 1,453/4,106 889/1,902 316/547 74/113 4,643/18,742

3.00-3.19 0% 0% 3% 8% 17% 19% 29% 39% 48% 53% 18%0/388 2/553 25/928 123/1,578 373/2,218 455/2,361 530/1,851 313/808 112/233 21/40 1,954/10,958

2.80-2.99 0% 1% 3% 6% 12% 16% 24% 27% 26% 29% 12%0/368 4/386 19/626 54/908 132/1,069 158/998 179/746 85/310 22/86 7/24 660/5,521

2.60-2.79 0% 0% 3% 5% 9% 15% 21% 28% 32% 30% 9%0/274 1/284 12/355 24/486 47/512 57/388 59/276 33/117 15/47 3/10 251/2,749

2.40-2.59 0% 0% 1% 3% 9% 14% 17% 16% 18% -- 6%0/196 0/151 2/179 8/240 19/221 22/152 18/109 6/37 3/17 79/1,304

2.20-2.39 0% 0% 0% 0% 8% 12% 15% 7% -- -- 4%0/132 0/77 0/94 0/91 7/88 8/68 6/39 1/14 22/611

2.00-2.19 0% 0% 0% 0% 0% 14% 18% -- 2%0/53 0/40 0/42 0/28 0/30 2/14 2/11 4/220

1.47-1.99 0% -- 0% 0% -- -- -- 0%0/42 0/10 0/12 0/90

All 0% 1% 5% 12% 24% 39% 57% 71% 79% 85% 44% 4/2,419 37/3,143 306/6,229 1,460/12,070 4,887/20,601 11,978/30,769 18,954/33,316 14,520/20,521 6,757/8,522 2,102/2,459 61,005/140,049

Old MCAT Total

44% of applicants

received one or more

acceptances

Percentages of 2012-2014 applicants admitted into one or more medical schools

4

GPA Total 5-14 15-17 18-20 21-23 24-26 27-29 30-32 33-35 36-38 39-45 All3.80-4.00 4% 2% 11% 21% 38% 61% 76% 84% 88% 91% 69%

3/80 4/162 57/524 317/1,526 1,363/3,554 4,232/6,978 7,110/9,361 6,298/7,504 3,688/4,176 1,385/1,519 24,457/35,384

3.60-3.79 0% 2% 8% 16% 27% 45% 64% 76% 80% 85% 52%0/177 8/367 83/1,024 371/2,332 1,332/4,866 3,725/8,284 5,997/9,359 4,513/5,973 1,782/2,235 435/514 18,246/35,131

3.40-3.59 0% 2% 5% 12% 22% 32% 48% 62% 70% 75% 36%1/336 13/553 67/1,278 314/2,607 1,010/4,691 2,307/7,151 3,600/7,455 2,382/3,854 819/1,176 176/234 10,689/29,335

3.20-3.39 0% 1% 4% 11% 18% 23% 35% 47% 58% 65% 25%0/370 5/561 41/1,168 249/2,262 604/3,344 1,012/4,369 1,453/4,106 889/1,902 316/547 74/113 4,643/18,742

3.00-3.19 0% 0% 3% 8% 17% 19% 29% 39% 48% 53% 18%0/388 2/553 25/928 123/1,578 373/2,218 455/2,361 530/1,851 313/808 112/233 21/40 1,954/10,958

2.80-2.99 0% 1% 3% 6% 12% 16% 24% 27% 26% 29% 12%0/368 4/386 19/626 54/908 132/1,069 158/998 179/746 85/310 22/86 7/24 660/5,521

2.60-2.79 0% 0% 3% 5% 9% 15% 21% 28% 32% 30% 9%0/274 1/284 12/355 24/486 47/512 57/388 59/276 33/117 15/47 3/10 251/2,749

2.40-2.59 0% 0% 1% 3% 9% 14% 17% 16% 18% -- 6%0/196 0/151 2/179 8/240 19/221 22/152 18/109 6/37 3/17 79/1,304

2.20-2.39 0% 0% 0% 0% 8% 12% 15% 7% -- -- 4%0/132 0/77 0/94 0/91 7/88 8/68 6/39 1/14 22/611

2.00-2.19 0% 0% 0% 0% 0% 14% 18% -- 2%0/53 0/40 0/42 0/28 0/30 2/14 2/11 4/220

1.47-1.99 0% -- 0% 0% -- -- -- 0%0/42 0/10 0/12 0/90

All 0% 1% 5% 12% 24% 39% 57% 71% 79% 85% 44%4/2 419 37/3 143 306/6 229 1 460/12 070 4 887/20 601 11 978/30 769 18 954/33 316 14 520/20 521 6 757/8 522 2 102/2 459 61 005/140 049

Old MCAT Total

9% of applicants with GPAs

above 3.8 and MCAT scores

above 38 are not admitted into any medical

schools


4

14

GPA Total 5-14 15-17 18-20 21-23 24-26 27-29 30-32 33-35 36-38 39-45 All3.80-4.00 4% 2% 11% 21% 38% 61% 76% 84% 88% 91% 69%

3/80 4/162 57/524 317/1,526 1,363/3,554 4,232/6,978 7,110/9,361 6,298/7,504 3,688/4,176 1,385/1,519 24,457/35,384

3.60-3.79 0% 2% 8% 16% 27% 45% 64% 76% 80% 85% 52%0/177 8/367 83/1,024 371/2,332 1,332/4,866 3,725/8,284 5,997/9,359 4,513/5,973 1,782/2,235 435/514 18,246/35,131

3.40-3.59 0% 2% 5% 12% 22% 32% 48% 62% 70% 75% 36%1/336 13/553 67/1,278 314/2,607 1,010/4,691 2,307/7,151 3,600/7,455 2,382/3,854 819/1,176 176/234 10,689/29,335

3.20-3.39 0% 1% 4% 11% 18% 23% 35% 47% 58% 65% 25%0/370 5/561 41/1,168 249/2,262 604/3,344 1,012/4,369 1,453/4,106 889/1,902 316/547 74/113 4,643/18,742

3.00-3.19 0% 0% 3% 8% 17% 19% 29% 39% 48% 53% 18%0/388 2/553 25/928 123/1,578 373/2,218 455/2,361 530/1,851 313/808 112/233 21/40 1,954/10,958

2.80-2.99 0% 1% 3% 6% 12% 16% 24% 27% 26% 29% 12%0/368 4/386 19/626 54/908 132/1,069 158/998 179/746 85/310 22/86 7/24 660/5,521

2.60-2.79 0% 0% 3% 5% 9% 15% 21% 28% 32% 30% 9%0/274 1/284 12/355 24/486 47/512 57/388 59/276 33/117 15/47 3/10 251/2,749

2.40-2.59 0% 0% 1% 3% 9% 14% 17% 16% 18% -- 6%0/196 0/151 2/179 8/240 19/221 22/152 18/109 6/37 3/17 79/1,304

2.20-2.39 0% 0% 0% 0% 8% 12% 15% 7% -- -- 4%0/132 0/77 0/94 0/91 7/88 8/68 6/39 1/14 22/611

2.00-2.19 0% 0% 0% 0% 0% 14% 18% -- 2%0/53 0/40 0/42 0/28 0/30 2/14 2/11 4/220

1.47-1.99 0% -- 0% 0% -- -- -- 0%0/42 0/10 0/12 0/90

All 0% 1% 5% 12% 24% 39% 57% 71% 79% 85% 44%4/2 419 37/3 143 306/6 229 1 460/12 070 4 887/20 601 11 978/30 769 18 954/33 316 14 520/20 521 6 757/8 522 2 102/2 459 61 005/140 049

Old MCAT Total

8% of applicants with GPAs 3.00 -

3.19 and MCAT scores 21-23 are admitted into at least one medical

school


4

GPA Total 5-14 15-17 18-20 21-23 24-26 27-29 30-32 33-35 36-38 39-45 All3.80-4.00 -- -- 72% 77% 87% 91% 91% 90% 89% 88% 90%

59/82 244/317 1,048/1,200 3,025/3,333 4,328/4,774 3,157/3,497 1,713/1,934 615/701 14,196/15,850

3.60-3.79 -- 58% 66% 74% 83% 89% 90% 89% 87% 90% 88%7/12 81/122 294/395 1,010/1,220 2,911/3,269 4,086/4,530 2,525/2,833 1,027/1,174 233/258 12,175/13,816

3.40-3.59 -- 40% 69% 72% 80% 87% 89% 90% 88% 89% 87%4/10 75/109 259/362 762/955 1,960/2,252 2,656/2,979 1,537/1,713 562/641 115/129 7,933/9,157

3.20-3.39 -- 55% 65% 65% 75% 85% 90% 89% 88% 84% 84%6/11 60/93 178/272 418/557 905/1,066 1,202/1,336 676/760 247/280 41/49 3,734/4,427

3.00-3.19 -- 43% 38% 62% 71% 84% 87% 90% 90% 81% 80%6/14 20/52 104/169 229/321 411/487 429/495 246/274 97/108 13/16 1,557/1,939

2.80-2.99 -- 38% 59% 62% 74% 85% 82% 88% -- 72%11/29 54/92 91/147 120/162 128/150 77/94 15/17 508/706

2.60-2.79 -- 44% 47% 63% 80% 86% 85% -- -- 70%8/18 18/38 32/51 45/56 42/49 22/26 171/246

2.40-2.59 -- -- 40% 61% 71% 86% -- -- 73%4/10 11/18 15/21 18/21 61/84

2.20-2.39 -- -- -- -- -- -- -- 44%8/18

2.00-2.19 -- -- -- --

1.47-1.99 -- -- -- --

All 50% 47% 62% 70% 81% 88% 90% 90% 88% 88% 87% 10/20 31/66 316/510 1,155/1,660 3,604/4,474 9,394/10,650 12,893/14,339 8,253/9,210 3,667/4,160 1,026/1,163 40,349/46,252

Old MCAT Total

87% of medical students

graduate in 4 years

The vast majority of medical students graduate in four years

44

15

GPA Total 5-14 15-17 18-20 21-23 24-26 27-29 30-32 33-35 36-38 39-45 All3.80-4.00 -- -- 72% 77% 87% 91% 91% 90% 89% 88% 90%

59/82 244/317 1,048/1,200 3,025/3,333 4,328/4,774 3,157/3,497 1,713/1,934 615/701 14,196/15,850

3.60-3.79 -- 58% 66% 74% 83% 89% 90% 89% 87% 90% 88%7/12 81/122 294/395 1,010/1,220 2,911/3,269 4,086/4,530 2,525/2,833 1,027/1,174 233/258 12,175/13,816

3.40-3.59 -- 40% 69% 72% 80% 87% 89% 90% 88% 89% 87%4/10 75/109 259/362 762/955 1,960/2,252 2,656/2,979 1,537/1,713 562/641 115/129 7,933/9,157

3.20-3.39 -- 55% 65% 65% 75% 85% 90% 89% 88% 84% 84%6/11 60/93 178/272 418/557 905/1,066 1,202/1,336 676/760 247/280 41/49 3,734/4,427

3.00-3.19 -- 43% 38% 62% 71% 84% 87% 90% 90% 81% 80%6/14 20/52 104/169 229/321 411/487 429/495 246/274 97/108 13/16 1,557/1,939

2.80-2.99 -- 38% 59% 62% 74% 85% 82% 88% -- 72%11/29 54/92 91/147 120/162 128/150 77/94 15/17 508/706

2.60-2.79 -- 44% 47% 63% 80% 86% 85% -- -- 70%8/18 18/38 32/51 45/56 42/49 22/26 171/246

2.40-2.59 -- -- 40% 61% 71% 86% -- -- 73%4/10 11/18 15/21 18/21 61/84

2.20-2.39 -- -- -- -- -- -- -- 44%8/18

2.00-2.19 -- -- -- --

1.47-1.99 -- -- -- --

All 50% 47% 62% 70% 81% 88% 90% 90% 88% 88% 87% 10/20 31/66 316/510 1,155/1,660 3,604/4,474 9,394/10,650 12,893/14,339 8,253/9,210 3,667/4,160 1,026/1,163 40,349/46,252

Old MCAT Total

81% of medical students

entering with MCAT scores between 24-26 graduate in 4

years

Medical students with a wide range MCAT scores and UGPAs graduated in 4 years

44


A new exam means new score scales

Decision Rules

• No overlap

• Emphasis on the center of the scale

30

16


14


After Lunch, You’ll Have a Chance to Work with the New Score Reports

32

17


Our Research on Fairness is Just Beginning

33


MCAT validity research The MCAT Validity Committee is responsible for evaluating the new exam and how well it:

• Supports diversity and is fair

• Predicts medical students’ academic success

• Helps admissions officers do their work

34

18



Validity committee members• Catherine Lucey, M.D. (Chair)• Aaron Saguil, M.D., LTC

(Vice-chair)• Ngozi Anachebe, M.D.• Ruth Bingham, Ph.D.• Barbara Beckman, Ph.D.• Kevin Busche, M.D.• Victoria Cannon• Steven Case, Ph.D.• Deborah Castellano, M.S.• Jerry Clark, Ph.D., M.B.A.• Julie A. Chanatry, Ph.D.• Daniel M. Clinchot, M.D.

• Liesel Copeland, Ph.D.• Hallen Chung, M.A.• Martha L. Elks, M.D.• William Gilliland, M.D.• Jorge A. Girotti, Ph.D., M.A.• Joshua Hanson, M.D., M.P.H.• Brandon Hunter• Loretta Jackson-Williams, M.D.• David Jones, Ph.D. • R. Stephen Manuel, PhD.• Stephanie C. McClure, M.D.,

FACP• Janet McHugh

2436

19


Validity committee members• Stephen Nicholas, M.D.• Wanda Parsons, M.D., FCFP• Boyd F. Richards, Ph.D.• Stuart Slavin, M.D.• Jenna Smith, Ph.D.• Aubrie Swan Sein, Ph.D., Ed.M.• Doug Taylor• Carol Terregino, M.D.• Ian W. Walker, M.D.• L. James (LJ) Willmore, M.D.• Robert A. Witzburg, M.D.• David Wofsy, M.D.

2437


The MCAT validity research will evaluate a number of questions, for example:

38

Diversity and fairness • Are differences in average scores for test takers in different racial/ethnic groups smaller or larger than differences on the old exam?

Academic preparation • Will more humanities and psychology majors take the new exam?

Admissions decision making

• How are medical school admissions committees using scores from the new exam in admissions?

Predicting medical student performance

• Will scores from the new exam predict academic performance throughout medical school?

20


Two working groups are studying fairness issues

39

Change to test

Changes to preparation

Potential differences in access

Potential increases in group

differences

People• Pre-health advisors• Admissions• Diversity affairs• Other

Data• MCAT registration• PMQ• AMCAS applications• Other

Questions• Access to preparation

resources• Access to courses• Barriers to success• Changes to diversity• Other


We will report validity data as they become available

40

April 2015 July 2015 Aug 2016 Jan. 2017

MCAT2015launches

Applicationsfor the 2016 classsent to medical schools

2016 classmatriculates

Acceptance data reported for 2016 class

Predictive validityresults for M1 students reported at AAMCAnnual Meeting

Nov.2017

Jan. 2018

Acceptance data reported for 2017 class

Predictive validityresults for M1, M2 students reported at AAMCAnnual Meeting

Nov.2018

MCAT session at the PDC, preliminary predictive validity results for PSBB

June2015

25

21


Activity 2.1

41


Fairness played a critical role in developing preparation resources for aspiring physicians

42

22


Helping Students Prepare

Understand

Practice

Study

43



Understand

Practice

Study

44

23



Understand

Practice

Study

45



UnderstandStudy

46

24


Khan Academy tutorials teach concepts targeted by the new test

47

25


What are your colleagues doing?

Jorge GirottiNgozi Anachebe

49


Activity 2.2

50


1

Session 2

List of Resources on exam content


FACTS data on MCAT scores (aamc.org/facts)

“Do Racial and Ethnic Group Differences in Performance on the MCAT Exam Reflect Test Bias” (Academic

Medicine collection of research on the new MCAT exam, 2013)

Khan Academy collection and other free and low cost test preparation resources


2

Activity 2.1

Fairness and the New MCAT Exam

Directions:

Take a few minutes to think about ways you can help your committee consider a wider range of scores in

student selection. Then, share your ideas for ways with the table. Take note of any suggestions that you might

bring back to your institution.

•

•

•


3


4

Activity 2.2

Fairness and the New MCAT Exam: Solutions for Preparing Students

Directions:

Take a few moments to think about ways you and colleagues at your institution have helped prepare pre‐health

students for the new exam. After you’ve gathered your thoughts, share your ideas with your table. Take note of

any suggestions that you might bring back to your institution.


5

1


Session #3

New MCAT Scores and What They Tell You



Resources for new MCAT scores


• Interactive Score Report (fact sheets)

• Videos

• Recorded presentations

2

2



• Know about the data from the April and May,2015 administrations

• Be able to explain the new scores and scorereports

• Develop strategies for setting thresholds toscreen applications

• Develop a plan to familiarize your committeemembers with new scores and score reports

3


Refer to information in this guide during this session

4

3


Results from the Launch of the New MCAT Exam in April and May 2015


April and May examinees• 13,000 examinees tested in May and April

• The percentage of examinees differed slightly from a typical year

• Slightly more • Whites• Females• First-timers

• Slightly fewer • Asians

6

4


How the score scale was developed1. Evaluate data for anomalies

2. Count the number of questions each examinee answered correctly

3. Weight the scores to a typical year

4. Create a table with raw-to-scale score translations that accomplished three goals

7


Goals for the new score scales• Create a symmetric distribution, centering the

mean and median score at the center of the scale

• Spread the score distribution across the entire range of scores

• Improve the precision of scores all along the scale

8

5

The score distributions from the old exam have drifted

0

2

4

6

8

10

12

14

16

18

20

22

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Per

cen

t

Biological Sciences Section Score1 158

Mean = 8.8SD = 2.5

429

The scores for the new MCAT exam are centered, with a wider spread

0

2

4

6

8

10

12

14

16

18

20

22

Per

cen

t

Biological and BiochemicalFoundations of Living Systems

Section Score

Mean = 125.0SD = 3.0

1810

6

The new MCAT scores are more precise.93

.86 .86 .85 .82

0.0

0.2

0.4

0.6

0.8

1.0

Total Score Biological andBiochemical

Foundations ofLiving Systems

Chemical andPhysical

Foundations ofBiologicalSystems

Critical Analysisand Reasoning

Skills

Psychological,Social, andBiological

Foundations ofBehavior

New MCAT exam11


Percentile Ranks Will Provide the Clearest Picture of Examinee

Performance in Early Use of the New MCAT Exam

7


Percentile ranks for the new section scores

Section PercentileScore Rank

132 100131 99130 97129 93128 87127 77126 67

125 54124 44123 32122 21121 14120 7119 3118 1

125 is at the 54th percentile

Biological and Biochemical Foundations of Living Systems

18

125 54

13






132 100131 99130 97129 93128 87

127 77126 67125 54124 44123 32122 21121 14120 7119 3118 1

18

127 77

14

8



122 is at the 21st percentile



132 100131 99130 97129 93128 87127 77126 67125 54124 44123 32

122 21121 14120 7119 3118 1

18

122 21

15

Percentile ranks for the new MCAT total score

500 is at the 51st percentile

17

500 51

16

9



17

507 74

17



17

492 25

18

10


Are Differences in Average MCAT Scores by Demographic Group

Similar on the New and Old Exams?


Are differences in average MCAT scores similar on the new and old exams?

• Average score differences by demographic group from the trial section conducted in 2013 and 2014 were comparable to differences on the old MCAT exam

• Data from April and May, 2015 provide the first look at how examinees scored on the new MCAT exam

20

11

New MCAT total scores, April & May 2015

472 479 486 493 500 507 514 521 528

Asian (mean=500.3; N=2,352)

Hispanic (mean=495.3; N=1,184)

Black (mean=492.5; N=1,070)

White (mean=502.3; N=6,334)

Female (mean=499.1; N=6,590)

Male (mean=501.4; N=5,210)

Total Score21

Old MCAT total scores, Jan. 2012-Sept. 2014

3 6 9 12 15 18 21 24 27 30 33 36 39 42 45

Asian (mean=26.0; N=69,701)

Hispanic (mean=21.9; N=27,980)

Black (mean=20.5; N=25,504)

White (mean=26.1; N=134,431)

Female (mean=24.2; N=148,145)

Male (mean=26.3; N=138,950)

Total Score

43

22

12


Challenges in directly comparing group differences on the new and old MCAT exams• Different scales (3 to 45 vs 472 to 528)

• Distributions have different shapes and variability

• We standardized these differences to make clear comparisons

23

1.00

0.72

0.97

0.73

0.00

0.20

0.40

0.60

0.80

1.00

1.20

White-Black/AA White-Hispanic

Standardized Difference Statistics for the New and Old MCAT Exams

New MCAT Exam

Old MCAT Exam

Standardized difference statistics are similar on the new and old MCAT exams

24

13


The MCAT Validity Committee will study how differences in academic preparation relate to group differences on the MCAT exam

• Course taking and other information about academic and test preparation

• Information about institutional characteristics (e.g., selectivity, under-resourced)

• The committee members’ experiences working with undergraduates, post-baccalaureate programs, and pipeline programs

25


We will also continue to look at characteristics of the exam:• Timing data

• Reading load

• Item development process

26

14


Data about the New Exam Will be Published at the End of Each Testing Year

27


Using the New Scores

15


New reports provide more information

15

29


Percentile ranks• Bring meaning to the new scores

30

16


Confidence bands• Support holistic review

19

31


New scores are more reliableMore test questions means:

• More reliable scores

• Better information aboutapplicants’ academicpreparation

32

17


Score profiles• Support holistic review

20

33


Activity 3.1

34

18

Sample Score Report #1

35


36

126124

19


Developing Thresholds for the New Exam


Using percentiles to develop thresholds

38

Start with old exam

Percentiles used to screen

65%

20


Using percentiles to develop thresholds

New Exam: Look at scores in 65%

Drop down 10

Look at applications w/ scores between 55% and 65%

39



Christina Grabowski

40

21

Percentile ranks: Old MCAT total score

41

24 43

41

24 is at the 43rd percentile

Percentile ranks: New MCAT total score

498 is at the 43rd percentile

17

498 43

495 34

42


22

Percentile ranks: New MCAT total score

Look carefully at applicants in this range to see if their transcripts and other information fit with your program

17

498 43

495 34

43


Resources for Using Scores from the New Exam

23



45


Interactive score report

46

24


Recorded presentations

30-minute presentation on using the new exam and scores

11-minute presentation on the new scores and score reports

47



Quinn Capers

48

25


Activity 3.2

49

Worksheet: New MCAT Scores and What They Tell You

Session 3

List of resources on the new score scales and score reports



Videos

Recorded presentations


Activity 3.1

Familiarize yourself with the new score report

Directions:

First, take a few minutes to review the two sample score reports on the next page. Identify the key elements of

the score report and what information each element provides about the applicant. Make some notes about how

you would explain each element of the score report to your committee. Then, practice explaining the new score

report to your table partner.

Key Element of the Score Report

Key Take Aways

New Scores

Percentile Ranks

Confidence Bands

Score Profiles




55%55%

127124124 126 126


Activity 3.2

Create a Plan to Orient Your Committee

Directions:

First, take a few minutes to think about how you will familiarize your committee members with the new scores

and score reports. Write down how you’ll orient them, what key messages about the score report you’ll

emphasize, and what resources you’ll use. As you develop your plan, you might think about the following

questions:

Is your committee already familiar with the score scale and report for the new MCAT exam?

In the past, what approaches worked (and didn’t work) in sharing new information with your

committee?

Finally, share your plan the table. Take note of any ideas that you might want to incorporate into your plan.

Committee Orientation Plan

When

Key messages

Resources

1


Session #4

Using MCAT Scores in the 2016 Admissions Cycle


Learning objectivesBy the end of this session, you will be able to:

• Triangulate new MCAT scores with otheracademic information

• Review 2016 applications with new scoresalongside applications with old scores

2

2


Refer to information in this guide during this session

3


Context for 2016 Application Processing

3


Examinee trends in 2014

94,000

40,000

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

90,000

100,000

Jan.-Sep. 2014 Oct. 2014-Jan. 2015

Num

ber

of E

xam

inee

s

Testing Year

• Expanded 2014testing calendar

• ~40,000 registeredto test November -January

5


Registrants and examinees for 2015• ~ 26,800 registered for April, May, and June

• ~34,000 registered so far July, August, andSeptember

13,000 have taken the new exam

6

4

Score release schedule

7

Test Date Score Release Date

April 17 June 16April 18 June 16May 22 June 30June 19 July 21June 20 July 21July 17 August 18July 18 August 18August 5 September 8August 6 September 8August 21 September 22August 22 September 22September 3 October 6September 11 October 13September 12 October 13September 23 October 27

July 1 AMCAS Applications Released


2016 application cycle scenarios

8

Evaluate applicants with old

exam scores the way you always

have

Evaluate applicants with scores from both

exams using new scores and percentile

ranks

Evaluate applicants with

new exam scores using percentile

ranks



have



ranks



ranks

Examinee with old exam score

Examinee with both exam scores

Examinee with new exam score

21

5



9



have



ranks



ranks



have



ranks



ranks

9




21



10



have



ranks



ranks



have



ranks



ranks

10




21

6





have



ranks



ranks



have



ranks



ranks

11




21


What are your colleagues thinking?

Bob WitzburgLeila Amiri

12

7


Activity 4.1 Setup

13

Claire S.

14

DegreePsychology

MajorPsychology

InstitutionTop-tier

8


Activity 4.1

Triangulate new MCAT scores with information

about academic preparation

15


Activity 4.2: Setup

Example Pair

16

9


Activity 4.2

Evaluate applicants with new and old MCAT

scores

17


Worksheet: Using MCAT Scores in the 2016 Admissions Cycle

Session 4

List of resources on using MCAT scores in the 2016 admissions cycle



Academic Medicine collection of research on the new MCAT exam

Academic Medicine Last Page comparing the new and old exams

Videos on the new MCAT exam content and scores

June 2015 PDC slide decks and recordings

Pre‐recorded presentations


Activity 4.1

Triangulate New MCAT Scores with Information about Academic Preparation

Directions:

The activity 4.1 packet on your table contains an excerpt of three sample applications. Each excerpted

application contains information about the applicant’s academic preparation for medical school.

Take a few minutes to review each application. Then, triangulate information about each applicant’s academic

preparation in relationship to their new MCAT scores. After you’ve made some notes, share your thinking with

the table.

Applicant Name Notes

Claire S.

(Example)

Andrew H.

Jessie L.

Tomas G.

Example Applicant: Claire S.

EDUCATION

Degree: B.A. (expected, May 2016)

Major: Psychology (Undergraduate)

Institution: Top-tier

METRICS

MCAT Score Report:

Scores from the MCAT Exam taken on 4/17/2015 Section Score Confidence

Band Percentile Rank of Score

Score Profile

Chemical and Physical Foundations of Biological Systems

126 125 127 67%

Critical Analysis and Reasoning Skills

127 126 128 81%


127 126 128 77%

Psychological, Social, and Biological Foundations of Behavior

130 129 131 97%

MCAT Total

510 508 512 83%

Selected Transcript:

Undergraduate GPA, Cumulative: 3.7

Undergraduate GPA, BCPM: 3.5

Course* Grade Course Grade Introduction to Psychology A+ Introduction to Biology A- Brain and Behavior A+ Organic Chemistry A- Health Psychology A Biochemistry B+ Developmental Psychology A- Sociology of Health and Illness A Statistics B Calculus B+

*AP credit for 2 semesters of general chemistry

118 132125

118 132125

118 132125

118 132125

Applicant: Andrew H.

EDUCATION

Degree: B.S. (expected, May 2016)

Major: Chemistry (Undergraduate)

Institution: Lower-tier

METRICS

MCAT Score Report:



Score Profile


125 124 126 55%


125 124 126 58%


126 125 127 67%


125 124 126 55%

MCAT Total

501 499 503 54%




Course Grade Course Grade Introduction to Biology A Cell and Molecular Biology A Human Biology A Introductory Physics A General Chemistry I A Physics II A General Chemistry II A Modern American Literature B+ Organic Chemistry A Introductory Psychology A Biochemistry A Health and Society A

118 132125

118 132125

118 132125

118 132125

Applicant: Jessie L.

EDUCATION

Degree: M.S. (expected, May 2016); B.S. (2012)

Major: Genetics (Graduate)

Sociology (Undergraduate)


METRICS

MCAT Score Report:



Score Profile


128 127 129 87%


128 127 129 87%


129 128 130 93%


127 126 128 75%

MCAT Total

512 510 514 87%

Scores from the MCAT Exam taken on 5/31/12 Section Score Percentile

Rank of Score

Biological Sciences 9 56% Verbal Reasoning 10 84% Physical Sciences 9 67% Writing Sample Q 86% MCAT Total 28 67%

118 132125

118 132125

118 132125

118 132125

Selected Transcript: Graduate GPA, Cumulative: 4.0



Graduate Course Grade Core Concepts: Biomolecular Science A Ethics in Biomedical Science A Macrophange Biology A Genomics A Molecular Immunology A

Undergradaute Course Grade Postbaccularate Course Grade Introduction to Biology A Introduction to Physics A General Chemistry B Organic Chemistry B+ Introduction to Sociology A- Biochemistry A- Sociology of Gender A The History of Madness, Mental Illness, and Psychiatry

A

Thomas G.

EDUCATION


Major: Biochemistry (Undergraduate)


METRICS

MCAT Score Report:



Score Profile


127 126 128 79%


126 125 127 70%


128 127 129 87%


124 123 125 43%

MCAT Total

505 503 507 68%




Course Grade Course Grade Introduction to Biology A Calculus A General Chemistry A Technology in America B+ Organic Chemistry A Robotics A Biochemistry A Statistics A Genetics A Introduction to Psychology B+ Physics A American Literature B

118 132125

118 132125

118 132125

118 132125


Activity 4.2

Evaluate Applicants with New and Old MCAT Scores

Directions:

The activity 4.2 packet on your table contains three pairs of sample applications. Each application contains

information about the applicants’ academic preparation for medical school, experiences, attributes, and letters

of recommendation.

Take a few minutes to review each sample application. Then, select one applicant per pair to invite for an

interview. After you’ve made your decisions, share your thinking with the table.

Pair Which applicant would you

invite for an interview?

(circle)

Rationale

Example Ryan L.

Claire S.

Pair 1 Stephanie C.

Andrew H.

Pair 2 Luis O.

Jessie L.

Pair 3 Tara G.

Thomas G.

Example Pair

Applicant: Ryan L.

EDUCATION




METRICS

MCAT Score Report:


Rank of Score

Biological Sciences 11 88% Verbal Reasoning 10 84% Physical Sciences 10 79% Writing Sample -- -- MCAT Total 31 83%




Course Grade Course Grade Introductory Biology A Physics II B Molecular Biology A Calculus I A General Chemistry I B Calculus II A Organic Chemistry I B American Literature B Biochemistry A Contemporary Philosophy B Physics B+ Anthropology B+

EXPERIENCES

Work: Work study: Intramural sports referee (8 hours per week)

Interned at Tampa Family Health Center

Research: Volunteered in a research lab using protein engineering techniques to engineer regulatory sites involved in cell death for 10 hours per week in the junior year.

Community Volunteered 5 hours every other week at Shriners Hospital for Children for one year

Service: Leadership: President of the campus chapter of the Alpha Phi Omega service fraternity

ATTRIBUTES

Race: White

Ethnicity: Not Spanish/Hispanic/Latino/Latina

Primary Language: English

Disadvantaged: No

Parental Education: Master’s degree, Bachelor degree

Parental Employment: Teacher, Accountant

Institutional Action/Felonies: No

Strength of Letters of Recommendation (1-5): 4

Letters are supportive; one is from Ryan’s college advisor and comments on his dedication and enthusiasm; a letter from his Organic Chemistry professor notes that he offered to volunteer to assist in grading a freshman Chemistry Lab.

Applicant: Claire S.

EDUCATION


Major: Psychology (Undergraduate)


METRICS

MCAT Score Report:



Score Profile


126 125 127 67%


127 126 128 81%


127 126 128 77%


130 129 131 97%

MCAT Total

510 508 512 83%




Course* Grade Course Grade Introduction to Psychology A+ Introduction to Biology A- Brain and Behavior A+ Organic Chemistry A- Health Psychology A Biochemistry B+ Developmental Psychology A- Sociology of Health and Illness A Statistics B Calculus B+

*AP credit for 2 semesters of general chemistry

118 132125

118 132125

118 132125

118 132125

EXPERIENCES

Work: Work Study Program: Library Assistant

Research: Research assistant for a project studying strategies to reduce childhood obesity. Conducted interviews, entered data, and presented a poster at the New England Psychological Association conference.

Community Volunteered two Saturdays a month at a local food bank for four years. Service: Served as an EMT at the Volunteer Fire Department for two years. Leadership: Coordinated the annual alumni giving campaign for two years.

ATTRIBUTES

Race: African American



Disadvantaged: No

Parental Education: Master’s degree, Professional

Parental Employment: Manager, Lawyer



Letters are very supportive and positive; include one from the physician scientist who is the PI of research study on childhood obesity.

Pair 1

Applicant: Stephanie C.

EDUCATION


Major: Physics (Undergraduate)

Institution: Middle-tier

METRICS

MCAT Score Report:


Rank of Score





Course Grade Course Grade Introduction to Biology A- General Relativity and Gravity B+ General Chemistry I B+ Galaxies and Cosmology B+ General Chemistry II B Electricity and Magnetism A- Biochemistry A- Calculus II B+ Introductory Physics I A Social Problems B Introductory Physics II B+ The American Revolution B+

EXPERIENCES

Work: Work study programs throughout undergraduate school: College Athletics; College Library;

Currently employed part-time at Ramunto’s Brick and Brew Pizzeria

Research: No research experiences

Community Volunteers every other weekend teaching swim lessons at the YMCA Service: Volunteered as life guard at Lebanon Head Start throughout school year Leadership: Captain of the intermural soccer team for two years

ATTRIBUTES

Race: African American



Disadvantaged: No

Parental Education: High School; Bachelor Degree

Parental Employment: Clerical; Sales Manager



Letters are supportive of Stephanie; letter from head start director commented on her friendly and altruistic personality; letter from Physics professor highlighted her innate interest in subject material

Applicant: Andrew H.

EDUCATION



Institution: Lower-tier

METRICS

MCAT Score Report:



Score Profile


125 124 126 55%


125 124 126 58%


126 125 127 67%


125 124 126 55%

MCAT Total

501 499 503 54%




Course Grade Course Grade Introduction to Biology A Cell and Molecular Biology A Human Biology A Introductory Physics A General Chemistry I A Physics II A General Chemistry II A Modern American Literature B+ Organic Chemistry A Introductory Psychology A Biochemistry A Health and Society A

118 132125

118 132125

118 132125

118 132125

EXPERIENCES

Work: No work experiences. Received financial aid from the “first in family” scholarship program for first-generation students.

Research: Participated in the Porphyrin Electrochemistry research group for 4 years.

Co-authored 2 posters on investigating the crystal and molecular structure of various proteins presented at the American Chemistry Society.

Community Volunteered at DC Coalition for Social Justice Summer Program for 3 consecutive summers

Service: Spent 8 weeks volunteering for Habitat for Humanity in Appalachia Leadership: Organized campus-wide Stop Hunger Now food drive

ATTRIBUTES

Race: White



Disadvantaged: Yes

Parental Education: High school, High school

Parental Employment: Sales Manager, Not employed



Letters are very supportive; letter from his research advisor describes Andrew as one of the more academically competent students she has taught throughout her career and mentions that he is the first member of his family to graduate college.

Pair 2

Applicant: Luis O.

EDUCATION

Degree: B.S. (2014)

Major: Biology (Undergraduate)


METRICS Scores from the MCAT Exam taken on 5/31/13 Section Score Percentile

Rank of Score





Course Grade Course Grade Introduction to Biology A+ Introduction to Psychology A General Chemistry A Neuroscience B Organic Chemistry B Biomedical Ethics A+ Biochemistry A Introduction to Chicano Studies A Physics B Biostatistics B- Human Genetics B Calculus A

EXPERIENCES

Work: Worked 25+ hours/week doing IT to help pay for college.

Research: None.

Community After graduating college, spent a year in the California SafeCorps, an Americorps Service: service opportunity that provides disaster preparedness training and disaster relief. Leadership: Organized a campus Relay for Life event to raise awareness and money for the

American Cancer Society

ATTRIBUTES

Race: Hispanic

Ethnicity: Spanish/Hispanic/Latino/Latina

Primary Language: Spanish

Disadvantaged: No

Parental Education: College, Professional

Parental Employment: Manager, Dentist



Letters are supportive; one is from his SafeCorps supervisor and describes his commitment to helping the under-served.

Applicant: Jessie L.

EDUCATION

Degree: M.S. (expected, May 2016); B.S. (2012)

Major: Genetics (Graduate)

Sociology (Undergraduate)


METRICS

MCAT Score Report:



Score Profile


128 127 129 87%


128 127 129 87%


129 128 130 93%


127 126 128 75%

MCAT Total

512 510 514 87%


Rank of Score

Biological Sciences 9 56% Verbal Reasoning 10 84% Physical Sciences 9 67% Writing Sample Q 86% MCAT Total 28 67%

Selected Transcript: Graduate GPA, Cumulative: 4.0



118 132125

118 132125

118 132125

118 132125

Graduate Course Grade Core Concepts: Biomolecular Science A Ethics in Biomedical Science A Macrophange Biology A Genomics A Molecular Immunology A

Undergradaute Course Grade Postbaccularate Course Grade Introduction to Biology A Introduction to Physics A General Chemistry B Organic Chemistry B+ Introduction to Sociology A- Biochemistry A- Sociology of Gender A The History of Madness, Mental Illness, and Psychiatry

A

EXPERIENCES

Work: Currently employed part-time as a Teaching Fellow for Introductory Biology. After college, worked for two years as a survey research analyst implementing internet-based research surveys for a social science research institute.

Research: Active member of a research lab investigating genetic markers of Alzheimer’s disease; first-author on multiple posters presented at national conferences.

Community Volunteered twice a week at an assisted living facility for patients for 6 months. Service: Participated in the Big Brother/Big Sister program for the past two years.

Leadership: Supervised two undergraduate research assistants in the Alzheimer’s lab

ATTRIBUTES

Race: Southeast Asian



Disadvantaged: No

Parental Education: College, College

Parental Employment: Elementary Teacher, Manager



Letters are supportive; one is from Jessie’s current graduate advisor and comments on the academic growth demonstrated since enrolling in the Master’s program.

Pair 3

Applicant: Tara G.

EDUCATION

Degree: B.S. (2014)

Major: Biology (Undergraduate)


METRICS Scores from the MCAT Exam taken on 5/31/13 Section Score Percentile

Rank of Score





Course Grade Course Grade Introduction to Biology A Introduction to Psychology A Organic Chemistry B Neuroscience B Biochemistry A Biomedical Ethics A Human Genetics B Public Health B Calculus A Biostatistics B

EXPERIENCES

Work: No work experiences

Research: Completed Independent Study with college advisor;

Participated in Research Experience in Biology which resulted in a peer-reviewed publication

Community Volunteered once a week at the Family Health Center (4 hours per week) Service: Leadership: Vice President of the Pre-Med Club

ATTRIBUTES

Race: Asian American

Ethnicity: Japanese


Disadvantaged: No

Parental Education: MD, Bachelor degree

Parental Employment: Physician, Business Owner



Letters are strongly supportive; letter from advisor highlights Tara’s exemplary performance in courses as well as dedication to research; letter from community service manager emphasizes Tara’s motivation and dependability while working as a volunteer

Applicant: Thomas G.

EDUCATION


Major: Biochemistry (Undergraduate)


METRICS

MCAT Score Report:



Score Profile


127 126 128 79%


126 125 127 70%


128 127 129 87%


124 123 125 43%

MCAT Total

505 503 507 68%




Course Grade Course Grade Introduction to Biology A Calculus A General Chemistry A Technology in America B+ Organic Chemistry A Robotics A Biochemistry A Statistics A Genetics A Introduction to Psychology B+ Physics A American Literature B

118 132125

118 132125

118 132125

118 132125

EXPERIENCES

Work: No work experiences

Research: Participated in Undergraduate Research Opportunities Program. Worked in a lab studying the tumor suppressor gene, p53, since freshmen year.

Community Dana-Farber Cancer Institute Book Cart Volunteer for 3 hours per week over the Service: summer for the last two years. Leadership: Eagle Scout

ATTRIBUTES

Race: Hispanic

Ethnicity: Spanish/Hispanic/Latino/Latina

Primary Language: Spanish

Disadvantaged: Yes

Parental Education: High School, High School

Parental Employment: Clerical, Customer Service



Letters are very supportive of his academic and research achievements; one letter commented challenges overcome as a child of immigrants and noted that he has a scholarship.

10


Accessing MCAT Score Reports

Accessing MCAT score reports: AMCAS AP• New and old scores available in the AP on the Summary and

MCAT/GPA tabs

• Graphic score report image available for new scores

• Can include percentile rank in filters (total and section scores)

• Only data appears in the VPA, graphic score report image must be viewed within AP

20

11


Accessing MCAT score reports: AMCAS Data Exchange Service • If you use a homegrown or vendor system, you must use the

Data Exchange Service to receive new MCAT Score Report images

• Data Exchange Service is a new API web service that allows data to be transferred quickly and securely from one location to another

• Technical requirements and specifications are available on the Admissions Hub (www.aamc.org/admissions)

• There is no charge to use the AMCAS Data Exchange Service

Please take a handout about the AMCAS Data Exchange Service to share with your IT support team or vendor!

21


Accessing MCAT score reports: AMCAS Data Exchange Service • Score data (without the image) are included in the

TAMF/DADF

• API users may opt to continue receiving data via the DADF, TAMF and/or ADI and simply use the AMCAS Data Exchange Service to download the new MCAT score report images this year

• As a backup, you can also log in to the AP to view score report images

• Eventually, AMCAS will only transmit data for use in other systems through the Data Exchange Service (2019 application cycle)

Please take a handout about the AMCAS Data Exchange Service to share with your IT support team or vendor!

22

12


MCAT Wrap Up



Resources for Using New MCAT Scores in Admissions


• Interactive Score Report (Facts sheets)

• Academic Medicine collection of research on the new MCAT exam

• Academic Medicine Last Page comparing the new and old exams

24

13



Resources for Using New MCAT Scores in Admissions

• June 2015 PDC slide decks and recordings

• Pre-recorded presentations

• Videos on the new MCAT exam content and score scales

25


Closing discussion and next steps• What else is on your minds?

• Please leave your questions and comments on your tables.

• Office Hours:

Contact Information

[email protected]

www.aamc.org/admissions

26

1

Notes

2

Notes

3

Notes

4

Notes

5

Notes

6

Notes

2015 admissions and student diversity affairs professional development conference€¦ · ·...

Documents