AAOS INSTRUCTIONAL COURSE LECTURE NEW ORLEANS, LOUISIANA ACGME REQUIREMENTS ICL 311 RESIDENCY ACCREDITATION MARCH 12, 2010 8:00A.M. 10:00A.M.

TOPICS (1) DIVERSITY, WOMEN, AND UNDER-REPRESENTED MINORITIES A COMPETENCY-BASED ORTHOPAEDIC

(2) DEVELOPING CURRICULUM

RICHARD E. GRANT, M.D. PROFESSOR OF ORTHOPAEDIC SURGERY UNIVERSITY HOSPITALS OF CLEVELAND/CASE MEDICAL CENTER CLEVELAND, OHIO 44106

RACE/ ETHNICITY AND ORTHOPAEDICS Over the past 20 years, the percentage of the U.S. population consisting of non-White minorities has continued to expand. Approximately 300 million people live in the United States: in 2000, 3.5 million U.S. residents were Hispanic, 36 million were African-American, and 14 million were Asian-American. By 2050, the aggregate of non-White minorities is expected to become the majority. Under-representation of minority health care providers has a negative effect on public health, as reflected in racial health disparities linked to chronic illness, excessive comorbidities, and early demise. Yet today, 80 percent of the 16,000 members of the profession of orthopaedic surgery are White males. Despite 50 years of effort and multiple programs to increase the representation of minorities in the health care professions, minority representation remains grossly deficient. There is a clear need for minority orthopaedic surgeons who can communicate with and comprehend a population of diverse patients to provide them with culturally competent care. At a time of rapidly increasing racial and ethnic diversity, all physicians must practice cultural competency. Several authors have explored the extent of existing racial/ethnic discrepancies in the profession of orthopaedic surgery. Jimenez noted that although one-third of the United States population in 1999 was comprised of Latinos, African-Americans, and Native Americans, only 7 percent of all orthopaedic surgeons were members of these groups. The 2004 American Academy of Orthopaedic Surgeons (AAOS) census revealed that 89 percent of certified practicing orthopaedic surgeons identified as Caucasian; among AAOS candidate members, 80 percent identified as Caucasian. Asian-Americans constituted 3.8 percent of practicing orthopaedic surgeons and 8 percent of AAOS candidate members. African-American respondents constituted 1.3 percent of practicing orthopaedic surgeons and 2.9 percent of AAOS candidate members. Other studies have focused on the racial/ethnic composition of residency programs that are preparing a new generation of orthopaedic surgeons. England and Pierce reported on the state of racial and ethnic diversity in orthopaedics in 1999, after examining the selection patterns of orthopaedic residents during the 12 previoustwelve years prior to publication of their data. The per. The percentage of diversity of orthopaedic residents during that periodose twelve years cchanged minimally, as represented by the acceptance of African-Americans, Hispanics, Native Americans, Puerto Ricans, and Mexican-Americans into orthopaedic residency education programs. The percentage of Asian and Pacific Islanders quadrupled (2.2 percent in 1983, to 9.8 percent in 1995) during the twelve years of the study. The percentage of White women residents remained unchanged. White male participation in orthopaedic surgery declined in direct relation to the increase in Asian or Pacific Islander men. A similar survey of 159 orthopaedic residency education programs garnered a fifty-six (56) percent response rate. The distribution of orthopaedic residents and fellows was as follows: White nNon-Hispanics 84.2 percent; Asians 6.6 percent; African-Americans 3.6 percent; Native Americans 2.2 percent; Puerto Ricans (1.2 percent); Mexican-Americans 0.8 percent;; and, other

1

Hispanics 1.0 percent. African-Americans and Hispanics were under-represented in orthopaedic training programs compared with their numbers in the general American population. Jimenez, R.L., noted that although one-third of the United States population, in 1999, was comprised of Latinos, African-Americans, and Native Americans, only seven (7) percent of all orthopaedic surgeons were represented by these minorities. Ostensibly, there is a need for minority orthopaedic surgeons who can communicate with and comprehend a population of diverse patients to provide them with culturally-competent care. Intrinsic and extrinsic barriers tend to impede women and under-represented minority medical students from choosing orthopaedics as a career. When the 2004 AAOS Orthopaedic Physician census results were published, among the certified practicing orthopaedic surgeons, eighty-nine (89) percent reported they were Caucasian. Among AAOS candidate members, eighty (80) percent identified as Caucasian. Asian-Americans constituted 3.8 percent of practicing orthopaedic surgeons, and eight (8) percent of AAOS candidate members. African-American respondents constituted only 1.3 percent of practicing orthopaedic surgeons, and 2.9 percent of AAOS candidate members. In 2004, the overall density of orthopaedic surgeons in the United States increased to 6.2 per 100,000 population base. Over the past twenty years, the percentage of the American population consisting of non-White minorities continues to expand at a steady rate. By 2050, the aggregate of non-White minorities are expected to become the majority. Despite fifty years of effort and multiple programs to increase the representation of minorities in the health care professions, minority representation remains grossly deficient. Under-representation of minority healthcare providers has a negative effect on public health, inclusive of critical bench marks of racial and health disparities linked to chronic illness, excessive comorbidities, and early demise. Such systemic deficits can be reduced or curtailed by increased efforts for the recruitment and development of both under-represented minority medical students, medical administrators, and full time academic faculty. Many authors have explored the reasons for the under-representation of minorities in the medical professions. Intrinsic and extrinsic barriers tend to impede women and under-represented minority medical students from choosing orthopaedics as a career. Nivet, Taylor, Butts, and Kondwani, et al., note that the recruitment of under-represented minority faculty is compromised by barriers resulting from decades of systemic segregation, discrimination, tradition, culture, and elitism in medicine and academic medical centers. The elimination of barriers to minority faculty would improve public health, expand the focus of contemporary research agendas, and enhance the education and mentoring of under-represented minority students.

Some authors have explored under-representation from a specialty-specific perspective. Ayers, C.E., describes orthopaedic surgery as a specialty attracting an abundance of applicants for limited residency slots. Performance measures reviewed by most orthopaedic residency programs include USMLE scores, election to the Alpha Omega Alpha Honor Society, grade point averages, medical school class rank, personal statements, the Dean's letter, and other letters of recommendation.

2

One bThe dynamics of barriers to minority candidates seeking entry into orthopaedic residencies isnclude the perceptionsense that the under-represented minority applicant pool is inadequate. Additional barriersfactors include certain methods related to applicant- screening criteria, the under-represented minority specialty preference, and the perception under-represented minorities have of orthopaedic surgeons. Performance measures reviewed by most orthopaedic residency programs include U.S. Medical Licensing Examination scores, election to the Alpha Omega Alpha honor society, grade point averages, medical school class rank, personal statements, and letters of recommendation from deans and others. The systemic deficits can be reduced by increased efforts for the recruitment and development of under-represented minority medical students, medical administrators, and full-time academic faculty. Seeking to address this issue, the Accreditation Council of Graduate Medical Education (ACGME) has adopted program requirements addressing cultural competency by adopting two of the six core competencies specifically directed toward the development of cultural competency with a direct link to medical communication skills and professionalism. Other Gebhardt, M.C., notes that majority medical students exposed to classmates from diverse cultures and contrasting life experiences different from theirs enable White male students in an orthopaedic residency to more effectively interact with female and minority patients as a result of interacting with peers who are women or under-represented minorities. The overall quality of care and the level of satisfaction for physicians and their patients are improved by such interaction. Ssuggested remedies for expanding culturally- competent care include increasing the minority applicant pool from kindergarten through the twelfth grade, and subsequent recruitingment o f talented minority students to enter the medical field. Given the extent of current pipeline deficits, some form of short- term affirmative action is necessary to increase the level of diversity of orthopaedic residency programs and orthopaedic faculty. The elimination of barriers to minority faculty would improve public health, expand the focus of contemporary research agendas, and enhance the education and mentoring of underrepresented minority students. The benefits, moreover, will extend to non-minority students as well. Gebhardt notes that White male medical students exposed to classmates from diverse cultures and contrasting life experiences are able to more effectively interact with female and minority patients as a result of their training experiences. The overall quality of care and the level of satisfaction for physicians and their patients was improved by such interaction.

Approximately three hundred million (300,000,000) people live in the United States. By 2000, 3.5 million American residents were Hispanic, thirty-six (36) million were African-American, and fourteen (14) million were Asian-American. By 2050, minority populations will outnumber the White population. Orthopaedic surgeons in the United States will treat an increasingly diverse population from a variety of ethnic groups and cultural backgrounds. Jimenez, R.L., explains the target audience for culturally-competent care education is not just the eighty (80) percent of orthopaedic surgeons who are White males (N=16,000). All physicians must practice cultural competency. The Accreditation Council of Graduate Medical Education (ACGME) has adopted program requirements addressing cultural competency by adopting two of the six core

3

competencies specifically directed toward the development of cultural competency with a direct link to medical communication skills and professionalism.

4

RESIDENT SELECTION CRITERIA: CHALLENGES FOR WOMEN AND UNDER-REPRESENTED MINORITIES

Thordarson and Patzatkis reported in 2007 that a system for selecting found the selection of best applicants for orthopaedic residency programs remains elusive. Only fair- to- poor correlations were noted between athe resident's initial ranking, ranking on graduation from residency, and his or her their U.S. Medical Licensing Examination (USMLE), LE, American Board of Orthopaedic Surgery (ABOS), and Orthopaedics-In- Training Examination (OITE) scores. The only relatively dependable correlation found was between the Orthopaedic In Training Examination (OITE) and the American Board of Orthopaedic Surgery (ABOS) scores. Faculty did not agree in their ranking of residents on graduation. Faculty and applicants, moreover, do not agree as to which criteria are most important for acceptance in to a residency program. Selection criteria for acceptance into orthopaedic residency can vary when comparing applicant opinions to those of faculty at academic medical centers. According to Bajajz and Carmichael's study, faculty favored performance on a local rotation (externship), class rank, and interview performance as essential determinants. Residency applicants thought externship performance, USMLE Step I scores, and letters of recommendation were the three most important determinants of obtaining a residency. Evarts, C.M., reflecting on the consensus of the American Orthopaedic Association (AOA) committee charged with studying the processes involved in the selection of orthopaedic residents, offered the following suggestions to orthopaedic program directors: 1) Use of standardized application forms; 2) fFull disclosure to applicants; 3) cCareful screening of candidates to be interviewed; 4) cCareful planning and implementation of the interview process and interview visit; 5) bBroad faculty representation and discussion of candidates at the time of selection; and, 6) Ddue diligence, when necessary, to resolve incomplete information or conflicting candidate data. Mallott, D., et al, emphasized the need to focus carefully on the applicant's behavior, character, and developing professionalism observed over the course of a four- year medical school curriculum. Such non-academic factors may play a significant role in determining compatibility between a medical student applicant and an existent orthopaedic residency program. Specific reference was made to non-academic factors, such as the candidate's dDean's letter, interview, and other affective domain issues. Luri, Lambert, and Grady-Weliky examined the relationship between the Ddean's letter rankings and later evaluations by residency program directors. Dean's letter rankings were found to serve as a significant predictor of performance in internship. Medical students identified by the dDean's letter as performing in the bottom half of their medical school class were most likely to either

5

under-perform or over-perform during their first year of pPost Ggraduate training.Year one (PGYI). Turner, Shaughnessy, Berg, Larson, and Hanssen developed a Quantitative Composite Scoring Tool (QCST) to be applied in a standardized manner for orthopaedic residency screening and selection, in addition to predicting orthopaedic residency performance. Four predictors were identified, including USMLE Part I scores, AOA status, junior year clinical clerkship honors grades, and their QCST score. Outcome measurements consisted of OITE scores, ABOS Cognitive Examination Part I scores, ABOS Oral Examination Part II scores, and internal assessments of performance meriting attainment of satisfactory PGY-V (Ppost Ggraduate Yyear 5five) cChief Rresident associate status. Honors grades during the junior year clinical clerkships were associated strongly with Cchief Rresident performance. Dirschl, Dahners, Adams, Crouch, and Wilson drew similar conclusions from their analysis of resident selection criteria and subsequent residency performance. Academic performance and clinical clerkships in medical school were identified as an important factors. The number of honors grades on medical school clinical rotations was their strongest predictor of residency performance; . AOA status was second.

In anothern additional study examining the reliability of a scoring system for orthopaedic residency, Dirschl concluded that no bench marks existed by which to define an acceptable intraclass correlation coefficient of a scoring model for resident applicants. Great intra-observer variability existed when subjective elements were included in residency screening scoring systems. Orthopaedic residency programs must first determine what data elements are essential and highly valued in their selection of residents. While systems based on calculating objective academic scores increase the objectivity of the residency selection process, such scoring systems do not appear to correlate with outcomes of residency education programs. An additional study by Carmichael, Westmoreland, Thomas, and Patterson evaluateding the relation of residency selection factors to subsequent OITE Orthopaedic In Training Examination performances, and they concluded those residents who previously scored 220 on the USMLE Step I had higher OITE scores than those scoring below 220. Additionally, it was noted that rResidents who were married also had higher average OITE scores than unmarried residents. A trend with regard to AOA status was also detected. Residents who attained AOA status scored slightly higher on the OITE. Obviously, few pre-residency variables correlate well with success during an orthopaedic residency. Even so, orthopaedic surgery remains one of the most competitive specialties, with more than a ninety-nine (99) percent match fill rate in the past several years. An over-supply of qualified applicants leads to intense competition for these limited residency spots. Thordarson's and Patzatkis study found a poor correlation between USMLE Part I scores and an applicant's position on their residency's programs initial rank list. Smilen, Funai, and Bianco questioned whether interviewers should be given applicants' USMLE Part I scores, maintaining.

6

that Kknowledge of those scores by interviewers of USMLE Part I scores may negate the interview as an independent means of evaluating potential rresidency candidates. Edmond, Deschenes, Eckler, and Wenzell found that ifif USMLE scores are used to screen applicants for residency interviews, a greater percentage of African-American applicants would be refused an interview. Gilbert, et al., observed increased difficulty with comparing candidates if the interview process changeds with each candidate. Structured interviews were preferred. Structured interviews employ standardized questions for all applicants, provide sample answers for comparison, and utilizes a panel for the interviews. The reliability and validity of the interviewing process improves as structure is added. In general, residency programs agree in principle about the important characteristics of a good resident. However, each program's perception of how itthey values certain resident characteristics, and how those characteristics might manifest in their residents eventual practice profile, may differ considerably. Ultimately, tThe identification of accurate selection criteria for residents is becoming increasingly important due to the economic pressures of current and future health care funding and diverse societal forces. Clark, R., et al,. noted that deficiencies in the affective domain (character and personality traits) were the most common reasons leading to discipline or dismissal of residents and may serve as the primary indicator of the resident's ability to function professionally as an orthopaedic surgeon. Dale, Schmitt, and Crosby focused on the misrepresentation of research criteria by orthopaedic residency applicants. Berstein, Jazrawi, Della Valle, and Zuckerman's analysis of residency selection criteria highlighted the relationship between the candidate's affective domain and his/hertheir performance during residency and practice. The authors anticipated the increasing importance of issues relevant to affective domain characteristics. They concluded that affective domain characteristics shwoould play a more important role in future residency selection committees' deliberations. White, A.A., questioned the selection process for orthopaedic residents, indicating that programs must put the horse (that is,, namely the consideration of certain societal goals and responsibilities), before the cart. (that is, The cart was identified as the selection criteria and processes for selecting residents into orthopaedic educational programs).

White concluded that one of the specific outcome goals that should be identified are efforts that can be directed to learning to recognize and evaluate characteristics of applicants that predict desired outcomes in competency. Traditional screening and selection of applicantsmethods, based largely on grades, test scores, and election to Alpha Omega Alpha Hhonorary sSociety, have certain historically- based biases and limitations. The historic ethnocentric impacts on wWestern medical culture are profound, longstanding, and thoroughly interwoven into the fabric of the our profession, oorthopaedics profession. White further concluded it is necessary to substantially change our residency selection process if we hope to achieve some highly- significant humanitarian and pragmatic societal goals.

7

8

ACCREDITATION COUNCIL OF GRADUATE MEDICAL EDUCATION (ACGME) CORE COMPETENCIES AND THE DEVELOPMENT OF A COMPETENCY-BASED ORTHOPAEDIC CORE CURRICULUM Since the beginning of July of 2002, Accreditation Council of Graduate Medical Education (ACGME)-accredited programs in the United States have been challenged to teach and assess competency-based curriculaums and better prepare residentsphysicians in residency programs to to practice in the rapidly- changing health care environment. Current accreditation requirements for residency education programs mandate that residents are to participate in educational experiences that ensure attainment in six general competencies: patient care; medical knowledge; practicebased learning and improvement; professionalism; interpersonal skills and communication; and, systems-based practice. Competency in these six areas must be documented by dependable and appropriate metrics or evaluation tools confirming an incremental growth in professional knowledge, patient-centered care, and orthopaedic surgical skill acquisition. Arnold, L., favors quantitative and qualitative approaches to the assessment of the desired level of professionalism implicit in the framing of the six competencies. Techniques of assessment increasing the validity and reliability of measuring a resident's progress include 360 assessments, performance-based assessments, learning portfolios, and system designs inclusive of infrastructure support. Resolution of a resident's unprofessional behavior would be addressed through due process documented by a warning, and constructive confrontation to institute a structured program of remediation. There should be concordance between the intervention and the etiology of the resident's lapse in professional behavior. The essentials of effective behavioral contracts are includsive of cognitive behavioral therapy, motivational interviewing, and continuous monitoring. According to Cruess emphasizes that , one of the six competencies, professionalism, must be explicitly taught. Once the concept of professionalism is defined and learned by residents and faculty alike, residents will require reinforcement through example. Residents must understand what will be taught, expected, evaluated, and incorporated into the knowledge base of residents and practicing faculty members. Cruess advocates specific learning experiences, includings inclusive of self-reflection on professionalism and the adaptation of such practices into the continuum of medical education. Cornwall concluded it is unlikely that professionalism is an innate or universal characteristic of college students entering medical school or of medical students entering a program of postgraduate residency education. Regardless, the core competency, professionalism, is becomes an essential value in effective medical practice. Beach, Bar-On, Baldwin, Kittredge, Trimm, and Henry evaluated the use of an interactive, on-line resource for competency-based curriculum development. Program directors were allowed to download competency-based curriculum building tools using a specific web site. Most users

9

favored pre-designed formats over self-selected lists of goals and objectives. Less frequently, web site subscribers downloaded resident evaluation forms and tutorials. Respondents to the association's educational guidelines for residency training confirmed that the on-line, competencybased curriculum building tools were easy to use, adaptable, and important to the understanding of mechanisms for integrating the ACGME competencies into their residency programs. Lurie, Mooney, and Lyness evaluated published evidence that the ACGME's six general competencies could each be measured in a valid and reliable way. Fifty-six (56) of 127 articles identified met their inclusion criteria. Studies of systems-based practice and practice-based learning and improvement demonstrated that viewed these competencies wereas inherent properties of systems and that they were not specific to individual resident behavior. Theisr review of the literature cited a lack of evidence that current core competency measurement tools were able to asses the six competencies independently of one another.

Varkey, Karlapudi, Rose, Nelson, and Warner conducted a residency program director selfassessment survey following a web-based, institution-wide curriculum designed to facilitate the teaching and assessment of practice-based learning and improvement (PBLI) and systems-based practice (SBP) in 115 ACGME-accredited residency and fellowship programs. Additional initiatives included didactic sessions for residents and fellows, program director workshops, and one-on-one consultations with program directors. While tThe authors documented a thirteen (13) percent increase in a program directors' perceived ability to measure competency in one area, systems-based practice (SBP), but found there was no change in the program directors' perceived confidence in the measurement of program-based learning and improvement (PBLI). Carraccio and Englander reviewed the current literature for all articles relevant to assessment of the six competencies and resident performance. Their evaluation of current "best practices" endorsed the use of a web-based evaluation portfolio, allowing faculty and program directors a variety of assessment tools to evaluate the diverse domains of competence and reflective learning. Their web-based portfolio assessment program facilitated the evaluation of resident competence and enhanced the faculty research infrastructure supporting theirir practice of evidence-based education. Yaszay, Kubiak, Agel, and Hanel conducted a national survey of orthopaedic program directors and selected orthopaedic residents in an effort to define the experiences of orthopaedic residencies working toward the incorporation of the ACGME's core competencies. Residents and program directors prioritized patient care and medical knowledge. PBLI and SBP's were relegated to the lowest level of priority among the six competencies. Orthopaedic program directors and residents suggested: (a) greater clarification of the definition of each of the core competencies; and, (b) greater commitment to the processes relevant to the development of competency in surgical procedures.

10

Lee, et al,. favored best practices for residency journal club activities and group discussion of existing medical literature. The core competency of PBLIpractice-based learning and improvement was enhanced by the use of a structured review checklist, explicit written learning objectives, and a formalized meeting structure and process. While Rosenfeld advocated the use of a redesigned residency the residency program's morbidity and mortality (M&M) conference to teach and assess the ACGME general competencies for instruction in PBLI, professionalism, interpersonal and communication skills, and SBPsystems-based practice. During the redesigned M&M conference, residents present the selected patient's history and discusses the complications according to traditional morbidity and mortality conference models. However, the resident is also required to analyze the case presentation in terms of health care system issues contributing to the patient's morbidity or mortality. In addition, the resident is tasked with identifying patient safety issues, and communication problems with the patient, the patient's family, andor with colleagues. Issues relevant to ethnic dissonance, ethical issues, and challenges of ccultural competency are also identified and explored by the resident. Subsequently, as an example relative to this process, the case presented is reviewed by faculty orthopaedic surgeons. Each ACGME competency is addressed as deemed relevant to the facts and outcomes of the case presentation. Each resident presenter completes a practice-based improvement log, analyzing factors precipitating the complication and/or the mortality. The resident is then tasked with providing suggestions for systems improvement, increasing patient safety, improving communications and ethnic culturalcompetency, or addressing ethical issues. References from the literature supporting solutions for improved outcomes areshould be included in the resident's practice-based learning and improvement log. Stiles et al. have suggested aAnother method to teach the core competencies to orthopaedic residents and faculty, and to determine whether both parties are satisfying the intended outcomes related to the core competencies has been suggested by Stiles, et al. Their general surgery program initiated a daily morning report as an effective "sign-out" and accounting of new admissions and consults from the previous day to their surgery service.

The morning general surgery report was restructured and presented in an evidence-based format addressing patient care, medical knowledge, professionalism, interpersonal skills and communications, and PBLIpractice-based learning and improvement. Their guidelines included the participation of the on-call Aattending, a review of all the pertinent imaging studies, provisions for follow-up of selected cases, and a critical review of the peer-reviewed literature. Surgical morning report and traditional orthopaedic fracture conferences appear to be very compatible with existent resident didactic sessions, and the incorporation of ACGME competency requirements.

11

RACE ETHNICITY AND ORTHOPAEDICS REFERENCES

Ayers, C.E., "Minorities and the Orthopaedic Profession," Clinical Orthopaedics and Related Research. 1999 May;((362)):58-64. Bollinger, L.C., "The Need for Diversity in Higher Education," Academic Medicine. 2003; (78):431-436. England, S.P., and Pierce, R.O. Jr., "Current Diversity in Orthopaedics: Issues of Race, Ethnicity, and Gender," Clinical Orthopaedics and Related Research. 1999 May;(362):40-3 Gebhardt, M.C., "Improving Diversity in Orthopaedic Residency Programs," Journal of American Academy of Orthopaedic Surgery. Vol 15, sup 1, September 2007, S49-S50, 2007, The American Academy of Orthopaedic Surgeons. Grant, R.E., Banks, W.J. Jr., Alleyne, K.R., "A Survey of the Ethnic and Racial Distribution in Orthopaedic Residency Programs in the United States," Journal of the National Medical Association. 1999 September;91(9): 509-12. Harris, D.L, Mullan S., Simpson, C.E. Jr., Harmon R.G., "The Current and Future Need for Minority Medical Faculty," Journal Association of Academic Minority Physicians. 1991;2(1):14-7. Ibrahim, S.A., "Racial and Ethnic Disparities in Hip and Knee Joint Replacement: A Review of Research in the Veteran's Affairs Healthcare System," Journal of American Academy of Orthopaedic Surgery. 2007;15,sup1: S87-94. Jimenez, R.L., "Barriers to Minorities in the Orthopaedic Profession," Clinical Orthopaedics and Related Research. 1999 May;(362):44-50. Keppel, K., Garcia, T., Hallquist, S., Ryskulova A., Agress, L., "Comparing Racial and Ethnic Populations Based on Healthy People 2010 Objectives," Healthy People STAT Notes. 2007 August; (26):1-16. Nelson, C.L., "Disparities in Orthopaedic Surgical Intervention," Journal of the American Academy of Orthopaedic Surgery. 2007;Vol 15, sup1:S13-7. Nivet, M.A., Taylor, V.S., Butts, G.C., Smith, Q.T., Rust, G., and Kondwani, K., "Diversity in Orthopaedic Medicine, Number One Case for Minority Faculty Development Today," Mount Sinai Journal of Medicine. 2008, December 1; 75(6):491-8. Petersdorf, R.G., Turner, K.S., Nickens, H.W., and Ready, T., "Minorities in Medicine, Past,

12

Present, and Future," Academic Medicine. 1990 November; 65(11):663-70. Thomas, C.L., "African-Americans and Women in Orthopaedic Residency: Experience," Clinical Orthopaedics and Related Research. 1999;362:65-71. The Johns Hopkins

Torucznik, M.A., "2004 Orthopaedic Physicians Census Results Released," American Academy of Orthopaedic Surgery Bulletin. 2005 February; pp. 42-44. White, A.A., "Resident Selection: Are We Putting the Cart Before the Horse?" Clinical Orthopaedics and Related Research. 2002;399:255-259.

13

RESIDENT SELECTION CRITERIA: CHALLENGES FOR WOMEN AND UNDER-REPRESENTED MINORITIES REFERENCES

Bajaj, G., Carmichael, K.D., "What Attributes aAre Necessary to be Selected for an Orthopaedic Surgery Residency Position: Perceptions of Faculty and Residents," Southern Medical Journal. 2004 December;97(12): 1179-85 Berner, E.S., Brooks, C.M., Erdmann, J.B., "Use of the USMLE to Select Residents," Academic Medicine. 1993 October;68(10):753-9. Berstein, A.D., Jazrawi, L.M., Della Valle, C.J., Zuckerman, J.D., "Orthopaedic Residents Selection Criteria," Journal of Bone and Joint Surgery. Am.2003 July;85-A(7):1400. Carmichael, K.D., Westmoreland, J.B., Thomas, J.A., Patterson, R.M., "Relation of Residency Selection Factors to Subsequent Orthopaedic In Training Examination Performance," Southern Medical Journal. 2003 May;98(5): 528-32 Case, S.M., Swanson, D.B., "Validity of NBME Part I and Part II Scores for Selection of Residents in Orthopaedic Surgery, Dermatology, and Preventative Medicine," Academic Medicine. 1993 February;68(2 sup): S51-6. Clark, R., Evans, E.B., Ivey, F.M., "Characteristics of Successful and Unsuccessful Applicants to Orthopaedic Residency Programs," Clinical Orthopaedics and Related Research. 1989;241:257264. Dailey, S.W., Brinker, M.R., Elliott, M.N., "Orthopaedic Resident's Perceptions of the Content and Adequacy of Their Residency Training," American Journal of Orthopaedics. 1999 January;28(1):55. Dale, J.A., Schmitt, C.M., Crosby, L., "Misrepresentation of Research Criteria by Orthopaedic Residency Applicants," Journal of Bone and Joint Surgery. 1999;81-A(12):1679-1681. Dirschl, D.R., "Scoring of Orthopaedic Residency Applications: Is a Scoring System Reliable?" Clinical Orthopaedics and Related Research. 339:260-264. Dirschl, D.R., Campion, E.R., Gilliam, K., "Resident Selection and Predictors of Performance: Can We Be Evidence-Based?" Clinical Orthopaedics and Related Research. 2006 August;449:44-9. Dirschl, D.R., Dahners, L.E., Adams, G.L., Crouch, J.H., and Wilson, F.C., "Correlating Selection

14

Criteria with Subsequent Performance as Residents," Clinical Orthopaedics and Related Research. 2002 June;399:265-71. Edmond, M.B, Deschenes, J.L., Eckler, M., Wenzell, R.P., "Racial Bias in Using USMLE Step I Scores to Grant Internal Medicine Residency Interviews," Academic Medicine. 2001;76:12531256. Evarts, C.M., "Resident Selection: A Key to the Future of Orthopaedics," Clinical Orthopaedics and Related Research. 2006 August;449:39-43. Fine, P.L., Hayward, R.A., "Do Tthe Criteria of Resident Selection Committees Predict Residents' Performances?" Academic Medicine. 1995; September;70(9):834-8.

Gilbaert, M., Cusimano, M., Regehr, G., "Evaluating Surgical Resident Selection Procedures," American Journal of Surgery. 2001;181(2001):221-225. Herndon, J.H., Allan, B.J., Dyer G., Gawa, A., Zurakowski, D., "Predictors of Success on the American Board of Orthopaedic Surgery Examination," Clinical Orthopaedics and Related Research. 2009 September;467(9):2436-45-EPUB 2009 June 26. Luri, S.J., Lambert, D.R., Grady-Weliky, T.A., "Relationship Between Dean's Letter Rankings and Later Evaluations by Residency Program Directors," Teach and Learn Medicine. 2007 Summer;19(3):251-6. Mallott, D., "Interview, Dean's Letter, and Affective Domain Issues," Clinical Orthopaedics and Related Research. 2006 August;449:56-61. Sherry, E., Mobbs, R., Henderson, A., "Becoming an Orthopaedic Surgeon: Background of Trainees and Their Opinions of Selection Criteria for Orthopaedic Training," AUST-NZ J Surgery. 1996;Vol 66,473-477. Smilen, S.W., Funai, E.F., Bianco, A.T., "Residency Selection: Should Interviewers be Given Applicants' Board Scores?" American Journal of Obstetrics and Gynecology. 2001;184:508-513. Thordarson, D.B, Patzatkis, M.J., "Resident Selection: How Are We Doing, and Why?" Clinical Orthopaedics and Related Research. 2007 June;459:255-9. Turner, N.S., Shaughnessy, W.J., Berg, E.J., Larson, D.R., Hanssen, A.D., "A Quantitative Composite Scoring Tool for Orthopaedic Residency Screening and Selection," Clinical Orthopaedics and Related Research. 2006 August;449:50-5.

15

White, A.A., "Resident Selection: Are We Putting the Cart Before the Horse?" Clinical Orthopaedics and Related Research. 2002 June;(399):253-4.

16

ACCREDITATION COUNCIL OF GRADUATE MEDICAL EDUCATION (ACGME) CORE COMPETENCIES AND THE DEVELOPMENT OF A COMPETENCY-BASED ORTHOPAEDIC CORE CURRICULUM REFERENCES

Arnold, L., "Responding to the Professionalism of Learners and Faculty in Orthopaedic Surgery," Clinical Orthopaedics and Related Research. 2002 August;449:205-13. Batalden, P., Leach, D., Swing, S., Dreyfus, H., Dreyfus, S., "General Competencies and Accreditation in Graduate Medical Education," Health Affairs. (Milwood) 2002 SeptemberOctober;21(5):103-111. Beach, P.S., Bar-On, M., Baldwin, C., Kittredge, D., Trimm, R.S., Henry, R., "Evaluation of the Use of an Interactive, On-Line Resource for Competency-Based Curriculum Development," Academic Medicine. 2009 September;84(9):1269-75. Brasel, K.J., Bragg, D., Simpson, D.E., Weigelt, J.A., "Competencies Using Established Residency Training Program Assessment Tools," Surgery. 2002 February;131(2):214-5. Carraccio, C., Englander, R., "Evaluating Competence Using a Portfolio: A Literature Review and Web-Based Application to the ACGME Competencies," Teach Learn Med. 2004 Fall;16(4):381.7. Cornwall, R., "Teaching Professionalism in Orthopaedic Residency," Journal of Bone and Joint Surgery. Am. 2001 April;83-A(4):626.8. Cruess, R.L., "Teaching Professionalism: Theory, Principles, and Practices," Clinical Orthopaedics and Related Research. 2006 August;449:177-85. Delzell, J.E., Ringdahl, E.N., Kruse, R.L., "The ACGME Core Competencies: A National Survey of Family Medicine Program Directors," Family Medicine. 2005 September;37(8):576-80. DiGiovanni, B.F., Gillespie, B.T., Flemister, A.S., Baumhauer, J.F., "Using Resident Input to Identify and Integrate Essential Components of a Foot and Ankle Rotation," Foot and Ankle International. 2006 September; 27(9):728-33. Gregory, J.K., Lachman, N., Camp, C.L., Chen, L.P., Pawlina, W., "Restructuring a Basic Science Course for Core Competencies: An Example from Anatomy Teaching," Medical Teaching. 2009 September;31(9):855-61. Heard, J.K., Allen, R.M., Clardy, J., "Assessing the Needs of Residency Program Directors to Meet the ACGME General Competencies," Academic Medicine. 2002 July;77(7):750.

17

Hurwitz, S.R., "Guidelines for Teaching the Foot and Ankle in Orthopaedic Residency," Foot and Ankle International. 1999 April;20(4):272-5. Hutol, O.A., Carpenter, R.O., Tarpley, J.L., Lomis, K.D., "Missed Opportunities: A Descriptive Assessment of Teaching and Attitudes Regarding Communication Skills in a Surgical Residency," Current Surgery. 2006 November-December;63(6):408-9. Lee, A.G., Beaver, H.A., Greenlee, Oetting, T.A., Boldt, H.C., Olsen, R.J., Abramoff, "Teaching and Assessing Systems-Based Competency in Ophthalmology Residency Training Programs," Survey of Ophthalmology. 2007 November-December;52(6):680-9.

Lee, A.G., Boldt, H.C., Golnik, K.C., Arnold, A.C., Oetting, T.A., Beaver, H.A., Olsen, R.J., Carter, K., "Using the Journal Club to Teach and Assess Competence Practice-Based Learning and Improvement: A Literature Review and Recommendation for Implementation," Survey of Ophthalmology. 2005 November-December; 50(6):542-8. Long, D.M., "Competency-Based Residency Training: The Next Advance in Graduate Medical Education," Academic Medicine. 2000 December;75(12):1178-83. Moskowitz, E.J., Nash, D.B., "Accreditation Council for Graduate Medical Education Competencies: Practice-Based Learning and Systems-Based Practice," American Journal of Medical Quality. 2007 September-October;22(5):351-82. Lurie, S.J., Mooney, C.J., Lyness, J.M., "Measurements of the General Competencies of the Accreditation Council for Graduate Medical Education: A Systematic Review," Academic Medicine. 2009 March;84(3):301-9. Pellegrini, C.A., "Invited Commentary: The ACGME Outcome Project, American Council of Graduate Medical Education," Surgery. 2002 February;131(2):205-9. Risner, B., Nyland, J., Crawford, C.H., Roberts, Johnson, J.R., "Orthopaedic In Training Examination Performance: A Nine Year Review of a Residency Program Data Base," Southern Medical Journal. 2008 August; 101(8):791-6. Rosenfeld, J.C., "Using the Morbidity and Mortality Conference to Teach and Assess the ACGME General Competencies," Current Surgery. 2005 November-December;62(6):664-9. Stiles, B.M., Reece, T.B., Hedric, T.L., Sawyer, R.G., "General Surgery Morning Report: A Competency-Based Conference that Enhances Patient Care and Resident Education," Current Surgery. 2006 November-December; 63(6):385-90. Swing, S.R., "The ACGME Outcome Project: Retrospective and Prospective," Medical Teaching. 2007 September; 29(7):648-54.

18

Thomas, R.L., Allen, R.M., "Use of Computer-Assisted Learning Module to Achieve ACGME Competencies in Orthopaedic Foot and Ankle Surgery," Foot and Ankle International. 2003 December;24(12):938-41. Varkey, P., Karlapudi, S., Rose, S., Nelson, R., Warner, M., "A Systems Approach for Implementing Practice-Based Learning and Improvement, and Systems-Based Practice in Graduate Medical Education," Academic Medicine. 2009 March;84(3):335-9. Yaszay, B., Kubiak, E., Agel, J., Hanel, D.P., "ACGME Core Competencies: Where Are We?" Orthopaedics. 2009 March;32(3):171.

19