Editorial

am delighted to write to all of you for the first time as editor I of Educational Measurement: fssues and Practice (EM;fP). EM;fP has a long history of addressing the most important measurement issues of the day and illustrating cutting-edge analytic methods and analyses. As I looked over past issues, I found that E M P has published articles on topics varying as widely in interest and subject matter as the introduction of di- rect writing assessments (1984 and 2002) to the so-called “Lake Wobegou Effect” (1990), as well as standard setting and differential item functioning (several times).

The quality ofEM;fP over the years is a testament to the ex- cellent work of the many authors who have chosen EMfP as an outlet for their work and, of course, to the rigor and care of past editors and editorial advisory boards. I will do my best to uphold the standards of quality and broad-mindedness that EMYP has established. I hope that all of you will continue to consider E M P as an outlet for your work and even consider writing manuscripts specifically targeted toward EM;fP and its audience.

I would like you to join me in thanking, ahead of time, a number of people who are crucial to pub1ishingEM;fP. First, I would like to acknowledge the newEM;fP Editorial Advisory Board. As a group, its members represent the diversity of our organization; as individuals, they embody the dedication of NCME members to advancing our field and the spirit of vol- unteerism that is so important to sustaining NCME. (I have been pleased with the responsiveness of several of our col- leagues who already have served as outside reviewers of the manuscripts I have received since last June.) I would also like to acknowledge Associate Editor Holly Cutting Baker. Holly has had a successful career as an editor of Educational Leadership and as book and journal editor for the National Council of Teachers of Mathematics. Along with Holly, I would like to acknowledge Kelly Westphalen, a research assistant at my organization, who manages correspondence for the man- uscript review process so reliably and efficiently. In addition, several AERA staff members play significant roles in publish- ing NCME’s journals, including Director of Publications Linda Dziobek, and Michael Snyder, the managing editor for this issue.

Finally, I want to express my admiration and gratitude to Professor Jeri Benson, the immediate past editor of E M P . Jeri nurturedEM;fP through the last 3 years, produced 12 su- pcrb issues, and passed along an excellent database and cor- respondence system. I especially appreciate her generous ad- vice and support to me during my transition as editor during the last 9 months.

Visual Displays on EMIP Covers I have enjoyed the artwork that Jeri Benson has published on the cover of EM;fP. In doing so she followed the lead of her predecessor as editor, the classicist Jeffrey Smith, and put her own contemporary local spin on the idea. As you can see on the cover of this issue, I have chosen a different approach. I intend to print on the cover of each issue ofEMPvisua1 displays that

(a) are emblematic of concepts and tools in our field or (b) represent significant problems in education and educational testing. Displays may include graphs, charts, and numerical and text tables. I will provide space in this editorial column for a brief discussion of the interpretation and significance of the displays that I select for the cover. I am glad to acknowledge the work of Edward Tufte and Howard Wainer, among others, both of whom have encouraged us to appreciate the aesthetic qualities of visual displays and taught us how to use visual dis- plays insightfully. I encourage all of you to submit visual dis- plays and supporting discussions for my consideration for the covers of the remaining 11 issues that I will edit. I already have received two submissions in response to informal invitations. Please give this invitation some thought.

This issue’s cover display highlights one of the most sig- nificant problems we face in education: persistent subgroup achievement gaps. I retrieved these line graphs from the website of the National Center for Education Statistics (http://nces.ed,gov./nationsreportcard/reading/results2003/ 1unch.asp). The line graphs represent average reading per- formance on NAEP, The Nation’s Report Card, in recent years. The darker black lines and scores reflect the per- formance of all students who participated in these NAEP administrations, including students with disabilities and English-language learners for whom test administration ac- commodations were permitted. The dotted gray lines and lighter-color scores for 1998 and 2000 portray performance when accommodations were not permitted. Official NAEP re- sults now include scores from students who are provided test administration accommodations.

I have chosen this display because it reveals several details of the achievement-gap story. First, and foremost for me, it re- minds us that achievement gaps can be explained by SES in- dicators (e.g., freeheduced-price lunch) just as readily as by raciavethnic group membership. We can see a glimmer of op- timism when we look at achievement differences this way: Schools can find ways to overcome the limiting effects of poverty on learning. Second, this display illustrates the per- sistence of achievement gaps, even over this relatively short time period. Third, it calls our attention to recent efforts to in- clude all students in mandatory educational assessments and to the information about the achievement of students with dis- abilities and English-language learners. Finally, I have chosen a display of reading scores because the development of read- ing proficiency in early grades (rather, early in life) is a gate- keeper for later school achievement and long-term academic success. In anticipation of some critiques, I acknowledge that I have not addressed all considerations for interpreting this display (e.g., background on the score scales and scaling pro- cedures, the practical significance of the differences between means). Please send your comments for possible publication to sferrara@air. org.

The Articles in This Issue I enjoyed very much reading the three articles in this issue on important practical matters in educational testing. Rebecca

4 Educational Measurement: Issues and Practice

Zwick and Lizabeth Schlemer describe results from their study of the validity of using SAT scores to predict freshman grade point averages for linguistic minorities. They use students’ self-reported first language in place of race and ethnicity. Concerns about prediction bias and other sources of invalidity for language and other minorities are well known and have been discussed for a range of educational tests. Zwick and Schlemer find that SAT scores improve the accuracy of pre- dictions of freshman. grade point average in many instances. Furthermore, they emphasize the importance of examining data from native and non-native English speakers separately in validation studies.

It may not be widely acknowledged, but it is clear that va- lidity theory and conception in educational assessment re- main far ahead of validation practice. This gap may be ex- plained by a number of factors. One factor may be the limited availability of practical guidance in conducting validity stud- ies that are consistent with current validity theory. Thomas Haladyna and Steven Downing have taken a big step in bridg- ing this gap by proposing a taxonomy of potential sources of construct-irrelevant test score variance. They define con- struct-irrelevant variance and situate it within contemporary views of validity. Their taxonomy organizes 21 instances of construct-irrelevant variance into four categories of poten- tial sources. And they evaluate the amount of research that

is available on each instance. The taxonomy provides much- needed guidance for our colleagues who are developing and examining conceptions of validity in educational measure- ment, those who conduct validation research, and those who manage assessment programs and must design and imple- ment validity research agendas.

Finally, Gary Skaggs provides practical advice to NCME and our field regarding the use of psychometric and statisti- cal analysis software in published studies. He has extended the work of an NCME ad hoe committee and examined the software cited in four educational measurement and sta- tistics journals (including NCME’s Journal ofEducational Measurement). He finds that the majority of these statistical and Psychometric software programs are written by article authors, may not be available to other analysts, and may not be well documented. He also notes that although the use of pro- prietary software appears not to be a problem at this time, it could limit the replication, evaluation and validation, and extension of published research in the future.

I hope that you enjoy and benefit from reading these ar- ticles as much as I have. And let me hear from you about EMYP.

Steve Ferrara Editor

Spring 2004 5