large scale assessment in oral communication

Large Scale Assessment in Oral Communication

P-12 and Higher Education

Third Edition

Large Scale Assessm

ent in Oral Com

munication

Oral CommunicationP-12 and Higher Education

Third Edition

Sherwyn P. MorrealeUniversity of Colorado at Colorado Springs

Philip M. BacklundCentral Washington University

in collaboration withEllen A. Hay

Augustana CollegeDouglas K. Jennings

Illinois State University

NCA00020_Text.indd iNCA00020_Text.indd i 6/11/07 8:25:21 PM6/11/07 8:25:21 PM

Large Scale Assessment in Oral Communication

P-12 and Higher Education

Third Edition

Sherwyn P. MorrealeUniversity of Colorado at Colorado Springs

Philip M. BacklundCentral Washington University

in collaboration withEllen A. Hay

Augustana CollegeDouglas K. Jennings

Illinois State University

NCA00020_Text.indd iNCA00020_Text.indd i 6/11/07 8:25:21 PM6/11/07 8:25:21 PM

Large Scale Assessment in

All NCA Publication Program materials are reviewed within the spirit of academic freedom, promoting the free exchange of ideas. The contents of this publication are the responsibility of its authors and do not necessarily refl ect the offi cial policies or positions of the National Communication Association, its members, its offi cers, or its staff.

© 2007 National Communication Association. All rights reserved.

Brief portions of this publication may be copied and quoted without further permission with the under-standing that appropriate citations of the source will accompany any excerpts. A limited number of cop-ies of brief excerpts may be made for scholarly or classroom use if:

1. the materials are distributed without charge or no fees above the actual duplication costs are charged;

2. the materials are reproductions, photocopies, or copies made by similar processes, and not reprints or republications;

3. the copies are used within a reasonable time after reproduction; 4. the materials include the full bibliographic citation; and 5. the following is also clearly displayed on all copies: "Copyright by the National Communication

Association Reproduced by permission of the publisher."

This permission does not extend to situations in which

1. extensive amounts of material are reproduced or stored in an electronic or similar data retrieval system2. a fee above actual duplicating costs is charged or if there exists a reasonable expectation of profi t3. the material is reproduced or reprinted for other than scholarly or educational purposes.

In such cases, permission must be obtained prior to reproduction and generally a reasonable fee will be assessed. Requests for permission to reproduce should be addressed to the Publications Manager.

National Communication Association 1765 N Street, NW Washington, D.C., 20036

NCA Non-Serial Publication Series Gust Yep, Editor

San Francisco State University

The NCA Non-serial Publications (NSP) Program publishes book-length projects focusing on theoretical and/or pedagogical issues related to the study and practice of human communication in a variety of contexts. Projects grounded in social scientifi c, interpretive, critical, performance, and rhetorical approaches and methodologies are included. Diverse views of communication ranging from microscopic (e.g., social cognition, affect and emotion in communication) to macroscopic (e.g., public discourse, systems of representation) are also included. Topics that have been central to the history of the discipline as well as those that have been marginalized and excluded in the discipline are included as are projects with an inclusive, interdisciplinary, and social justice agenda.

NCA00020_Text.indd iiNCA00020_Text.indd ii 6/11/07 8:25:21 PM6/11/07 8:25:21 PM

All NCA Publication Program materials are reviewed within the spirit of academic freedom, promoting the free exchange of ideas. The contents of this publication are the responsibility of its authors and do not necessarily refl ect the offi cial policies or positions of the National Communication Association, its members, its offi cers, or its staff.

© 2007 National Communication Association. All rights reserved.

Brief portions of this publication may be copied and quoted without further permission with the under-standing that appropriate citations of the source will accompany any excerpts. A limited number of cop-ies of brief excerpts may be made for scholarly or classroom use if:

1. the materials are distributed without charge or no fees above the actual duplication costs are charged;

2. the materials are reproductions, photocopies, or copies made by similar processes, and not reprints or republications;

3. the copies are used within a reasonable time after reproduction; 4. the materials include the full bibliographic citation; and 5. the following is also clearly displayed on all copies: "Copyright by the National Communication

Association Reproduced by permission of the publisher."

This permission does not extend to situations in which

1. extensive amounts of material are reproduced or stored in an electronic or similar data retrieval system2. a fee above actual duplicating costs is charged or if there exists a reasonable expectation of profi t3. the material is reproduced or reprinted for other than scholarly or educational purposes.

In such cases, permission must be obtained prior to reproduction and generally a reasonable fee will be assessed. Requests for permission to reproduce should be addressed to the Publications Manager.

National Communication Association 1765 N Street, NW Washington, D.C., 20036

NCA Non-Serial Publication Series Gust Yep, Editor

San Francisco State University

The NCA Non-serial Publications (NSP) Program publishes book-length projects focusing on theoretical and/or pedagogical issues related to the study and practice of human communication in a variety of contexts. Projects grounded in social scientifi c, interpretive, critical, performance, and rhetorical approaches and methodologies are included. Diverse views of communication ranging from microscopic (e.g., social cognition, affect and emotion in communication) to macroscopic (e.g., public discourse, systems of representation) are also included. Topics that have been central to the history of the discipline as well as those that have been marginalized and excluded in the discipline are included as are projects with an inclusive, interdisciplinary, and social justice agenda.

NCA00020_Text.indd iiNCA00020_Text.indd ii 6/11/07 8:25:21 PM6/11/07 8:25:21 PM

Acknowledgements

We are profoundly indebted to the editors and reviewers of the volume’s fi rst edition. Most particularly, we acknowledge the editors of the fi rst edition, Don Rubin and Nancy Mead, who worked in collaboration with John Daly, Patrick Dickson, James McCroskey, and Janis Patterson. We also would like to acknowledge the editorial contributions to the second edition of Judith Dallinger, Ray Fenton, Neil O’Leary, and Don Rubin. For this, the third edition, we are indebted particularly to the editors of the two main sections of the volume, Ellen Hay and Douglas Jennings. We are also indebted to the editor of the NCA Non-serial Publications Series, Gust Yep, and the staff of the NCA National Offi ce, whose support made this third volume possible.

NCA00020_Text.indd iiiNCA00020_Text.indd iii 6/11/07 8:25:21 PM6/11/07 8:25:21 PM

iii

Acknowledgements

We are profoundly indebted to the editors and reviewers of the volume’s fi rst edition. Most particularly, we acknowledge the editors of the fi rst edition, Don Rubin and Nancy Mead, who worked in collaboration with John Daly, Patrick Dickson, James McCroskey, and Janis Patterson. We also would like to acknowledge the editorial contributions to the second edition of Judith Dallinger, Ray Fenton, Neil O’Leary, and Don Rubin. For this, the third edition, we are indebted particularly to the editors of the two main sections of the volume, Ellen Hay and Douglas Jennings. We are also indebted to the editor of the NCA Non-serial Publications Series, Gust Yep, and the staff of the NCA National Offi ce, whose support made this third volume possible.

NCA00020_Text.indd iiiNCA00020_Text.indd iii 6/11/07 8:25:21 PM6/11/07 8:25:21 PM

iii

NCA00020_Text.indd ivNCA00020_Text.indd iv 6/11/07 8:25:22 PM6/11/07 8:25:22 PM NCA00020_Text.indd ivNCA00020_Text.indd iv 6/11/07 8:25:22 PM6/11/07 8:25:22 PM

Large Scale Assessment of Oral Communication P-12 or Higher Education v

Table of Contents

Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .iii

Editors’ Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vi Background to this Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vi Objectives and Description of this Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vi Structure and Coverage in this Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Guiding Principles for the Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Part I. Understanding the Assessment of Oral Communication . . . . . . . . . . . . . . . . . . . . . . . . 1 The Importance of Oral Communication Skills and Instruction . . . . . . . . . . . . . . . . . . . 1 The Usefulness of Oral Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Assessment Utility and Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Oral Assessment for Programmatic Decision-Making . . . . . . . . . . . . . . . . . . . . . . . . 3 Oral Assessment for Certifying Student Competencies . . . . . . . . . . . . . . . . . . . . . . . . 3 Procedures for Reviewing Assessment Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Instrument Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Instrument Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Test Fairness and Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Issues in Assessing Oral Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Content of Assessment Instruments for Speaking and Listening . . . . . . . . . . . . . . . . . 5 Criteria of Assessment Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Types of Responses and Scoring Procedures Used in Assessment Instruments . . . . . . 7 Administrative Feasibility of Assessment Instruments . . . . . . . . . . . . . . . . . . . . . . . . . 8 Target Populations and Potential Sources of Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Locally Developed Assessment Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Speaking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Listening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Alternative Assessment and Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Final Questions to Ask About Assessment and What Should be Tested . . . . . . . . . . . . 12 Are the Measures You Choose to Use or Develop Valid? . . . . . . . . . . . . . . . . . . . . . . 12 Are Assessment Instruments Reliable?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 What Measurement Techniques are Presently Available? . . . . . . . . . . . . . . . . . . . . . 13 Are Assessment Instruments Inherently Susceptible to Group and Other Biases? . . 13 Should Communication Competence Be Assessed? . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Part II: Oral Communication Assessment Instruments for P- 12 . . . . . . . . . . . . . . . . . . . . . . 17 Background and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Table of Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Reviews of Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Part III: Oral Communication Assessment Instruments for Higher Education . . . . . . . . . . 48 Background and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Table of Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Reviews of Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

NCA00020_Text.indd vNCA00020_Text.indd v 6/11/07 8:25:22 PM6/11/07 8:25:22 PM

Large Scale Assessment of Oral Communication P-12 or Higher Education v

Table of Contents

Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .iii

Editors’ Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vi Background to this Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vi Objectives and Description of this Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vi Structure and Coverage in this Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Guiding Principles for the Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Part I. Understanding the Assessment of Oral Communication . . . . . . . . . . . . . . . . . . . . . . . . 1 The Importance of Oral Communication Skills and Instruction . . . . . . . . . . . . . . . . . . . 1 The Usefulness of Oral Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Assessment Utility and Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Oral Assessment for Programmatic Decision-Making . . . . . . . . . . . . . . . . . . . . . . . . 3 Oral Assessment for Certifying Student Competencies . . . . . . . . . . . . . . . . . . . . . . . . 3 Procedures for Reviewing Assessment Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Instrument Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Instrument Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Test Fairness and Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Issues in Assessing Oral Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Content of Assessment Instruments for Speaking and Listening . . . . . . . . . . . . . . . . . 5 Criteria of Assessment Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Types of Responses and Scoring Procedures Used in Assessment Instruments . . . . . . 7 Administrative Feasibility of Assessment Instruments . . . . . . . . . . . . . . . . . . . . . . . . . 8 Target Populations and Potential Sources of Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Locally Developed Assessment Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Speaking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Listening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Alternative Assessment and Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Final Questions to Ask About Assessment and What Should be Tested . . . . . . . . . . . . 12 Are the Measures You Choose to Use or Develop Valid? . . . . . . . . . . . . . . . . . . . . . . 12 Are Assessment Instruments Reliable?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 What Measurement Techniques are Presently Available? . . . . . . . . . . . . . . . . . . . . . 13 Are Assessment Instruments Inherently Susceptible to Group and Other Biases? . . 13 Should Communication Competence Be Assessed? . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Part II: Oral Communication Assessment Instruments for P- 12 . . . . . . . . . . . . . . . . . . . . . . 17 Background and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Table of Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Reviews of Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Part III: Oral Communication Assessment Instruments for Higher Education . . . . . . . . . . 48 Background and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Table of Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Reviews of Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

NCA00020_Text.indd vNCA00020_Text.indd v 6/11/07 8:25:22 PM6/11/07 8:25:22 PM

vi Large Scale Assessment of Oral Communication P-12 or Higher Education

Editors’ Foreword

Sherwyn P. Morreale and Philip M. BacklundThe typical practice of educational and psychological measurement is guided by the adage, “If a trait exists, it ex-

ists to some degree. If it exists to some degree, it can be measured.” The mission of the measurement and assessment enterprise, as traditionally construed, is encoded in the criteria of psychometric adequacy. That mission, stripped to its technical essence, is just this: “Thou shalt maximize the proportion of trait-relevant variance in any test score.” Now, however, the educational measurement community recognizes that achieving accurate measurement is only one of several goals for assessment. Assessment and measurement have expanded to accommodate considerations of the consequences of assessment for learners, teachers, and communities.

Given these more inclusive conceptions of assessment and measurement validity, what might be a proper mission for the community of communication educators and researchers who are concerned with the ways in which schools and colleges assess and test oral communication skills? Relatedly, what is a proper mission for the third edition of a publication such as this one?

Background to This Publication

When the fi rst incarnation of this volume was published in 1985, the mission was primarily to use assessment as a tool for consciousness-raising about the career and life values of oral communication. Speaking and listening were to be treated as more than merely the precursors of literate behavior. Efforts at raising educators’ consciousness about the centrality of oral communication may be judged a qualifi ed success. Witness, for example, that oral com-munication is infused in the national standards for English and Language Arts. At least one-quarter of the standards directly address classroom interaction and students’ speaking and listening skills. Similarly, “The College Board Standards for College Success” include not just reading and writing but also speaking, listening, and media literacy (VanderVeen, 2006).

Accordingly, the second edition of this NCA volume, published in 1995, identifi ed a second stage in communica-tion assessment. That volume expanded its mission to address the effective and appropriate use of communication assessment processes and measures by:

• Ensuring attention to developing and using alternative assessment methods, while continuing to adhere to ap-propriate standards for validity and reliability in traditional assessment;

• Promoting test fairness by mitigating sources of cultural interference; and• Contributing to assessment utility by developing effective means for communicating results to stakeholders.

Objectives and Description of this Publication

The challenge of editing this third edition of Large Scale Assessment was fi rst to determine what a third stage might be for communication assessment and then to provide the resources for such a stage of development. Experi-ence tells that communication assessment in higher education is now somewhat institutionalized in campus-wide assessment programs. Therefore, faculty and administrators need to know what instruments and methods for assess-ing communication are now available and how best to use them in their assessment programs. P-12 teachers and administers have similar needs but may have less access to expertise and counsel regarding the process of assessing oral communication. Given this landscape, four primary objectives guided the development of this edition:

• To provide background for understanding oral communication assessment at elementary, secondary, and post-secondary schools;

• To identify and review existing assessment instruments in oral communication for two academic levels: kin-dergarten through grade 12, and colleges and universities (including an evaluation of all instruments in the second edition and a search for new instruments);

• To abstract and describe the assessment instruments and report their availability to interested teachers, schol-ars, and administrators;

NCA00020_Text.indd viNCA00020_Text.indd vi 6/11/07 8:25:22 PM6/11/07 8:25:22 PM

vi Large Scale Assessment of Oral Communication P-12 or Higher Education

Editors’ Foreword

Sherwyn P. Morreale and Philip M. BacklundThe typical practice of educational and psychological measurement is guided by the adage, “If a trait exists, it ex-

ists to some degree. If it exists to some degree, it can be measured.” The mission of the measurement and assessment enterprise, as traditionally construed, is encoded in the criteria of psychometric adequacy. That mission, stripped to its technical essence, is just this: “Thou shalt maximize the proportion of trait-relevant variance in any test score.” Now, however, the educational measurement community recognizes that achieving accurate measurement is only one of several goals for assessment. Assessment and measurement have expanded to accommodate considerations of the consequences of assessment for learners, teachers, and communities.

Given these more inclusive conceptions of assessment and measurement validity, what might be a proper mission for the community of communication educators and researchers who are concerned with the ways in which schools and colleges assess and test oral communication skills? Relatedly, what is a proper mission for the third edition of a publication such as this one?

Background to This Publication

When the fi rst incarnation of this volume was published in 1985, the mission was primarily to use assessment as a tool for consciousness-raising about the career and life values of oral communication. Speaking and listening were to be treated as more than merely the precursors of literate behavior. Efforts at raising educators’ consciousness about the centrality of oral communication may be judged a qualifi ed success. Witness, for example, that oral com-munication is infused in the national standards for English and Language Arts. At least one-quarter of the standards directly address classroom interaction and students’ speaking and listening skills. Similarly, “The College Board Standards for College Success” include not just reading and writing but also speaking, listening, and media literacy (VanderVeen, 2006).

Accordingly, the second edition of this NCA volume, published in 1995, identifi ed a second stage in communica-tion assessment. That volume expanded its mission to address the effective and appropriate use of communication assessment processes and measures by:

• Ensuring attention to developing and using alternative assessment methods, while continuing to adhere to ap-propriate standards for validity and reliability in traditional assessment;

• Promoting test fairness by mitigating sources of cultural interference; and• Contributing to assessment utility by developing effective means for communicating results to stakeholders.

Objectives and Description of this Publication

The challenge of editing this third edition of Large Scale Assessment was fi rst to determine what a third stage might be for communication assessment and then to provide the resources for such a stage of development. Experi-ence tells that communication assessment in higher education is now somewhat institutionalized in campus-wide assessment programs. Therefore, faculty and administrators need to know what instruments and methods for assess-ing communication are now available and how best to use them in their assessment programs. P-12 teachers and administers have similar needs but may have less access to expertise and counsel regarding the process of assessing oral communication. Given this landscape, four primary objectives guided the development of this edition:

• To provide background for understanding oral communication assessment at elementary, secondary, and post-secondary schools;

• To identify and review existing assessment instruments in oral communication for two academic levels: kin-dergarten through grade 12, and colleges and universities (including an evaluation of all instruments in the second edition and a search for new instruments);

• To abstract and describe the assessment instruments and report their availability to interested teachers, schol-ars, and administrators;

NCA00020_Text.indd viNCA00020_Text.indd vi 6/11/07 8:25:22 PM6/11/07 8:25:22 PM

Large Scale Assessment of Oral Communication P-12 or Higher Education vii

• To encourage the development of new instruments and the testing and utilization of existing instruments of merit.

As with the second edition, Part One introduces the reader, whether in P-12 or higher education to the background information needed to engage successfully in oral communication assessment. Then Part Two provides background information for P-12 readers, reviews of assessment instruments appropriate for P-12, and searchable tables to fa-cilitate the instrument selection process. Part Three provides the same resources but for faculty and administrators in higher education.

Structure and Coverage in this Publication

For this publication, The National Communication Association’s (NCA) Criteria for the Assessment of Oral Communication was used to evaluate the reviewed instruments for inclusion in this volume (1993). Particular at-tention was paid to assessment instruments that evaluate communication behaviors, as opposed to instruments that describe behaviors but assign no judgments of quality. In addition, emphasis was placed on measures of communi-cation per se, verbal and nonverbal encoding and decoding in situations ranging from high interaction (interpersonal communication) to extended and uninterrupted discourse (public speaking). The effort was directed toward instru-ments whose main purpose was the measurement of the behavioral or affective domains of communication, or the measurement of a related construct (such as self-esteem, empathy, self-monitoring, loneliness, confl ict style, etc.). The effort did not extend to instruments that directly measure knowledge or cognition in regard to communication. However, inferences about cognition (and affect) can be made based on assessing observed communication behav-iors. Finally, attempts were made to include instruments appropriate for a variety of individuals including nonnative speakers, students of culturally diverse backgrounds, and students with special needs.

Guiding Principles for the Publication

In order to help the reader understand why certain instruments are included, several principles guided those deci-sions and choices.

The fi rst guiding principle was to try to be inclusive rather than exclusive. Perceiving a need in the communica-tion discipline for more information on more assessment tools, we attempted to include tools rather than exclude them. Complete consensus sometimes was diffi cult to obtain regarding which instruments to include and what types of information to provide in each review. Inclusion of an instrument in this volume does not represent an endorse-ment of it by the editors, the reviewers, or by NCA. Our goal is to present suffi cient information regarding each tool, such that the end-user can make an informed choice about its appropriateness, use, and value.

The second decision, and resultant guiding principle, involved the inclusion of assessment tools that are instruc-tionally valuable. The fi rst edition of this volume focused on program, student outcome, and basic skill assessment. As in the second edition, this third edition includes some assessment tools that have their primary purpose in class-room use. Many of the self-report instruments that assess trait or communicator style indicators are designed solely to give students feedback about their thoughts, attitudes, and self-perceived behaviors. They are not designed, nor should they be used, for any type of student outcome assessment. They are included in this volume but identifi ed clearly in the tables that accompany each part, as there is no other volume that catalogues and describes these instru-ments. Communication instructors may fi nd these instruments useful in their classrooms.

To support further the inclusion of various self-report measures, and measures of trait or style, the reader’s at-tention is called to three main purposes or uses for the results of oral assessment: to advise and counsel students, to redirect curriculum content and pedagogy, and to respond to accountability concerns. The fi rst purpose, advis-ing students, can be achieved using self-report tools and measures of self-perceived traits or styles, such as those included in this volume. However, the reader again is cautioned that these instruments were not designed for and should not be used for outcome assessment.

A third decision related to the inclusion of instruments where a paucity of information exists about the tool. For example, information related to psychometric concerns such as reliability, validity, or capacity to operate without biases could not always be located. But in some cases, the reviewers for Parts II and III are of the opinion that the

NCA00020_Text.indd viiNCA00020_Text.indd vii 6/11/07 8:25:22 PM6/11/07 8:25:22 PM

Large Scale Assessment of Oral Communication P-12 or Higher Education vii

• To encourage the development of new instruments and the testing and utilization of existing instruments of merit.

As with the second edition, Part One introduces the reader, whether in P-12 or higher education to the background information needed to engage successfully in oral communication assessment. Then Part Two provides background information for P-12 readers, reviews of assessment instruments appropriate for P-12, and searchable tables to fa-cilitate the instrument selection process. Part Three provides the same resources but for faculty and administrators in higher education.

Structure and Coverage in this Publication

For this publication, The National Communication Association’s (NCA) Criteria for the Assessment of Oral Communication was used to evaluate the reviewed instruments for inclusion in this volume (1993). Particular at-tention was paid to assessment instruments that evaluate communication behaviors, as opposed to instruments that describe behaviors but assign no judgments of quality. In addition, emphasis was placed on measures of communi-cation per se, verbal and nonverbal encoding and decoding in situations ranging from high interaction (interpersonal communication) to extended and uninterrupted discourse (public speaking). The effort was directed toward instru-ments whose main purpose was the measurement of the behavioral or affective domains of communication, or the measurement of a related construct (such as self-esteem, empathy, self-monitoring, loneliness, confl ict style, etc.). The effort did not extend to instruments that directly measure knowledge or cognition in regard to communication. However, inferences about cognition (and affect) can be made based on assessing observed communication behav-iors. Finally, attempts were made to include instruments appropriate for a variety of individuals including nonnative speakers, students of culturally diverse backgrounds, and students with special needs.

Guiding Principles for the Publication

In order to help the reader understand why certain instruments are included, several principles guided those deci-sions and choices.

The fi rst guiding principle was to try to be inclusive rather than exclusive. Perceiving a need in the communica-tion discipline for more information on more assessment tools, we attempted to include tools rather than exclude them. Complete consensus sometimes was diffi cult to obtain regarding which instruments to include and what types of information to provide in each review. Inclusion of an instrument in this volume does not represent an endorse-ment of it by the editors, the reviewers, or by NCA. Our goal is to present suffi cient information regarding each tool, such that the end-user can make an informed choice about its appropriateness, use, and value.

The second decision, and resultant guiding principle, involved the inclusion of assessment tools that are instruc-tionally valuable. The fi rst edition of this volume focused on program, student outcome, and basic skill assessment. As in the second edition, this third edition includes some assessment tools that have their primary purpose in class-room use. Many of the self-report instruments that assess trait or communicator style indicators are designed solely to give students feedback about their thoughts, attitudes, and self-perceived behaviors. They are not designed, nor should they be used, for any type of student outcome assessment. They are included in this volume but identifi ed clearly in the tables that accompany each part, as there is no other volume that catalogues and describes these instru-ments. Communication instructors may fi nd these instruments useful in their classrooms.

To support further the inclusion of various self-report measures, and measures of trait or style, the reader’s at-tention is called to three main purposes or uses for the results of oral assessment: to advise and counsel students, to redirect curriculum content and pedagogy, and to respond to accountability concerns. The fi rst purpose, advis-ing students, can be achieved using self-report tools and measures of self-perceived traits or styles, such as those included in this volume. However, the reader again is cautioned that these instruments were not designed for and should not be used for outcome assessment.

A third decision related to the inclusion of instruments where a paucity of information exists about the tool. For example, information related to psychometric concerns such as reliability, validity, or capacity to operate without biases could not always be located. But in some cases, the reviewers for Parts II and III are of the opinion that the

NCA00020_Text.indd viiNCA00020_Text.indd vii 6/11/07 8:25:22 PM6/11/07 8:25:22 PM

viii Large Scale Assessment of Oral Communication P-12 or Higher Education

instruments appear to be potentially valuable and worth inclusion. The reviewers indicate, on the review for each in-strument, where such information is not available. The user of this volume should also be advised that the reviewers have not necessarily used all of the instruments reviewed herein. The intention of the review is therefore informa-tive, not necessarily valuative. If the reviewer has not had fi rst-hand experience with the tool, comments regarding both strengths and weaknesses may be limited. Finally, the user should note that the prices indicated on each review are the prices available at the time of this publication.

NCA00020_Text.indd viiiNCA00020_Text.indd viii 6/11/07 8:25:22 PM6/11/07 8:25:22 PM

viii Large Scale Assessment of Oral Communication P-12 or Higher Education

instruments appear to be potentially valuable and worth inclusion. The reviewers indicate, on the review for each in-strument, where such information is not available. The user of this volume should also be advised that the reviewers have not necessarily used all of the instruments reviewed herein. The intention of the review is therefore informa-tive, not necessarily valuative. If the reviewer has not had fi rst-hand experience with the tool, comments regarding both strengths and weaknesses may be limited. Finally, the user should note that the prices indicated on each review are the prices available at the time of this publication.

NCA00020_Text.indd viiiNCA00020_Text.indd viii 6/11/07 8:25:22 PM6/11/07 8:25:22 PM

Large Scale Assessment of Oral Communication P-12 or Higher Education 1

A well-known adage has it that of all the creatures inhabiting the earth, fi sh are the least likely to ever dis-cover water. So it is with communication. Speech comes to us as part of our innate endowment as human beings. We are engulfed by communication in all our daily af-fairs, and usually we are not directly aware of our oral communication environment. But, like fi sh and water, communication is nonetheless vital to our well-being and survival.

The Importance of Oral Communication Skills and Instruction

Competence in oral communication – in speaking and listening – is prerequisite to students’ personal and academic success in life. For instance, teachers deliver most instruction for classroom procedures orally to stu-dents. Consequently, students with ineffective listening skills fail to absorb much of the material to which they are exposed. Their problems are intensifi ed when they respond incorrectly or inappropriately, as a result of poor speaking skills. Students who cannot clearly articulate what they know may be wrongly judged as uneducated or poorly informed. Additionally, some speech styles of students can trigger stereotyped expectations of poor ability, expectations that may become self-fulfi lling. And of equal concern, students who aren’t able to effec-tively ask for help from a teacher will not receive it, and typically those reticent students progress more slowly despite what may be a normal level of aptitude.

Beyond the confi nes of school, oral communication competence can contribute to students’ social adjustment and participation in satisfying interpersonal relationships. Youngsters with poor communication skills are some-times viewed as unattractive by their peers and enjoy few friendships. Antisocial and violent behaviors often go hand-in-hand with underdeveloped social and confl ict management skills. On the positive side, the ability to communicate orally is essential to psychological devel-opment. Early research called attention to the fact that self concept is acquired through interaction with others. In psychological terms, achieving self-actualization usu-ally involves communication activities such as making contributions in groups, exerting infl uence over others, and being recognized in a socially acceptable manner.

As students mature to adulthood, numerous national surveys and studies highlight the importance of com-

munication competence – and therefore communication instruction – to professional success in life. According to a U.S. Dept of Labor Report, communication skills will be in demand across most occupations well into the 21st century (Hecker, 2005). Robert Diamond (1997), who conducts curriculum reform workshops across the country, surveys attendees about how academic curri-cula should be reformed. Diamond reports that more than 1,000 representatives of a cross section of disci-plines identifi ed communication at the top of a list of basic competencies for every student graduating from U.S. schools. The Wall Street Journal reported the fi nd-ings of a survey of 480 companies that found that em-ployers ranked communication abilities fi rst among the desirable personal qualities of future employees (Work Week, 1998). And in another survey, 1,000 human re-source managers identifi ed oral communication skills as valuable for both obtaining employment and successful job performance (Winsor, Curtis, and Stephens, 1997). Fortune 500 company executives indicated that their future employees need better communication skills, in-cluding the ability to work in teams and with people of diverse backgrounds (Association Trends, 1997). And, cases studies conducted in high-wage companies reported that essential skills for future workers include problem solving, working in groups, and the ability to communicate effectively orally and in writing (Murane and Levy, 1996).

In sum, students should learn to communicate “at a level that will serve the student both within the univer-sity and in postgraduate professional and personal life” (Kenny, 2002). A meta-analysis of nearly 100 articles, commentaries, and publications calls attention to this importance of the study of communication in contem-porary society (Morreale, Osborn & Pearson, 2000, p.1). Four themes in the bibliography provide support for the importance of communication education to:

• the development of the whole person; • the improvement of the educational enterprise;• the development of responsible world citizens,

both socially and culturally; and• succeeding in one’s career and in the business en-

terprise.

Given this importance of communication instruc-tion, a solid communication assessment program also is

Part I. Understanding the Assessment of Oral Communication

NCA00020_Text.indd Sec1:1NCA00020_Text.indd Sec1:1 6/11/07 8:25:22 PM6/11/07 8:25:22 PM


A well-known adage has it that of all the creatures inhabiting the earth, fi sh are the least likely to ever dis-cover water. So it is with communication. Speech comes to us as part of our innate endowment as human beings. We are engulfed by communication in all our daily af-fairs, and usually we are not directly aware of our oral communication environment. But, like fi sh and water, communication is nonetheless vital to our well-being and survival.

The Importance of Oral Communication Skills and Instruction

Competence in oral communication – in speaking and listening – is prerequisite to students’ personal and academic success in life. For instance, teachers deliver most instruction for classroom procedures orally to stu-dents. Consequently, students with ineffective listening skills fail to absorb much of the material to which they are exposed. Their problems are intensifi ed when they respond incorrectly or inappropriately, as a result of poor speaking skills. Students who cannot clearly articulate what they know may be wrongly judged as uneducated or poorly informed. Additionally, some speech styles of students can trigger stereotyped expectations of poor ability, expectations that may become self-fulfi lling. And of equal concern, students who aren’t able to effec-tively ask for help from a teacher will not receive it, and typically those reticent students progress more slowly despite what may be a normal level of aptitude.

Beyond the confi nes of school, oral communication competence can contribute to students’ social adjustment and participation in satisfying interpersonal relationships. Youngsters with poor communication skills are some-times viewed as unattractive by their peers and enjoy few friendships. Antisocial and violent behaviors often go hand-in-hand with underdeveloped social and confl ict management skills. On the positive side, the ability to communicate orally is essential to psychological devel-opment. Early research called attention to the fact that self concept is acquired through interaction with others. In psychological terms, achieving self-actualization usu-ally involves communication activities such as making contributions in groups, exerting infl uence over others, and being recognized in a socially acceptable manner.

As students mature to adulthood, numerous national surveys and studies highlight the importance of com-

munication competence – and therefore communication instruction – to professional success in life. According to a U.S. Dept of Labor Report, communication skills will be in demand across most occupations well into the 21st century (Hecker, 2005). Robert Diamond (1997), who conducts curriculum reform workshops across the country, surveys attendees about how academic curri-cula should be reformed. Diamond reports that more than 1,000 representatives of a cross section of disci-plines identifi ed communication at the top of a list of basic competencies for every student graduating from U.S. schools. The Wall Street Journal reported the fi nd-ings of a survey of 480 companies that found that em-ployers ranked communication abilities fi rst among the desirable personal qualities of future employees (Work Week, 1998). And in another survey, 1,000 human re-source managers identifi ed oral communication skills as valuable for both obtaining employment and successful job performance (Winsor, Curtis, and Stephens, 1997). Fortune 500 company executives indicated that their future employees need better communication skills, in-cluding the ability to work in teams and with people of diverse backgrounds (Association Trends, 1997). And, cases studies conducted in high-wage companies reported that essential skills for future workers include problem solving, working in groups, and the ability to communicate effectively orally and in writing (Murane and Levy, 1996).

In sum, students should learn to communicate “at a level that will serve the student both within the univer-sity and in postgraduate professional and personal life” (Kenny, 2002). A meta-analysis of nearly 100 articles, commentaries, and publications calls attention to this importance of the study of communication in contem-porary society (Morreale, Osborn & Pearson, 2000, p.1). Four themes in the bibliography provide support for the importance of communication education to:

• the development of the whole person; • the improvement of the educational enterprise;• the development of responsible world citizens,

both socially and culturally; and• succeeding in one’s career and in the business en-

terprise.

Given this importance of communication instruc-tion, a solid communication assessment program also is

Part I. Understanding the Assessment of Oral Communication


2 Large Scale Assessment of Oral Communication P-12 or Higher Education

important. A thorough assessment process for oral com-munication allows academic institutions to demonstrate the value, credibility, and potency of their courses and instruction. With strong assessment programs, adminis-trators and faculty can demonstrate that communication skills are fundamental academic skills and are “criti-cal to students’ success in college and beyond” (Allen, 2002, p.27).

The Usefulness of Oral Assessment

If students’ speaking and listening profi ciencies were systematically evaluated, it is likely that schools would begin to implement more oral communication instruc-tion. One substantial benefi t of large scale assessment of oral communication is that such testing can guide in-novation in the curriculum. Indeed, early experience in Great Britain and elsewhere demonstrates that speech assessment has a “wash back” effect on the amount and kind of speech teaching undertaken in classrooms (Barnes, 1980). Another benefi t of oral communication assessment is that test results can be used to make deci-sions about the best manner in which to place individual students in instructional sequences. Assessment proce-dures that yield micro, rather than macro, judgments can be used for individual diagnostic purposes (Rubin, 1981). For example, students who are assessed as hav-ing diffi culty with fundamental vocal production factors might concentrate on oral reading and presenting, while those whose diffi culties lie in the area of organization might focus on developing skills related to organizing and outlining. Students who demonstrate strengths in, say, literal comprehension of spoken materials might advance to instructional units emphasizing more criti-cal listening. Whatever the purposes of an individual assessment program, we can easily make the claim that better data about student knowledge and skill will lead to better curricular decisions.

Assessment Utility and Communication

An assessment procedure of the highest psychomet-ric adequacy is of no value if its results are not effec-tively communicated to stakeholders (Cronbach, 1988). The stakeholders of educational assessment are easy to identify: students, parents, teachers, administrators, policy makers, and the public. Of these, the ones who often receive the least clear feedback from assessment, particularly from high stakes assessment, are students. Students of course are informed of the bottom line of assessment-based decisions: a passing grade, a fail-ure to receive some certifi cation, a placement in some

particular program. Often, however, no one bothers to help students understand how they can use assessment data for decision making. (A marked exception to that generalization is the reporting of vocational aptitude as-sessment, which usually is deliberately geared toward client/learner decision making.)

Many school districts, it is true, make efforts to pro-vide parents (if not students) with some background materials that will help them interpret their children’s test results. But the Standards for the Assessment of Reading and Writing (Joint Task Force on Assessment, in press) demand something further. They demand that results be reported to parents in a manner that will en-courage dialogue between parents and children about the assessment.

Communicating assessment results for the purpose of promoting dialogue is exactly the point of “construc-tive evaluation” (Johnston, 1992). Constructive evalua-tion, with its concomitant dialogue among stakeholders, is most likely to ensue when assessors use examples of work products, teacher narratives, and other types of “thick description” to help report results. Assessment re-sults that are enhanced by thick descriptions of classroom discourse can empower classroom teachers to make their own local sense of the assessment results. Denuded nu-merical scores by themselves are not likely to support that kind of equity in utilizing assessment data.

Assessment can serve a crucial function in promot-ing the public dialogue regarding educational reform. Unfortunately, reports of test results are often followed in the public forum by sensationalistic indictments and accusations against educators, institutions, and stu-dents. This sort of sensationalism does little to promote productive public discussion. Were assessment results communicated along with more contextual informa-tion about the status of the schools and the nature of the assessment instruments themselves, the quality of the ensuing public responses would no doubt be more con-structive (Joint Task Force on Assessment, in press).

These are some potential missions for communica-tion assessment in the coming decade: to participate in alternative assessment, while continuing to utilize more traditional methods informally; to work toward cultur-ally fairer testing; and to promote the public dialogue regarding educational reform. These tasks build upon the work of the decade past; that is, they raise educators’ consciousness regarding the centrality of oral commu-nication.

To fulfi ll these missions, the most necessary ingredi-ent is a critical mass of tests, measures, and assessment



important. A thorough assessment process for oral com-munication allows academic institutions to demonstrate the value, credibility, and potency of their courses and instruction. With strong assessment programs, adminis-trators and faculty can demonstrate that communication skills are fundamental academic skills and are “criti-cal to students’ success in college and beyond” (Allen, 2002, p.27).

The Usefulness of Oral Assessment

If students’ speaking and listening profi ciencies were systematically evaluated, it is likely that schools would begin to implement more oral communication instruc-tion. One substantial benefi t of large scale assessment of oral communication is that such testing can guide in-novation in the curriculum. Indeed, early experience in Great Britain and elsewhere demonstrates that speech assessment has a “wash back” effect on the amount and kind of speech teaching undertaken in classrooms (Barnes, 1980). Another benefi t of oral communication assessment is that test results can be used to make deci-sions about the best manner in which to place individual students in instructional sequences. Assessment proce-dures that yield micro, rather than macro, judgments can be used for individual diagnostic purposes (Rubin, 1981). For example, students who are assessed as hav-ing diffi culty with fundamental vocal production factors might concentrate on oral reading and presenting, while those whose diffi culties lie in the area of organization might focus on developing skills related to organizing and outlining. Students who demonstrate strengths in, say, literal comprehension of spoken materials might advance to instructional units emphasizing more criti-cal listening. Whatever the purposes of an individual assessment program, we can easily make the claim that better data about student knowledge and skill will lead to better curricular decisions.

Assessment Utility and Communication

An assessment procedure of the highest psychomet-ric adequacy is of no value if its results are not effec-tively communicated to stakeholders (Cronbach, 1988). The stakeholders of educational assessment are easy to identify: students, parents, teachers, administrators, policy makers, and the public. Of these, the ones who often receive the least clear feedback from assessment, particularly from high stakes assessment, are students. Students of course are informed of the bottom line of assessment-based decisions: a passing grade, a fail-ure to receive some certifi cation, a placement in some

particular program. Often, however, no one bothers to help students understand how they can use assessment data for decision making. (A marked exception to that generalization is the reporting of vocational aptitude as-sessment, which usually is deliberately geared toward client/learner decision making.)

Many school districts, it is true, make efforts to pro-vide parents (if not students) with some background materials that will help them interpret their children’s test results. But the Standards for the Assessment of Reading and Writing (Joint Task Force on Assessment, in press) demand something further. They demand that results be reported to parents in a manner that will en-courage dialogue between parents and children about the assessment.

Communicating assessment results for the purpose of promoting dialogue is exactly the point of “construc-tive evaluation” (Johnston, 1992). Constructive evalua-tion, with its concomitant dialogue among stakeholders, is most likely to ensue when assessors use examples of work products, teacher narratives, and other types of “thick description” to help report results. Assessment re-sults that are enhanced by thick descriptions of classroom discourse can empower classroom teachers to make their own local sense of the assessment results. Denuded nu-merical scores by themselves are not likely to support that kind of equity in utilizing assessment data.

Assessment can serve a crucial function in promot-ing the public dialogue regarding educational reform. Unfortunately, reports of test results are often followed in the public forum by sensationalistic indictments and accusations against educators, institutions, and stu-dents. This sort of sensationalism does little to promote productive public discussion. Were assessment results communicated along with more contextual informa-tion about the status of the schools and the nature of the assessment instruments themselves, the quality of the ensuing public responses would no doubt be more con-structive (Joint Task Force on Assessment, in press).

These are some potential missions for communica-tion assessment in the coming decade: to participate in alternative assessment, while continuing to utilize more traditional methods informally; to work toward cultur-ally fairer testing; and to promote the public dialogue regarding educational reform. These tasks build upon the work of the decade past; that is, they raise educators’ consciousness regarding the centrality of oral commu-nication.

To fulfi ll these missions, the most necessary ingredi-ent is a critical mass of tests, measures, and assessment



procedures that effectively measure various dimensions of oral communication. It is important that, collectively, these instruments exhibit diversity. They must gather data using diverse methods, and they must address di-verse types of communication behaviors and skills. And it is important that these instruments exhibit accuracy in measurement. After all, we demand the highest quality of information to meet the challenge of school improve-ment.

Oral Assessment for Programmatic Decision-Making

Speaking and listening tests can also provide valu-able information for program evaluation. Because large scale programs of oral communication improvement are new to some academic institutions, it is especially important to evaluate their effectiveness and to secure data that will enable these programs to be “fi ne tuned.” Program (and teacher) effectiveness is best judged with reference to student achievement on program objectives. If students are not achieving criterion performance lev-els in language use, for example, teachers and admin-istrators will recognize that additional instructional ef-fort needs to be directed to this area. It is worth noting, however, that student achievement can be interpreted as an indicator of program success only when student aptitude and institutional resources-the raw materials with which the program has to work-are also taken into account. Also, student achievement is not the only data that might contribute to program evaluation. Attitudinal outcomes, self- and peer-evaluations, are also useful in-formation for this purpose. The National Communica-tion Association exhibits exemplary models of program assessment in oral communication on their assessment resources web site. These examples can illustrate this process for the teacher or administrator.

Oral Assessment for Certifying Student Competencies

Another use for speaking and listening assessment tests is to certify students as having attained (or not at-tained) levels or degrees of oral communication compe-tency. Certifi cation in basic skills may be increasingly demanded by standards-based education movements such as Goals 2000 (National Education Goals Panel, 1992). Surveys of state and local jurisdictions indicate that some have already adopted large scale tests of speaking and listening skills as a means of certifying students’ competencies in basic skills (National Com-munication Association, 1994).

In summary, the process and results of assessment of oral communication can be of benefi t to students, teach-ers, and administrators. For students, assessment results can be used for advising, counseling, and placement purposes in addition to ensuring that students have de-veloped the expected and necessary oral communication competencies prior to graduation. For faculty, examina-tion of assessment results can be used to revise course content and to redirect teaching. For administrators, assessment of oral communication can be used to im-prove curriculum structure and resource allocation, and to assure accreditation agencies and the public that each college graduate has acquired the necessary knowledge, skills, and behaviors to communicate competently.

Procedures for Reviewing Assessment Instruments

To begin the revision of this volume, committee mem-bers volunteered to research selected areas of instru-ments. The reviewers extensively surveyed the available sources, compared instruments to the NCA criteria, and developed reviews of each instrument according to a common format. A catalogue of instruments that met the criteria of the reviewing committees is presented in two tables. A table and reviews of instruments for grades P-12 appear in Part II of this publication. A table and reviews for instruments for two- and four-year col-leges and universities appear in Part III. The contents of the reviews refl ect the views of the individual reviewers as infl uenced by their expert judgment.

In reviewing the instruments, two points provided general guidance to the reviewers. First, in the absence of a consensually acceptable model of competent com-munication, it is diffi cult to evaluate instruments’ con-struct and content validity. Objectives and competency lists adopted by one assessor may diverge widely from those that guide test selection by another assessor. Sec-ond, it is anticipated that some users of these reviews will not be speech communication scholars, but rather evaluation specialists and school administrators. The instrument review form refl ects the general concerns of this target audience with respect to psychometric adequacy and administrative feasibility. More specifi c discussions of the review and selection process for the instruments are included in the introductions to Parts II and III. To facilitate the reader’s use of Parts II and III, the concepts of validity and reliability are reviewed here as they relate to the instruments reviewed in both Parts II and III followed by a discussion of test fair-ness.



procedures that effectively measure various dimensions of oral communication. It is important that, collectively, these instruments exhibit diversity. They must gather data using diverse methods, and they must address di-verse types of communication behaviors and skills. And it is important that these instruments exhibit accuracy in measurement. After all, we demand the highest quality of information to meet the challenge of school improve-ment.

Oral Assessment for Programmatic Decision-Making

Speaking and listening tests can also provide valu-able information for program evaluation. Because large scale programs of oral communication improvement are new to some academic institutions, it is especially important to evaluate their effectiveness and to secure data that will enable these programs to be “fi ne tuned.” Program (and teacher) effectiveness is best judged with reference to student achievement on program objectives. If students are not achieving criterion performance lev-els in language use, for example, teachers and admin-istrators will recognize that additional instructional ef-fort needs to be directed to this area. It is worth noting, however, that student achievement can be interpreted as an indicator of program success only when student aptitude and institutional resources-the raw materials with which the program has to work-are also taken into account. Also, student achievement is not the only data that might contribute to program evaluation. Attitudinal outcomes, self- and peer-evaluations, are also useful in-formation for this purpose. The National Communica-tion Association exhibits exemplary models of program assessment in oral communication on their assessment resources web site. These examples can illustrate this process for the teacher or administrator.

Oral Assessment for Certifying Student Competencies

Another use for speaking and listening assessment tests is to certify students as having attained (or not at-tained) levels or degrees of oral communication compe-tency. Certifi cation in basic skills may be increasingly demanded by standards-based education movements such as Goals 2000 (National Education Goals Panel, 1992). Surveys of state and local jurisdictions indicate that some have already adopted large scale tests of speaking and listening skills as a means of certifying students’ competencies in basic skills (National Com-munication Association, 1994).

In summary, the process and results of assessment of oral communication can be of benefi t to students, teach-ers, and administrators. For students, assessment results can be used for advising, counseling, and placement purposes in addition to ensuring that students have de-veloped the expected and necessary oral communication competencies prior to graduation. For faculty, examina-tion of assessment results can be used to revise course content and to redirect teaching. For administrators, assessment of oral communication can be used to im-prove curriculum structure and resource allocation, and to assure accreditation agencies and the public that each college graduate has acquired the necessary knowledge, skills, and behaviors to communicate competently.

Procedures for Reviewing Assessment Instruments

To begin the revision of this volume, committee mem-bers volunteered to research selected areas of instru-ments. The reviewers extensively surveyed the available sources, compared instruments to the NCA criteria, and developed reviews of each instrument according to a common format. A catalogue of instruments that met the criteria of the reviewing committees is presented in two tables. A table and reviews of instruments for grades P-12 appear in Part II of this publication. A table and reviews for instruments for two- and four-year col-leges and universities appear in Part III. The contents of the reviews refl ect the views of the individual reviewers as infl uenced by their expert judgment.

In reviewing the instruments, two points provided general guidance to the reviewers. First, in the absence of a consensually acceptable model of competent com-munication, it is diffi cult to evaluate instruments’ con-struct and content validity. Objectives and competency lists adopted by one assessor may diverge widely from those that guide test selection by another assessor. Sec-ond, it is anticipated that some users of these reviews will not be speech communication scholars, but rather evaluation specialists and school administrators. The instrument review form refl ects the general concerns of this target audience with respect to psychometric adequacy and administrative feasibility. More specifi c discussions of the review and selection process for the instruments are included in the introductions to Parts II and III. To facilitate the reader’s use of Parts II and III, the concepts of validity and reliability are reviewed here as they relate to the instruments reviewed in both Parts II and III followed by a discussion of test fair-ness.



Instrument Validity

The section of the instrument review form on validity is concerned with the extent to which the instrument actually measures the skills or knowledge it intends to measure. Va-lidity may be determined in many ways, and the presence of multiple validity studies using different methods and different target populations strengthens the case that the instrument actually measures what it purports to measure.

Predictive validity deals with the ability of the instru-ment to predict performance on another measure that is known to be valid and that is theoretically related to the instrument in question. For example, a test of communi-cation competence might be assumed to predict success in jobs that rely heavily on oral communication.

Concurrent validity is similar to predictive validity except that it focuses on the relationship between indi-viduals’ performance on the instrument in question and on other instruments that measure the same thing. If a group of students were administered a speaking perfor-mance test and were also rated by their speech teacher, then the correlation on these two measures would be a test of concurrent validity of the speaking performance test. Or, if students were administered two different lis-tening tools, one already tested would support the valid-ity of the other. Both predictive and concurrent validity are referred to as criterion validity.

Content validity indicates the degree to which the con-tent of an instrument represents the domain of knowl-edge and skills it intends to measure. Content validity is usually determined through expert judgment. One com-mon method is where experts are given a description of the test’s objectives and then asked to categorize each item by these objectives. Content validity is measured by the degree of agreement among judges in the cat-egory assignments. Content validity is also referred to as face validity or intrinsic representativeness.

Building a case for construct validity takes many forms. Construct validity may derive from any experi-ment that sheds light on the nature of the phenomenon that the instrument is trying to measure. Common bases for construct validity include factor analysis of the items an instrument contains and the known groups method. Also, the instrument has construct validity if it can be theoretically supported in terms of its method of devel-opment or base in the literature.

Instrument Reliability

The section of the review form on reliability reports the measurement accuracy of the instrument. There are various methods for determining reliability.

Test-retest reliability measures the stability of an in-strument over time. Assuming that the respondents have not been exposed to instruction and have not under-gone a major growth in the knowledge or skills being measured, they should receive approximately the same score on an instrument at two points in time. However, test-retest is not always a good measure of reliability, especially of skills that might be expected to change.

In some cases instruments are designed to have al-ternate forms that are equivalent in content and diffi -culty. The correlation between individuals’ scores on the different forms is a test of alternate forms reliability. When we offer students a choice of topics about which to speak in a speech performance evaluation situation, we are assuming that these topics constitute equivalent forms of the same test. Similarly, when we use different small group confi gurations or different interview scripts to elicit oral performance, we must assume that these constitute equivalent forms.

Taking the concept of alternate forms reliability a step further, it is possible to think of an instrument as a random set of items, each of which is a “test” of some part of the content domain. The degree to which the re-spondents’ performance on one item is related to their performance on other items is a measure of internal consistency reliability. Typically, a correlation coeffi -cient is calculated to analyze the relatedness of items. For example, a Cronbach’s alpha considers the correla-tion of each item and all other items on the instrument.

Performance tests are markedly different from pa-per-and-pencil tests. For these tests, measurement takes place within the person who assigns the rating or score. Here the reliability of the scorer is at issue, not the reli-ability of the test. Scoring (or interrater) reliability can be assessed by having more than one person rate the same performance. The correlations or percentages of agreement between raters in these ratings is a test of scoring reliability. Usually scorers are evaluated for reliability after training but before they begin rating. However, to insure that scorers remain consistent over time, it is important to check their reliability periodi-cally during the scoring process as well. If this is done effectively, one rater using a valid instrument can be trained to use that instrument reliably, eliminating the need for multiple raters.

As a part of the development of some large scale as-sessment instruments, norming or criterion referencing studies are conducted, and these are also discussed on the review form. For these studies, the instrument is ad-ministered to a large number of respondents, and the



Instrument Validity

The section of the instrument review form on validity is concerned with the extent to which the instrument actually measures the skills or knowledge it intends to measure. Va-lidity may be determined in many ways, and the presence of multiple validity studies using different methods and different target populations strengthens the case that the instrument actually measures what it purports to measure.

Predictive validity deals with the ability of the instru-ment to predict performance on another measure that is known to be valid and that is theoretically related to the instrument in question. For example, a test of communi-cation competence might be assumed to predict success in jobs that rely heavily on oral communication.

Concurrent validity is similar to predictive validity except that it focuses on the relationship between indi-viduals’ performance on the instrument in question and on other instruments that measure the same thing. If a group of students were administered a speaking perfor-mance test and were also rated by their speech teacher, then the correlation on these two measures would be a test of concurrent validity of the speaking performance test. Or, if students were administered two different lis-tening tools, one already tested would support the valid-ity of the other. Both predictive and concurrent validity are referred to as criterion validity.

Content validity indicates the degree to which the con-tent of an instrument represents the domain of knowl-edge and skills it intends to measure. Content validity is usually determined through expert judgment. One com-mon method is where experts are given a description of the test’s objectives and then asked to categorize each item by these objectives. Content validity is measured by the degree of agreement among judges in the cat-egory assignments. Content validity is also referred to as face validity or intrinsic representativeness.

Building a case for construct validity takes many forms. Construct validity may derive from any experi-ment that sheds light on the nature of the phenomenon that the instrument is trying to measure. Common bases for construct validity include factor analysis of the items an instrument contains and the known groups method. Also, the instrument has construct validity if it can be theoretically supported in terms of its method of devel-opment or base in the literature.

Instrument Reliability

The section of the review form on reliability reports the measurement accuracy of the instrument. There are various methods for determining reliability.

Test-retest reliability measures the stability of an in-strument over time. Assuming that the respondents have not been exposed to instruction and have not under-gone a major growth in the knowledge or skills being measured, they should receive approximately the same score on an instrument at two points in time. However, test-retest is not always a good measure of reliability, especially of skills that might be expected to change.

In some cases instruments are designed to have al-ternate forms that are equivalent in content and diffi -culty. The correlation between individuals’ scores on the different forms is a test of alternate forms reliability. When we offer students a choice of topics about which to speak in a speech performance evaluation situation, we are assuming that these topics constitute equivalent forms of the same test. Similarly, when we use different small group confi gurations or different interview scripts to elicit oral performance, we must assume that these constitute equivalent forms.

Taking the concept of alternate forms reliability a step further, it is possible to think of an instrument as a random set of items, each of which is a “test” of some part of the content domain. The degree to which the re-spondents’ performance on one item is related to their performance on other items is a measure of internal consistency reliability. Typically, a correlation coeffi -cient is calculated to analyze the relatedness of items. For example, a Cronbach’s alpha considers the correla-tion of each item and all other items on the instrument.

Performance tests are markedly different from pa-per-and-pencil tests. For these tests, measurement takes place within the person who assigns the rating or score. Here the reliability of the scorer is at issue, not the reli-ability of the test. Scoring (or interrater) reliability can be assessed by having more than one person rate the same performance. The correlations or percentages of agreement between raters in these ratings is a test of scoring reliability. Usually scorers are evaluated for reliability after training but before they begin rating. However, to insure that scorers remain consistent over time, it is important to check their reliability periodi-cally during the scoring process as well. If this is done effectively, one rater using a valid instrument can be trained to use that instrument reliably, eliminating the need for multiple raters.

As a part of the development of some large scale as-sessment instruments, norming or criterion referencing studies are conducted, and these are also discussed on the review form. For these studies, the instrument is ad-ministered to a large number of respondents, and the



results provide performance benchmarks for future us-ers of the instrument. Norming studies for standardized achievement tests yield charts that transform raw scores into normed scores, most frequently grade equivalence. Standard setting studies are sometimes conducted for tests that measure mastery of specifi c objectives--that is, criterion referenced tests. Data collected from sam-ples of students are usually compared with data from another source, such as teacher ratings, to determine what test scores represent mastery level. A caution for all norming and criterion referencing data is that the characteristics of the original population assessed may be different from the population that the present user is assessing.

Test Fairness and Communication

Typically, the process of test construction and of de-velopment--particularly for high stakes testing-includes several elements designed to minimize adverse impact on minority group test-takers. A bias review panel may review test items to eliminate culture-exclusive content. Statistical analysis will indicate items that function dif-ferentially for different minority groups, and such items will be eliminated from the item pool.

But what of the more subtle, and perhaps more perva-sive, culture-typical communication demands of an as-sessment? Sociolinguistic test analysis (Rubin, 1992) is designed to evaluate ways in which the language and dis-course patterns of assessment procedures can mismatch test-takers’ own cultural norms about communication.

Three kinds of sociolinguistic interference can be de-tected. First, test-takers’ oral dialect patterns may make it more diffi cult for them to ascertain the intent of a question. For example, a speaker of Black English Ver-nacular may experience some additional cognitive load in trying to decode a negative comparative question like, “Why didn’t Jack close as many contracts as Mary?” Second, test-takers may have different discourse norms than those presupposed by a test. By way of illustration, members of certain Native American Indian groups might be very effi cacious in presenting evidence in a persuasive appeal, but then consider it very rude to state explicitly for a listener the conclusion to which the evi-dence points. Finally, sociolinguistic analysis can point out that the entire testing situation might be more alien a communication context for some groups than for oth-ers. The idea of speaking with an interlocutor who sits stone-faced and silent while I respond to her questions might be especially discomforting to people from some African-American communities.

Test fairness in interview and communication assess-ment programs presents especially complex issues (Va-ladez, 1989). The very criteria by which most assessment instruments judge communication effectiveness-elabo-rated speech, unique expression, pointed argumenta-tion, and the like-must be recognized as culture bound, characteristic mainly of middle-class Euro-American communication style (see reviews in Morreale, Gomez, & Shockley-Zalabak, 1992, and in Stiggins, Backlund, & Bridgeford, 1985). To hold minority test-takers to these standards may or may not be justifi ed, but in any event must be understood as a statement about privileg-ing certain culture stances and devaluing others.

A strong case can sometimes be made for a particular culture-linked communication practice as a bona fi de job qualifi cation. For example, the headquarters of a national organization might have legitimate warrant for wishing to employ telephone operators without marked regional accents. On the other hand, test-developers must be careful not to impose a culture-bound standard where none need be. Police dispatchers, for example, need only be highly intelligible. It is quite conceivable that a dispatcher could achieve that quality while yet retaining a speech style prominently marked for region or ethnicity.

Issues in Assessing Oral Communication

Users of oral communication assessment instruments should take into consideration issues that are somewhat unique to this discipline. Several of the most important of these issues are described here to assist the reader in determining the best procedure available to meet the goals of his or her assessment program.

Content of Assessment Instruments for Speaking and Listening

N. A. Mead (1982) makes a useful distinction between process knowledge and content knowledge as related to the goal of assessment. For most academic areas, the educational experience involves acquiring knowledge for application later in life. Testing in those areas (such as history, the sciences, etc.) focuses on the student’s level of knowledge acquisition. However, oral commu-nication is a process skill, and, as such, it makes more sense to assess the student’s performance with speaking and listening than it does to assess the student’s knowl-edge about speaking and listening. Thus, virtually none of the instruments reviewed in this volume assesses the student’s knowledge of the communication process. The content of the instruments focuses on skill acqui-



results provide performance benchmarks for future us-ers of the instrument. Norming studies for standardized achievement tests yield charts that transform raw scores into normed scores, most frequently grade equivalence. Standard setting studies are sometimes conducted for tests that measure mastery of specifi c objectives--that is, criterion referenced tests. Data collected from sam-ples of students are usually compared with data from another source, such as teacher ratings, to determine what test scores represent mastery level. A caution for all norming and criterion referencing data is that the characteristics of the original population assessed may be different from the population that the present user is assessing.

Test Fairness and Communication

Typically, the process of test construction and of de-velopment--particularly for high stakes testing-includes several elements designed to minimize adverse impact on minority group test-takers. A bias review panel may review test items to eliminate culture-exclusive content. Statistical analysis will indicate items that function dif-ferentially for different minority groups, and such items will be eliminated from the item pool.

But what of the more subtle, and perhaps more perva-sive, culture-typical communication demands of an as-sessment? Sociolinguistic test analysis (Rubin, 1992) is designed to evaluate ways in which the language and dis-course patterns of assessment procedures can mismatch test-takers’ own cultural norms about communication.

Three kinds of sociolinguistic interference can be de-tected. First, test-takers’ oral dialect patterns may make it more diffi cult for them to ascertain the intent of a question. For example, a speaker of Black English Ver-nacular may experience some additional cognitive load in trying to decode a negative comparative question like, “Why didn’t Jack close as many contracts as Mary?” Second, test-takers may have different discourse norms than those presupposed by a test. By way of illustration, members of certain Native American Indian groups might be very effi cacious in presenting evidence in a persuasive appeal, but then consider it very rude to state explicitly for a listener the conclusion to which the evi-dence points. Finally, sociolinguistic analysis can point out that the entire testing situation might be more alien a communication context for some groups than for oth-ers. The idea of speaking with an interlocutor who sits stone-faced and silent while I respond to her questions might be especially discomforting to people from some African-American communities.

Test fairness in interview and communication assess-ment programs presents especially complex issues (Va-ladez, 1989). The very criteria by which most assessment instruments judge communication effectiveness-elabo-rated speech, unique expression, pointed argumenta-tion, and the like-must be recognized as culture bound, characteristic mainly of middle-class Euro-American communication style (see reviews in Morreale, Gomez, & Shockley-Zalabak, 1992, and in Stiggins, Backlund, & Bridgeford, 1985). To hold minority test-takers to these standards may or may not be justifi ed, but in any event must be understood as a statement about privileg-ing certain culture stances and devaluing others.

A strong case can sometimes be made for a particular culture-linked communication practice as a bona fi de job qualifi cation. For example, the headquarters of a national organization might have legitimate warrant for wishing to employ telephone operators without marked regional accents. On the other hand, test-developers must be careful not to impose a culture-bound standard where none need be. Police dispatchers, for example, need only be highly intelligible. It is quite conceivable that a dispatcher could achieve that quality while yet retaining a speech style prominently marked for region or ethnicity.

Issues in Assessing Oral Communication

Users of oral communication assessment instruments should take into consideration issues that are somewhat unique to this discipline. Several of the most important of these issues are described here to assist the reader in determining the best procedure available to meet the goals of his or her assessment program.

Content of Assessment Instruments for Speaking and Listening

N. A. Mead (1982) makes a useful distinction between process knowledge and content knowledge as related to the goal of assessment. For most academic areas, the educational experience involves acquiring knowledge for application later in life. Testing in those areas (such as history, the sciences, etc.) focuses on the student’s level of knowledge acquisition. However, oral commu-nication is a process skill, and, as such, it makes more sense to assess the student’s performance with speaking and listening than it does to assess the student’s knowl-edge about speaking and listening. Thus, virtually none of the instruments reviewed in this volume assesses the student’s knowledge of the communication process. The content of the instruments focuses on skill acqui-



sition and attitude development. Educators usually are more interested in what students do with speaking and listening skills rather than in how much students know about those skills. Comments about content below con-cern themselves with student performance and skill in regard to speaking and listening, respectively.

Speaking. The content of speaking assessment pro-cedures is varied. One way in which this focus can be categorized is in terms of mode of discourse. At the elementary-age level, most speaking tasks are either narrative or descriptive. A number of tests designed for non-native speakers also rely on storytelling. For older native English speakers, greater variety is evident. The speaking tasks often call for exposition in the form of extended monologues. Other modes of discourse in-clude extended persuasive monologues or simulated persuasive conversations, telephone conversations, in-troductions, and responses to questions in an interview.

Speech assessment procedures can be categorized in terms of communication situations as well. In particu-lar, it is useful to examine the types of audiences that are featured in oral performance tasks. Of course, in all speaking tests students will be aware of the examiner as an ultimate audience. However, in the majority of instruments reviewed, the examiner is the sole audience to whom students speak. It should be noted that speak-ers do not typically communicate in order that their oral profi ciency be evaluated. Indeed, evaluation usually inhibits communication. To the extent that assessment procedures offer no pretense for speaking other than evaluation, these procedures may yield somewhat inac-curate samples of communication performance.

A single examiner-audience is most natural in inter-view situations. One pitfall of interview situations is that the interviewer may exert overriding infl uence on students’ speech behaviors, resulting in considerable unreliability. A single examiner-audience is most anom-alous in those situations in which students are called upon to deliver a speech to that individual; we do not usually deliver formal speeches to an audience of one. However, this type of testing situation has the advan-tage of not being confounded by audience reaction. The problem of unnatural audiences is somewhat relieved by procedures that simulate situations involving real-istic speaker/audience relations. These procedures may ask students to simulate an emergency telephone call to a police operator, give directions to a stranger, or per-suade a friend to grant some favor. These simulation tasks, however, may confuse speech profi ciency with role-playing ability. A balance must be struck between

the confl icting needs of internal validity (testing what you think you are testing) and external generalizability (to real situations).

Listening. The content of listening assessment instru-ments and procedures is as varied as that of speaking tests. Listening is not a unitary skill, but it is rather a complex of sub skills, each of which is brought into play to greater or lesser degrees depending on the na-ture of the listening task (Lundsteen, 1979). It is natu-ral, therefore, that tests of listening ability tap a variety of skills. Test users should make sure that the selected listening test conforms to their particular measurement objectives.

Most often, listening tests measure literal comprehen-sion of spoken material. It should be noted that compre-hension is generally confused with recall or retention, because questions typically follow some extended dis-course. Some testing methods alleviate this confusion by providing a verbal context that may lessen reliance on memory. Similarly, tests that deliberately select brief passages and present few questions for each passage may tax memory to a lesser extent.

Many listening tests focus on listening for directions, a type of purposeful listening that is readily measured by accuracy of behavioral response (e.g., circling the correct item, drawing the proper path on a map). Other listening skills frequently measured include recogni-tion of speaker’s purpose, or making inferences or in-terpretations beyond material given. Other listening instruments include subtests refl ecting ability to inter-pret paralinguistic cues and also ability to render social judgments from speech. Some sub skills such as vo-cabulary, syntax, and phoneme recognition are critical to communicative listening but are so narrow that they might better be considered receptive language or decod-ing skills. Phonemic discrimination and identifi cation are viewed as essential for reading readiness but should not be construed as measures of listening ability.

Outside of the classroom, the bulk of listening activ-ity takes place in the course of interaction. When refer-ential communication tasks assessing skills of both the speakers and the listeners permit free oral interchange, these tests approximate interactive communication. Other instruments measure interactive listening skill more indirectly by including conversational speech among their listening passages.

Criteria of Assessment Instruments

Another issue related to the content of assessment in-struments that requires examination is their evaluation



sition and attitude development. Educators usually are more interested in what students do with speaking and listening skills rather than in how much students know about those skills. Comments about content below con-cern themselves with student performance and skill in regard to speaking and listening, respectively.

Speaking. The content of speaking assessment pro-cedures is varied. One way in which this focus can be categorized is in terms of mode of discourse. At the elementary-age level, most speaking tasks are either narrative or descriptive. A number of tests designed for non-native speakers also rely on storytelling. For older native English speakers, greater variety is evident. The speaking tasks often call for exposition in the form of extended monologues. Other modes of discourse in-clude extended persuasive monologues or simulated persuasive conversations, telephone conversations, in-troductions, and responses to questions in an interview.

Speech assessment procedures can be categorized in terms of communication situations as well. In particu-lar, it is useful to examine the types of audiences that are featured in oral performance tasks. Of course, in all speaking tests students will be aware of the examiner as an ultimate audience. However, in the majority of instruments reviewed, the examiner is the sole audience to whom students speak. It should be noted that speak-ers do not typically communicate in order that their oral profi ciency be evaluated. Indeed, evaluation usually inhibits communication. To the extent that assessment procedures offer no pretense for speaking other than evaluation, these procedures may yield somewhat inac-curate samples of communication performance.

A single examiner-audience is most natural in inter-view situations. One pitfall of interview situations is that the interviewer may exert overriding infl uence on students’ speech behaviors, resulting in considerable unreliability. A single examiner-audience is most anom-alous in those situations in which students are called upon to deliver a speech to that individual; we do not usually deliver formal speeches to an audience of one. However, this type of testing situation has the advan-tage of not being confounded by audience reaction. The problem of unnatural audiences is somewhat relieved by procedures that simulate situations involving real-istic speaker/audience relations. These procedures may ask students to simulate an emergency telephone call to a police operator, give directions to a stranger, or per-suade a friend to grant some favor. These simulation tasks, however, may confuse speech profi ciency with role-playing ability. A balance must be struck between

the confl icting needs of internal validity (testing what you think you are testing) and external generalizability (to real situations).

Listening. The content of listening assessment instru-ments and procedures is as varied as that of speaking tests. Listening is not a unitary skill, but it is rather a complex of sub skills, each of which is brought into play to greater or lesser degrees depending on the na-ture of the listening task (Lundsteen, 1979). It is natu-ral, therefore, that tests of listening ability tap a variety of skills. Test users should make sure that the selected listening test conforms to their particular measurement objectives.

Most often, listening tests measure literal comprehen-sion of spoken material. It should be noted that compre-hension is generally confused with recall or retention, because questions typically follow some extended dis-course. Some testing methods alleviate this confusion by providing a verbal context that may lessen reliance on memory. Similarly, tests that deliberately select brief passages and present few questions for each passage may tax memory to a lesser extent.

Many listening tests focus on listening for directions, a type of purposeful listening that is readily measured by accuracy of behavioral response (e.g., circling the correct item, drawing the proper path on a map). Other listening skills frequently measured include recogni-tion of speaker’s purpose, or making inferences or in-terpretations beyond material given. Other listening instruments include subtests refl ecting ability to inter-pret paralinguistic cues and also ability to render social judgments from speech. Some sub skills such as vo-cabulary, syntax, and phoneme recognition are critical to communicative listening but are so narrow that they might better be considered receptive language or decod-ing skills. Phonemic discrimination and identifi cation are viewed as essential for reading readiness but should not be construed as measures of listening ability.

Outside of the classroom, the bulk of listening activ-ity takes place in the course of interaction. When refer-ential communication tasks assessing skills of both the speakers and the listeners permit free oral interchange, these tests approximate interactive communication. Other instruments measure interactive listening skill more indirectly by including conversational speech among their listening passages.

Criteria of Assessment Instruments

Another issue related to the content of assessment in-struments that requires examination is their evaluation



criteria. Along what dimensions of quality do the instru-ments render judgments? Listening tests are primarily concerned with accuracy of recall, of following direc-tions, and of perceptions about social relationships. Conceivably, tests could be devised that provide in-formation about listening activity as well. Such instru-ments would indicate what type of listening (critical, aesthetic, informative) students are engaging in during the course of a stimulus passage, and the degree of con-centration or constructive assimilation that character-izes their listening processes.

Speech assessment procedures exhibit a fair amount of consistency in their evaluation criteria. Typically speech rating scales refl ect only three clusters of judg-ment, despite the fact that they may include a larger number of variously labeled criteria. These clusters typically are content, delivery, and language. These criteria, with the addition of organization, account for most speech performance rating scales reviewed in this publication. Despite similarities in criteria, rat-ing scales differ in the weight accorded each criterion. For example, instruments may vary in their treatment of language. Some instruments may weight language most heavily of the criteria, whereas others apportion emphasis more equally among all dimensions of speech evaluation. The defi nition of language quality adopted by some instruments stresses conformity to the conven-tions of standard American English. Other instruments, particularly those designed for nonnative speakers, con-vey more detailed information about the types of gram-matical structures mastered.

Just as some listening tests were characterized as nar-row tests of receptive language, so some speaking tests are measures of productive language, not of commu-nication. This is certainly true of procedures that ask students to imitate words or sentences in isolation, and then apply criteria that evaluate articulation or gram-matical interference of a fi rst language. It is no less true of procedures that incorporate some communicative context like an interview, and then rate speakers on ex-clusively linguistic grounds. Merely eliciting language by means of a communicative task does not constitute a test of communication competence.

A few speaking instruments that emphasize lan-guage quality criteria refl ect the contextual and inter-active aspects of communication better than many of the more conventional rating scales. These instruments measure the degree to which language is appropriate for or adapted to the demands of the communication task. For example, ratings of a response may depend on

the type of question asked. Or a test may measure the degree of elaboration, not just simple labeling, that is expected in a response to a narrative task. Rating scale items may express communication-oriented criteria like “appropriateness” or “intelligibility” rather than formal linguistic properties like sentence structure, standard usage, or correct pronunciation.

Types of Responses and Scoring Procedures Used in Assessment Instruments

Multiple-choice formats are the stock-in-trade of standardized testing. Questions are designed so that each has a single correct answer; tests can be graded easily by machine or template without any problems of unreliability in scoring. Item diffi culty is readily ascer-tained and controlled, and test forms can be equated by well-established methods. Some indirect tests of speak-ing ability attempt to utilize multiple-choice responses, but the technique is more widely represented among tests of listening profi ciency. Multiple-choice questions can be used not only to measure literal comprehension but also to assess higher order abilities like recognition of speaker’s purpose and inference-making, and aspects of critical listening. One of the drawbacks of many multiple-choice listening tests, however, is that students must read printed questions and response alternatives, thus confounding listening ability with reading abil-ity. Some listening tests combat this problem by using tape-recorded presentations of questions and response options. Others use pictures instead of verbal response options. Another technique employed in some measures of listening skill is behavioral response. In particular, this type of performance measure is used in tasks in-volving directions. In general, these tasks approximate normal listening activity, and thus they are more valid than less direct measures.

The most common means for assessing speaking skills are performance rating scales. Stiggins, Back-lund, and Bridgeford (1985) discussed a number of fac-tors pertaining to the use of this technique in large scale assessment. Their major disadvantages lie in the poten-tial for unreliable scoring and in the relatively large ex-penditures of staff time. Some systems seek to avoid the costs of external raters by having classroom teachers evaluate students’ typical or elicited speech. This ap-proach would seem to exacerbate the problem of rating error, although with effective training, rater error can be minimized. In addition to using performance testing, alternatives in assessing speaking ability are techniques



criteria. Along what dimensions of quality do the instru-ments render judgments? Listening tests are primarily concerned with accuracy of recall, of following direc-tions, and of perceptions about social relationships. Conceivably, tests could be devised that provide in-formation about listening activity as well. Such instru-ments would indicate what type of listening (critical, aesthetic, informative) students are engaging in during the course of a stimulus passage, and the degree of con-centration or constructive assimilation that character-izes their listening processes.

Speech assessment procedures exhibit a fair amount of consistency in their evaluation criteria. Typically speech rating scales refl ect only three clusters of judg-ment, despite the fact that they may include a larger number of variously labeled criteria. These clusters typically are content, delivery, and language. These criteria, with the addition of organization, account for most speech performance rating scales reviewed in this publication. Despite similarities in criteria, rat-ing scales differ in the weight accorded each criterion. For example, instruments may vary in their treatment of language. Some instruments may weight language most heavily of the criteria, whereas others apportion emphasis more equally among all dimensions of speech evaluation. The defi nition of language quality adopted by some instruments stresses conformity to the conven-tions of standard American English. Other instruments, particularly those designed for nonnative speakers, con-vey more detailed information about the types of gram-matical structures mastered.

Just as some listening tests were characterized as nar-row tests of receptive language, so some speaking tests are measures of productive language, not of commu-nication. This is certainly true of procedures that ask students to imitate words or sentences in isolation, and then apply criteria that evaluate articulation or gram-matical interference of a fi rst language. It is no less true of procedures that incorporate some communicative context like an interview, and then rate speakers on ex-clusively linguistic grounds. Merely eliciting language by means of a communicative task does not constitute a test of communication competence.

A few speaking instruments that emphasize lan-guage quality criteria refl ect the contextual and inter-active aspects of communication better than many of the more conventional rating scales. These instruments measure the degree to which language is appropriate for or adapted to the demands of the communication task. For example, ratings of a response may depend on

the type of question asked. Or a test may measure the degree of elaboration, not just simple labeling, that is expected in a response to a narrative task. Rating scale items may express communication-oriented criteria like “appropriateness” or “intelligibility” rather than formal linguistic properties like sentence structure, standard usage, or correct pronunciation.

Types of Responses and Scoring Procedures Used in Assessment Instruments

Multiple-choice formats are the stock-in-trade of standardized testing. Questions are designed so that each has a single correct answer; tests can be graded easily by machine or template without any problems of unreliability in scoring. Item diffi culty is readily ascer-tained and controlled, and test forms can be equated by well-established methods. Some indirect tests of speak-ing ability attempt to utilize multiple-choice responses, but the technique is more widely represented among tests of listening profi ciency. Multiple-choice questions can be used not only to measure literal comprehension but also to assess higher order abilities like recognition of speaker’s purpose and inference-making, and aspects of critical listening. One of the drawbacks of many multiple-choice listening tests, however, is that students must read printed questions and response alternatives, thus confounding listening ability with reading abil-ity. Some listening tests combat this problem by using tape-recorded presentations of questions and response options. Others use pictures instead of verbal response options. Another technique employed in some measures of listening skill is behavioral response. In particular, this type of performance measure is used in tasks in-volving directions. In general, these tasks approximate normal listening activity, and thus they are more valid than less direct measures.

The most common means for assessing speaking skills are performance rating scales. Stiggins, Back-lund, and Bridgeford (1985) discussed a number of fac-tors pertaining to the use of this technique in large scale assessment. Their major disadvantages lie in the poten-tial for unreliable scoring and in the relatively large ex-penditures of staff time. Some systems seek to avoid the costs of external raters by having classroom teachers evaluate students’ typical or elicited speech. This ap-proach would seem to exacerbate the problem of rating error, although with effective training, rater error can be minimized. In addition to using performance testing, alternatives in assessing speaking ability are techniques



that use particular discourse features as indicators of quality of expression. These alternatives include the use of measures of syntactic complexity for assessing oral profi ciency and the use of total number of words uttered, lexical diversity, articulation, and sentence ex-pansion to measure linguistic features.

When speaking tasks are structured in a way that per-mits objective measurement of success, it is possible to derive measures of communication effectiveness. For example, it is possible to use “shift of opinion ballots,” which ask audience members to indicate their attitudes toward a topic both before and after the delivery of a persuasive speech to measure the effectiveness of the speaker. Referential communication tasks (Dickson & Patterson, 1979; Yule, 1997) measure communication effectiveness of a speaker by seeing whether a listener can identify the correct object from an array based on the speaker’s description of the object. Effectiveness of small group communication can be evaluated by assign-ing a problem-solving task to a group and then record-ing the accuracy and speed with which the group solves the problem.

These techniques, however, do not elicit uncontami-nated measures of individual communication compe-tence, because audience characteristics, listener skill, and group composition are factors beyond the control of the speaker and can affect communication success. The effectiveness of some referential communication tasks, however, can be assessed without recourse to measur-ing listener accuracy. For example, some tasks require the speaker to state the attributes of an object or geo-metric fi gure that uniquely describe it. Communication effectiveness is evaluated simply by counting the num-ber of critical features that the speaker identifi es (Piche, Rubin, & Turner, 1980).

Administrative Feasibility of Assessment Instruments

If measures of oral communication competency are to be adopted for large scale assessment programs, they must be administratively feasible. Such feasibility considers issues related to amount of student time, rea-sonable allocation of personnel for administration and scoring, and the need for specialized training for admin-istration, scoring, and interpretation. Communication is a complex, interactive behavior. Oral communication is a dynamic and ephemeral process, leaving behind no permanent trace. Therefore, tests of communication competence, speaking particularly, are apt to be more expensive in terms of labor and money than many other

large scale assessment procedures. Administrative sup-port for such assessment procedures could be an indi-cator of those administrators’ commitment to students’ oral competency.

Many tests of listening ability are, however, amena-ble to group administration. Even skill at following di-rections can be assessed in this manner. Tape-recorded administration instructions and response options not only reduce unreliability and confounding with reading ability, but they also contribute to ease of testing. Some listening measures, on the other hand, allow for a wider range of response modes, and these require individual administration.

Tests of speaking skill conducted as interviews or as extended monologues naturally demand individual ad-ministration. Moreover, it is advisable to use multiple raters to insure reliability. This can be accomplished by assigning two staff members for “live” rating or by tape-recording performances for subsequent evaluation by two raters. One other approach attempts to reduce the testing burden by requiring classroom teachers to screen their students based on their typical classroom communication behavior and to refer only those stu-dents “in question” for individual assessment. As al-ready suggested, another approach is to train a group of raters to a high level of reliability and allow them to rate individually.

It is possible to reduce administration costs by using group communication tasks, since a number of students can be evaluated during the course of, say, a 20-minute discussion session. Similarly, referential communica-tion tasks may also be adapted to simultaneous admin-istration to several dyads. The least practical but per-haps the most valuable methods of oral examination are those that require transcription and analysis of speech samples by multiple raters. Summarily, feasibility is-sues should involve consideration of the NCA criteria that suggest that for oral communication skills to be as-sessed, the student must either speak or listen (NCA, 1993).

Target Populations and Potential Sources of Bias

The instruments reviewed in Part II are intended for the K- 12 grade range, although the elementary grades receive particular emphasis, especially among commer-cially-developed instruments. The instruments reviewed in Part III are intended for all ages at two- and four-year academic institutions, and also include instruments cat-egorized as being for adults. In choosing among these



that use particular discourse features as indicators of quality of expression. These alternatives include the use of measures of syntactic complexity for assessing oral profi ciency and the use of total number of words uttered, lexical diversity, articulation, and sentence ex-pansion to measure linguistic features.

When speaking tasks are structured in a way that per-mits objective measurement of success, it is possible to derive measures of communication effectiveness. For example, it is possible to use “shift of opinion ballots,” which ask audience members to indicate their attitudes toward a topic both before and after the delivery of a persuasive speech to measure the effectiveness of the speaker. Referential communication tasks (Dickson & Patterson, 1979; Yule, 1997) measure communication effectiveness of a speaker by seeing whether a listener can identify the correct object from an array based on the speaker’s description of the object. Effectiveness of small group communication can be evaluated by assign-ing a problem-solving task to a group and then record-ing the accuracy and speed with which the group solves the problem.

These techniques, however, do not elicit uncontami-nated measures of individual communication compe-tence, because audience characteristics, listener skill, and group composition are factors beyond the control of the speaker and can affect communication success. The effectiveness of some referential communication tasks, however, can be assessed without recourse to measur-ing listener accuracy. For example, some tasks require the speaker to state the attributes of an object or geo-metric fi gure that uniquely describe it. Communication effectiveness is evaluated simply by counting the num-ber of critical features that the speaker identifi es (Piche, Rubin, & Turner, 1980).

Administrative Feasibility of Assessment Instruments

If measures of oral communication competency are to be adopted for large scale assessment programs, they must be administratively feasible. Such feasibility considers issues related to amount of student time, rea-sonable allocation of personnel for administration and scoring, and the need for specialized training for admin-istration, scoring, and interpretation. Communication is a complex, interactive behavior. Oral communication is a dynamic and ephemeral process, leaving behind no permanent trace. Therefore, tests of communication competence, speaking particularly, are apt to be more expensive in terms of labor and money than many other

large scale assessment procedures. Administrative sup-port for such assessment procedures could be an indi-cator of those administrators’ commitment to students’ oral competency.

Many tests of listening ability are, however, amena-ble to group administration. Even skill at following di-rections can be assessed in this manner. Tape-recorded administration instructions and response options not only reduce unreliability and confounding with reading ability, but they also contribute to ease of testing. Some listening measures, on the other hand, allow for a wider range of response modes, and these require individual administration.

Tests of speaking skill conducted as interviews or as extended monologues naturally demand individual ad-ministration. Moreover, it is advisable to use multiple raters to insure reliability. This can be accomplished by assigning two staff members for “live” rating or by tape-recording performances for subsequent evaluation by two raters. One other approach attempts to reduce the testing burden by requiring classroom teachers to screen their students based on their typical classroom communication behavior and to refer only those stu-dents “in question” for individual assessment. As al-ready suggested, another approach is to train a group of raters to a high level of reliability and allow them to rate individually.

It is possible to reduce administration costs by using group communication tasks, since a number of students can be evaluated during the course of, say, a 20-minute discussion session. Similarly, referential communica-tion tasks may also be adapted to simultaneous admin-istration to several dyads. The least practical but per-haps the most valuable methods of oral examination are those that require transcription and analysis of speech samples by multiple raters. Summarily, feasibility is-sues should involve consideration of the NCA criteria that suggest that for oral communication skills to be as-sessed, the student must either speak or listen (NCA, 1993).

Target Populations and Potential Sources of Bias

The instruments reviewed in Part II are intended for the K- 12 grade range, although the elementary grades receive particular emphasis, especially among commer-cially-developed instruments. The instruments reviewed in Part III are intended for all ages at two- and four-year academic institutions, and also include instruments cat-egorized as being for adults. In choosing among these



instruments, the user should be aware of potential bi-ases in the testing procedure. Instruments vary consid-erably in their efforts to minimize group bias effects. Some technical manuals document the work of minor-ity group reviewers who examined items in order to eliminate potential bias. It should be noted, however, that differences in central tendency are not, themselves, evidence of test bias. Rather, a test is biased if it over- or under-predicts scores on some independently adminis-tered criterion measure. If an instrument contained in Part II or Part III has been tested in any way for any form of bias, that test is reported in the review. In the fi nal analysis, the user or assessor should be sensitive to whether an instrument operates with bias relative to the particular target population being assessed.

Locally Developed Assessment Instruments

Developing instruments locally for assessing any di-mension of oral communication requires considerable time and effort as well as familiarity with measurement and content concerns. Often it is not feasible to submit locally developed instruments to the same degree of tech-nical review for reliability or validity as commercially de-veloped instruments. However, there are some situations where local development of assessment instruments is desirable. For example, a school district may adopt a set of specifi c speaking and listening competencies and de-velop an instructional program directed toward building those competencies. In order to measure its success, the district may fi nd that it is better to develop a test locally that is tailored to its specifi c competencies than to use existing tests that only measure some of those competen-cies or that only measure those competencies indirectly. The following brief step-by-step descriptions of the de-velopment process provide direction to local agencies that wish to develop their own instruments. Those agen-cies and assessors should be advised that these sugges-tions may also be used in developing assessment instru-ments and processes for areas of communication other than speaking and listening.

Speaking

In determining the speaking skills and types of speak-ing tasks that are important, local developers must fi rst defi ne the types of speaking skills and tasks students should be able to perform. In developing this list, devel-opers will fi nd it helpful to review curricular objectives, instructional materials, and teaching practices. They should involve a full range of people concerned with the

results of the assessment process, (e.g., teachers, curric-ulum specialists, administrators, parents, and students). The resulting list may focus on specifi c skills important in all speaking situations (e.g., speaking distinctly or speaking in an organized fashion). The list may also fo-cus on specifi c tasks that are considered important (e.g., giving directions, giving a speech, or asking questions). It is critical that the speaking skills and tasks listed be as specifi c as possible, so that they may be observed and measured.

Two types of approaches are used in assessing speak-ing behaviors. First, in an observational approach, the student’s behavior may be observed and assessed unob-trusively. Second, in a structured assessment approach, the student may be asked to perform one or more struc-tured speaking tasks, and his or her performance on the tasks is then assessed.

If an observational approach is taken, the developer must decide what speaking behaviors will be observed (e.g., asking questions, responding to questions, or speaking in group discussions). In addition, the devel-oper needs to decide how many times each student will be observed, and for how long. The observer may be the regular classroom teacher, or someone from outside the classroom, such as a teacher from another grade level, a chairperson, or a counselor. It should be noted that op-timal evaluation can only be provided by a well-trained speech communication professional.

If a structured approach is adopted, the developer must decide what type of speaking tasks will be used. Also, the developer must decide on the setting for the tasks. The student might be asked to perform certain tasks in front of the entire class, in a small group setting, or in a one-on-one situation with the assessor. Again, the assessor may be the classroom teacher or some-one from outside the classroom. But again, training in speech communication will contribute to optimal evalu-ation.

Next, a scoring system that describes acceptable and unacceptable levels of performance for the speaking skills or tasks already identifi ed in the fi rst step must be developed. The scoring system may involve a two-point determination (e.g., the behavior of interest is either present or absent, or the student can be heard or cannot be heard). Alternatively, the scoring system may defi ne a continuum of behaviors that ranges from lowest to highest (e.g., while speaking, the student is much dis-organized, somewhat disorganized, fairly organized, or very organized. However, when a continuum is used, it is necessary to describe each level of the scale in terms



instruments, the user should be aware of potential bi-ases in the testing procedure. Instruments vary consid-erably in their efforts to minimize group bias effects. Some technical manuals document the work of minor-ity group reviewers who examined items in order to eliminate potential bias. It should be noted, however, that differences in central tendency are not, themselves, evidence of test bias. Rather, a test is biased if it over- or under-predicts scores on some independently adminis-tered criterion measure. If an instrument contained in Part II or Part III has been tested in any way for any form of bias, that test is reported in the review. In the fi nal analysis, the user or assessor should be sensitive to whether an instrument operates with bias relative to the particular target population being assessed.

Locally Developed Assessment Instruments

Developing instruments locally for assessing any di-mension of oral communication requires considerable time and effort as well as familiarity with measurement and content concerns. Often it is not feasible to submit locally developed instruments to the same degree of tech-nical review for reliability or validity as commercially de-veloped instruments. However, there are some situations where local development of assessment instruments is desirable. For example, a school district may adopt a set of specifi c speaking and listening competencies and de-velop an instructional program directed toward building those competencies. In order to measure its success, the district may fi nd that it is better to develop a test locally that is tailored to its specifi c competencies than to use existing tests that only measure some of those competen-cies or that only measure those competencies indirectly. The following brief step-by-step descriptions of the de-velopment process provide direction to local agencies that wish to develop their own instruments. Those agen-cies and assessors should be advised that these sugges-tions may also be used in developing assessment instru-ments and processes for areas of communication other than speaking and listening.

Speaking

In determining the speaking skills and types of speak-ing tasks that are important, local developers must fi rst defi ne the types of speaking skills and tasks students should be able to perform. In developing this list, devel-opers will fi nd it helpful to review curricular objectives, instructional materials, and teaching practices. They should involve a full range of people concerned with the

results of the assessment process, (e.g., teachers, curric-ulum specialists, administrators, parents, and students). The resulting list may focus on specifi c skills important in all speaking situations (e.g., speaking distinctly or speaking in an organized fashion). The list may also fo-cus on specifi c tasks that are considered important (e.g., giving directions, giving a speech, or asking questions). It is critical that the speaking skills and tasks listed be as specifi c as possible, so that they may be observed and measured.

Two types of approaches are used in assessing speak-ing behaviors. First, in an observational approach, the student’s behavior may be observed and assessed unob-trusively. Second, in a structured assessment approach, the student may be asked to perform one or more struc-tured speaking tasks, and his or her performance on the tasks is then assessed.

If an observational approach is taken, the developer must decide what speaking behaviors will be observed (e.g., asking questions, responding to questions, or speaking in group discussions). In addition, the devel-oper needs to decide how many times each student will be observed, and for how long. The observer may be the regular classroom teacher, or someone from outside the classroom, such as a teacher from another grade level, a chairperson, or a counselor. It should be noted that op-timal evaluation can only be provided by a well-trained speech communication professional.

If a structured approach is adopted, the developer must decide what type of speaking tasks will be used. Also, the developer must decide on the setting for the tasks. The student might be asked to perform certain tasks in front of the entire class, in a small group setting, or in a one-on-one situation with the assessor. Again, the assessor may be the classroom teacher or some-one from outside the classroom. But again, training in speech communication will contribute to optimal evalu-ation.

Next, a scoring system that describes acceptable and unacceptable levels of performance for the speaking skills or tasks already identifi ed in the fi rst step must be developed. The scoring system may involve a two-point determination (e.g., the behavior of interest is either present or absent, or the student can be heard or cannot be heard). Alternatively, the scoring system may defi ne a continuum of behaviors that ranges from lowest to highest (e.g., while speaking, the student is much dis-organized, somewhat disorganized, fairly organized, or very organized. However, when a continuum is used, it is necessary to describe each level of the scale in terms



of specifi c behaviors that represent that point in the scale. The resulting scoring system will be used either for observation ratings or structured ratings, as deter-mined previously in the second step.

Once the basic approach is established and the scor-ing system is developed, it is necessary to train raters in the use of the system. Training should include thorough instruction in the categories in the scoring system, pro-vision of examples of performance that represent the various categories, and opportunities for the raters to practice rating student performance. Raters should have ample opportunity to ask questions about the categories and discuss their practice ratings.

Often the training will lead to alterations in the scor-ing system. It is possible that some initial distinctions made in the scoring system will prove impossible to ob-serve in actual performance. Once the system is fi nal-ized and raters are comfortable in their abilities to make ratings, raters should be tested for interrater reliability. They should be given several samples of performance and asked to rate them without discussion. The degree of agreement among the raters is a measure of interrater reliability. Raters should also be trained in test adminis-tration procedures.

The fi nal steps in the assessment process are data collection and scoring. These activities may happen si-multaneously or in stages. Ratings may be made on the spot, or the speaking performance of students may be audiotaped or videotaped and scored at a later time. The advantage of recording performance is that it allows for scoring in a more controlled environment.

In addition to testing interrater reliability at the end of training, the reliability of the ratings also should be checked during the assessment period. While effectively trained raters can conduct reliable assessments individ-ually, periodic multiple rater assessment should be used to check interrater reliability. Checking the reliability does not have to occur for every rating, but it should be conducted at random for a reasonable percentage of the ratings.

Listening

To determine the knowledge of the listening process, the listening skills, and types of listening tasks that are important, local developers may begin by defi ning the types of listening knowledge the students should know and the types of listening skills and tasks that students should be able to perform. The steps in this process are the same as they are for speaking. The resulting list may focus on skills that are important to all listening situa-

tions (e.g., understanding main ideas and details). These skills may be similar to reading comprehension skills. The list may also focus on specifi c listening tasks that are considered important (e.g., listening to directions, listening to a lecture, or listening on the telephone). As with speaking, it is critical that the knowledge, skills, and tasks listed be as specifi c as possible so that they may be objectively measured.

The next step in developing listening assessment in-struments is to assemble stimuli that the students will listen to in the assessment. These stimuli should refl ect the listening tasks identifi ed in the fi rst step. Listening material may be drawn from existing sources. Natural listening material such as public service announce-ments, commercials, or news stories make particularly good material. It is also possible to write material that particularly refl ects the tasks identifi ed in the fi rst step. Care should be taken to use material that is relatively short, is interesting to students, and does not refl ect a bias toward a particular gender, racial/ethnic, socioeco-nomic, or geographic group.

The actual production of stimulus material may take two forms. The material may be written in script form so that it may be read aloud by the test administrator, or it may be recorded on audiotape or videotape. The advantage of taped materials is that they guarantee standard administration and allow for variety in stimu-lus material, such as various voices, conversations, or sound effects.

Several possible types of listening items may be de-veloped. The most typical is multiple-choice items that ask a question about the listening stimulus and provide several possible response options. Another type is short-answer items that ask a question and require the student to write or give orally a short response. A third type, used for following direction tasks, presents graphic ma-terial, such as a map, and asks the student to complete a certain task, such as drawing a route onto the map. A variation of this listening item is to describe an object and ask the student to draw the object or to select the appropriate object from a set of pictures. In all cases, item development should follow established item con-struction standards that may be found in measurement textbooks.

It is impossible to identify all the possible confusing or problematic aspects of stimulus materials or items until they have been fi eld tested with a sample of stu-dents who are similar to those who will be assessed. The results of fi eld testing may be used to pick the best stimuli and items. Measurement textbooks provide



of specifi c behaviors that represent that point in the scale. The resulting scoring system will be used either for observation ratings or structured ratings, as deter-mined previously in the second step.

Once the basic approach is established and the scor-ing system is developed, it is necessary to train raters in the use of the system. Training should include thorough instruction in the categories in the scoring system, pro-vision of examples of performance that represent the various categories, and opportunities for the raters to practice rating student performance. Raters should have ample opportunity to ask questions about the categories and discuss their practice ratings.

Often the training will lead to alterations in the scor-ing system. It is possible that some initial distinctions made in the scoring system will prove impossible to ob-serve in actual performance. Once the system is fi nal-ized and raters are comfortable in their abilities to make ratings, raters should be tested for interrater reliability. They should be given several samples of performance and asked to rate them without discussion. The degree of agreement among the raters is a measure of interrater reliability. Raters should also be trained in test adminis-tration procedures.

The fi nal steps in the assessment process are data collection and scoring. These activities may happen si-multaneously or in stages. Ratings may be made on the spot, or the speaking performance of students may be audiotaped or videotaped and scored at a later time. The advantage of recording performance is that it allows for scoring in a more controlled environment.

In addition to testing interrater reliability at the end of training, the reliability of the ratings also should be checked during the assessment period. While effectively trained raters can conduct reliable assessments individ-ually, periodic multiple rater assessment should be used to check interrater reliability. Checking the reliability does not have to occur for every rating, but it should be conducted at random for a reasonable percentage of the ratings.

Listening

To determine the knowledge of the listening process, the listening skills, and types of listening tasks that are important, local developers may begin by defi ning the types of listening knowledge the students should know and the types of listening skills and tasks that students should be able to perform. The steps in this process are the same as they are for speaking. The resulting list may focus on skills that are important to all listening situa-

tions (e.g., understanding main ideas and details). These skills may be similar to reading comprehension skills. The list may also focus on specifi c listening tasks that are considered important (e.g., listening to directions, listening to a lecture, or listening on the telephone). As with speaking, it is critical that the knowledge, skills, and tasks listed be as specifi c as possible so that they may be objectively measured.

The next step in developing listening assessment in-struments is to assemble stimuli that the students will listen to in the assessment. These stimuli should refl ect the listening tasks identifi ed in the fi rst step. Listening material may be drawn from existing sources. Natural listening material such as public service announce-ments, commercials, or news stories make particularly good material. It is also possible to write material that particularly refl ects the tasks identifi ed in the fi rst step. Care should be taken to use material that is relatively short, is interesting to students, and does not refl ect a bias toward a particular gender, racial/ethnic, socioeco-nomic, or geographic group.

The actual production of stimulus material may take two forms. The material may be written in script form so that it may be read aloud by the test administrator, or it may be recorded on audiotape or videotape. The advantage of taped materials is that they guarantee standard administration and allow for variety in stimu-lus material, such as various voices, conversations, or sound effects.

Several possible types of listening items may be de-veloped. The most typical is multiple-choice items that ask a question about the listening stimulus and provide several possible response options. Another type is short-answer items that ask a question and require the student to write or give orally a short response. A third type, used for following direction tasks, presents graphic ma-terial, such as a map, and asks the student to complete a certain task, such as drawing a route onto the map. A variation of this listening item is to describe an object and ask the student to draw the object or to select the appropriate object from a set of pictures. In all cases, item development should follow established item con-struction standards that may be found in measurement textbooks.

It is impossible to identify all the possible confusing or problematic aspects of stimulus materials or items until they have been fi eld tested with a sample of stu-dents who are similar to those who will be assessed. The results of fi eld testing may be used to pick the best stimuli and items. Measurement textbooks provide



some simple techniques for reviewing fi eld test data. In addition, fi eld testing provides information about the amount of time it takes most students to complete the items. This information should be used to establish the time limits for the fi nalized test.

Alternative Assessment and Communication

The fi eld of assessment is undergoing exciting fer-ment. Public schools as well as institutions of higher education are espousing goals such as “value-added education,” “outcome-based education,” and “national standards.” Each of these goals is intimately bound up with evaluation. If we are to articulate the standards that we wish our students and teachers to exceed, then we must also devise the means by which we can recognize when those standards are met. We are entering an epoch in which assessment is at the forefront of educational reform.

At the same time, however, many educators are be-ginning to regard traditional psychometrically-driven testing as simply not up to the job of describing stu-dent growth and achievement in complex domains such as written expression or mathematical problem solv-ing (Johnston, 1992; Wolf, Bixby, Glenn, & Gardner, 1991). Instead of standardized multiple-choice testing, the frontier of assessment is allied with practices like portfolio assessment, learning logs, and exhibitions. Herman, Aschbacher, and Winters (1992, p. 13) sum-marized these recent trends in assessment as involving fi ve kinds of reforms:

1) Changes from behavioral to cognitive views of learning and assessment;

2) From paper-pencil to authentic assessment;3) From single-occasion assessment to samples over

time;4) From single attribute to multidimensional assess-

ment;5) From near-exclusive emphasis on individual as-

sessment to group assessment.

In virtually every case, alternative or “authentic” as-sessment programs involve performance-based evalua-tions. They may entail students directly demonstrating some task, or they may ask students to collect artifacts (in a portfolio) that indirectly document task perfor-mance. And these new types of assessment tasks ask learners to refl ect on their performance and to commu-nicate those refl ections. Thus in a typical writing port-folio assessment, students choose some subset of the

writings they have produced over a period of time. Then they may write a “biography” of each piece, explaining its genesis and the process by which they composed it. Or they may answer a series of questions inquiring why they regard each piece as important to their own devel-opment as writers.

Communication assessment has long been perfor-mance-based. It has considerable expertise to contrib-ute to the present movement for alternative assessment. For example, public speaking assessment has continu-ally wrestled with the issue of preparation for a per-formance test. Should students be provided speech top-ics ahead of time so that they can prepare in a realistic fashion, or do impromptu speaking tasks provide more accurate estimates of individuals’ abilities? In addition, the experience of oral communication assessment can provide insight regarding ways to evaluate collaborative group processes, as well as optimal ways to conduct in-terviews about work products.

In exchange, the practice of oral communication as-sessment can be enhanced by adapting aspects of al-ternative assessment (Hay, 1994). Too many public speaking assessment procedures demand that students engage in “dummy run” extended monologues, rather than authentic communication. Few communication as-sessment procedures allow for sampling multiple per-formances across time; they rely instead on a single performance of unknown test-retest stability. Few oral communication assessment programs have been de-veloped to exploit the power of portfolios to promote learner refl ection. These remain as challenges for those involved in the assessment of oral communication com-petency.

Before departing from the topic of alternative assess-ment, it is worth remarking that the communication as-sessment movement need not – indeed must not – aban-don the pursuit of rigorous psychometric adequacy for other kinds of measurement strategies. Though some proponents of alternative testing do sometimes seem philosophically opposed to reporting reliability coef-fi cients that ought not to be the approach taken when assessing oral communication. Conventional indices of psychometric adequacy are discussed in Part I of this book, and it is crucial for the continued credibility of this enterprise that assessment specialists remain mindful of such indicators. There may be times, however (e.g., high stakes testing), when one sacrifi ces a degree of authen-ticity in exchange for a degree of reliability. Conversely, there may be times (e.g., program evaluation) when one trades off some internal consistency in exchange for dif-



some simple techniques for reviewing fi eld test data. In addition, fi eld testing provides information about the amount of time it takes most students to complete the items. This information should be used to establish the time limits for the fi nalized test.

Alternative Assessment and Communication

The fi eld of assessment is undergoing exciting fer-ment. Public schools as well as institutions of higher education are espousing goals such as “value-added education,” “outcome-based education,” and “national standards.” Each of these goals is intimately bound up with evaluation. If we are to articulate the standards that we wish our students and teachers to exceed, then we must also devise the means by which we can recognize when those standards are met. We are entering an epoch in which assessment is at the forefront of educational reform.

At the same time, however, many educators are be-ginning to regard traditional psychometrically-driven testing as simply not up to the job of describing stu-dent growth and achievement in complex domains such as written expression or mathematical problem solv-ing (Johnston, 1992; Wolf, Bixby, Glenn, & Gardner, 1991). Instead of standardized multiple-choice testing, the frontier of assessment is allied with practices like portfolio assessment, learning logs, and exhibitions. Herman, Aschbacher, and Winters (1992, p. 13) sum-marized these recent trends in assessment as involving fi ve kinds of reforms:

1) Changes from behavioral to cognitive views of learning and assessment;

2) From paper-pencil to authentic assessment;3) From single-occasion assessment to samples over

time;4) From single attribute to multidimensional assess-

ment;5) From near-exclusive emphasis on individual as-

sessment to group assessment.

In virtually every case, alternative or “authentic” as-sessment programs involve performance-based evalua-tions. They may entail students directly demonstrating some task, or they may ask students to collect artifacts (in a portfolio) that indirectly document task perfor-mance. And these new types of assessment tasks ask learners to refl ect on their performance and to commu-nicate those refl ections. Thus in a typical writing port-folio assessment, students choose some subset of the

writings they have produced over a period of time. Then they may write a “biography” of each piece, explaining its genesis and the process by which they composed it. Or they may answer a series of questions inquiring why they regard each piece as important to their own devel-opment as writers.

Communication assessment has long been perfor-mance-based. It has considerable expertise to contrib-ute to the present movement for alternative assessment. For example, public speaking assessment has continu-ally wrestled with the issue of preparation for a per-formance test. Should students be provided speech top-ics ahead of time so that they can prepare in a realistic fashion, or do impromptu speaking tasks provide more accurate estimates of individuals’ abilities? In addition, the experience of oral communication assessment can provide insight regarding ways to evaluate collaborative group processes, as well as optimal ways to conduct in-terviews about work products.

In exchange, the practice of oral communication as-sessment can be enhanced by adapting aspects of al-ternative assessment (Hay, 1994). Too many public speaking assessment procedures demand that students engage in “dummy run” extended monologues, rather than authentic communication. Few communication as-sessment procedures allow for sampling multiple per-formances across time; they rely instead on a single performance of unknown test-retest stability. Few oral communication assessment programs have been de-veloped to exploit the power of portfolios to promote learner refl ection. These remain as challenges for those involved in the assessment of oral communication com-petency.

Before departing from the topic of alternative assess-ment, it is worth remarking that the communication as-sessment movement need not – indeed must not – aban-don the pursuit of rigorous psychometric adequacy for other kinds of measurement strategies. Though some proponents of alternative testing do sometimes seem philosophically opposed to reporting reliability coef-fi cients that ought not to be the approach taken when assessing oral communication. Conventional indices of psychometric adequacy are discussed in Part I of this book, and it is crucial for the continued credibility of this enterprise that assessment specialists remain mindful of such indicators. There may be times, however (e.g., high stakes testing), when one sacrifi ces a degree of authen-ticity in exchange for a degree of reliability. Conversely, there may be times (e.g., program evaluation) when one trades off some internal consistency in exchange for dif-



fuse pedagogical impact. But these trade-offs need to be deliberate, and they ought to be considered short-term exigencies. Eventually we ought to be able to invent mea-sures that are at once authentic and reliable. The use of a multiplicity of measurement approaches, some authentic and others more traditional, would yield a comprehen-sive assessment of oral communication.

Final Questions to Ask About Assessment and What Should be Tested

Ideally we should test what we teach or what we ex-pect to impact. But in many cases, adoption of standards for oral communication competency by state and local educational agencies is a new phenomenon. Moreover, even when such competencies are mandated, the extent and fi delity with which they fi lter down to classroom practice is unknown. Therefore, a priority in assess-ment is to continue efforts to determine current class-room practices in speech communication. The former Committee on Assessment and Testing (now the Com-munication Assessment Division of NCA) sponsored a number of such survey efforts in the 1980s, begin-ning with that by Backlund, Booth, Moore, Parks, and VanRheenen, (1982) with periodic updates since then. Not surprisingly, these surveys found that the deliberate teaching of oral communication skills is still neglected in U.S. public schools. Therefore, the content domain of communication assessment probably cannot be de-fi ned by what is taught, but by what ought to be taught. The result would be that the specifi cation of a content domain for testing would exert impact on instruction. That is, the instruments and administrative selection of an oral assessment tool or process can, in some cases, direct what may be included in the curriculum.

As such, the determination of what should be assessed is a pivotal decision. That decision can be informed by determining whether assessment of oral communica-tion should take place with regard to the pedagogical process, impact on students, or any other of a variety of outcomes. Which of these three factors is, or are, of interest to the assessor? Is the process of teaching it-self what should be assessed? Are we concerned with how we are doing what we are doing? Or is impact on students important to assess? Are we concerned with impact as measured by students’ perceptions of a course or perhaps by students’ improvement between pre- and post-scores on a given communication assessment tool? Finally, is some other outcome factor such as retention of students, improved grade point averages, or even job placement of importance to the assessor?

Are the Measures You Choose to Use or Develop Valid?

When selecting an assessment procedure, one criti-cal factor is validity. Does the procedure measure what it claims to measure? Does it relate to and measure purposes and objectives? With the possible exception of the Communication Competency Assessment Instru-ment (Rubin, 1982), existing instruments for testing communication competence have not been subjected to studies of concurrent or predictive validity. In the time period since the publication of the fi rst edition of this book, little work has been done in this area, especially in public education. One reason for this may be the frus-tration that state and local education agencies have ex-perienced in attempting to set up assessment programs. The continued lack of accepted criteria against which tests may be validated constitutes another reason. This likely the primary reason that high-stakes testing has not included oral communication in any consistent way. Some states, such as Washington, include a listening testing component, but none include assessment of oral communication.

Establishing criterion-referenced validity seems par-ticularly crucial in assessment tasks that are obviously contrived solely for the purpose of evaluation. Several assessment procedures require students to communi-cate in role-playing situations. While such procedures permit evaluation of “life role” communication skills, the relationship between role-playing performance and natural communication performance is still relatively unknown. Rubin (1985) and Morreale, Moore, Surges-Tatum, & Webster (in press) have research data that supports the relationship between the testing procedure and natural performance, but more is needed.

Future research and development priorities relating to validity in measures of speaking and listening might include:

• establishing criterion measures for measuring concurrent and predictive validity,

• exploring naturalistic criterion measures for these purposes, or

• investigating criterion-referenced validity of con-trived communication tasks.

Are Assessment Instruments Reliable?

The discipline of communication has increasingly pursued investigations of test reliability since the fi rst edition of this volume was published. As noted in the fi rst edition, some attention had been given to internal



fuse pedagogical impact. But these trade-offs need to be deliberate, and they ought to be considered short-term exigencies. Eventually we ought to be able to invent mea-sures that are at once authentic and reliable. The use of a multiplicity of measurement approaches, some authentic and others more traditional, would yield a comprehen-sive assessment of oral communication.

Final Questions to Ask About Assessment and What Should be Tested

Ideally we should test what we teach or what we ex-pect to impact. But in many cases, adoption of standards for oral communication competency by state and local educational agencies is a new phenomenon. Moreover, even when such competencies are mandated, the extent and fi delity with which they fi lter down to classroom practice is unknown. Therefore, a priority in assess-ment is to continue efforts to determine current class-room practices in speech communication. The former Committee on Assessment and Testing (now the Com-munication Assessment Division of NCA) sponsored a number of such survey efforts in the 1980s, begin-ning with that by Backlund, Booth, Moore, Parks, and VanRheenen, (1982) with periodic updates since then. Not surprisingly, these surveys found that the deliberate teaching of oral communication skills is still neglected in U.S. public schools. Therefore, the content domain of communication assessment probably cannot be de-fi ned by what is taught, but by what ought to be taught. The result would be that the specifi cation of a content domain for testing would exert impact on instruction. That is, the instruments and administrative selection of an oral assessment tool or process can, in some cases, direct what may be included in the curriculum.

As such, the determination of what should be assessed is a pivotal decision. That decision can be informed by determining whether assessment of oral communica-tion should take place with regard to the pedagogical process, impact on students, or any other of a variety of outcomes. Which of these three factors is, or are, of interest to the assessor? Is the process of teaching it-self what should be assessed? Are we concerned with how we are doing what we are doing? Or is impact on students important to assess? Are we concerned with impact as measured by students’ perceptions of a course or perhaps by students’ improvement between pre- and post-scores on a given communication assessment tool? Finally, is some other outcome factor such as retention of students, improved grade point averages, or even job placement of importance to the assessor?

Are the Measures You Choose to Use or Develop Valid?

When selecting an assessment procedure, one criti-cal factor is validity. Does the procedure measure what it claims to measure? Does it relate to and measure purposes and objectives? With the possible exception of the Communication Competency Assessment Instru-ment (Rubin, 1982), existing instruments for testing communication competence have not been subjected to studies of concurrent or predictive validity. In the time period since the publication of the fi rst edition of this book, little work has been done in this area, especially in public education. One reason for this may be the frus-tration that state and local education agencies have ex-perienced in attempting to set up assessment programs. The continued lack of accepted criteria against which tests may be validated constitutes another reason. This likely the primary reason that high-stakes testing has not included oral communication in any consistent way. Some states, such as Washington, include a listening testing component, but none include assessment of oral communication.

Establishing criterion-referenced validity seems par-ticularly crucial in assessment tasks that are obviously contrived solely for the purpose of evaluation. Several assessment procedures require students to communi-cate in role-playing situations. While such procedures permit evaluation of “life role” communication skills, the relationship between role-playing performance and natural communication performance is still relatively unknown. Rubin (1985) and Morreale, Moore, Surges-Tatum, & Webster (in press) have research data that supports the relationship between the testing procedure and natural performance, but more is needed.

Future research and development priorities relating to validity in measures of speaking and listening might include:

• establishing criterion measures for measuring concurrent and predictive validity,

• exploring naturalistic criterion measures for these purposes, or

• investigating criterion-referenced validity of con-trived communication tasks.

Are Assessment Instruments Reliable?

The discipline of communication has increasingly pursued investigations of test reliability since the fi rst edition of this volume was published. As noted in the fi rst edition, some attention had been given to internal



consistency or dimensionality in studies of speech rat-ing scales, and it appears to be common practice for commercial tests of listening skill to report this aspect of reliability. But other types of reliability need to be addressed more extensively. For example, measures of speaking profi ciency (e.g., Rubin, 1982; Morreale, et al. (in press)) offer students a choice of topics, with no ap-parent evidence of equivalence between topics. Also, in-terviewer-assessor idiosyncrasies can alter performance in speaking assessments. Some writers have commented that single samples of speech are not reliable indicators of communication competence, and that several sam-ples ranging over a variety of speech functions and situ-ations should be taken for a fair assessment. At present, though, we lack the sort of precise information concern-ing the requisite size for a reliable sample. And insuf-fi cient information is available concerning test-retest reliability of most speaking and listening assessment instruments. Students do have good days and bad days. Instruction and practice may (ideally) change their skill over time. Both situations may weaken test-retest reli-ability results. Although individual test developers have done considerable work in establishing training proce-dures to support interrater consistency and reliability, these procedures need to be instituted whenever and wherever the assessment of oral communication occurs. Guidelines for choosing appropriate statistical methods to evaluate interrater reliability are included in other NCA (1994) publications.

Future research and development priorities relating to reliability in measures of speaking and listening might include:

• determining the impact of varying topics and communication tasks on the testing process,

• determining the impact of assessor or test admin-istrator on student performances,

• determining the size and diversity of speech sam-ples required for a reliable indication of compe-tence,

• ascertaining test-retest reliabilities of existing in-struments, or

• supporting the institutionalization of methods to enhance inter-rater reliability.

What Measurement Techniques are Presently Available?

Clearly the authors of this volume wish to encourage the development of new measurement techniques or the adaptation of new research methodologies for purposes

of assessing oral communication. However, it is also worthwhile to examine some of the instruments already available. The instruments reviewed in Parts II and III are worth considering for use. In general, these instru-ments range on a continuum from direct (obtrusive) to indirect (unobtrusive) measures. Overall, the most optimal assessment procedures are those that are least intrusive. Naturalistic observation, and even classroom teacher ratings, however, introduce problems of rater bias and problems of consistency.

Indirect tests of speaking ability would alleviate many sources of inconsistency as they assess communication behavior as it naturally occurs within the classroom or context. However, it may be diffi cult to construct such tests. Moreover, indirect tests may be contaminated by extraneous factors like reading ability and “test-wise-ness.” Also, such indirect tests are likely to exert del-eterious “wash back” effects on speech communication instructional practices, and this can lead to a focus on rote knowledge rather than internalized skill.

Direct tests of listening ability present fewer problems involving consistency, particularly when test stimuli are tape-recorded. Even so, measures of listening in conver-sation are elusive. Interactive listening--where the listener is an equal conversational partner who responds and is ever ready to switch into the role of speaker--probably calls on different skills than procedures in which test-takers listen to a tape-recorded conversation, and calls on still other skills from procedures in which test-takers demonstrate their understanding of oral commands.

In tests of speaking ability, use of rating scales pre-dominate. Typically, rating scales are applied to either extended talks or interview situations. It would be use-ful to adapt the use of rating scales to less intrusive, more interactive, communication situations.

Future research and development priorities relating to presently available measurement techniques might include:

• enhancing reliability of naturalistic observation procedures,

• developing more measures of communication in interactive situations, or

• extending performance rating scales to more in-teractive communication.

Are Assessment Instruments Inherently Susceptible to Group and Other Biases?

Consistent group differences in test scores are not, in and of themselves, evidence of test bias. Rather, test



consistency or dimensionality in studies of speech rat-ing scales, and it appears to be common practice for commercial tests of listening skill to report this aspect of reliability. But other types of reliability need to be addressed more extensively. For example, measures of speaking profi ciency (e.g., Rubin, 1982; Morreale, et al. (in press)) offer students a choice of topics, with no ap-parent evidence of equivalence between topics. Also, in-terviewer-assessor idiosyncrasies can alter performance in speaking assessments. Some writers have commented that single samples of speech are not reliable indicators of communication competence, and that several sam-ples ranging over a variety of speech functions and situ-ations should be taken for a fair assessment. At present, though, we lack the sort of precise information concern-ing the requisite size for a reliable sample. And insuf-fi cient information is available concerning test-retest reliability of most speaking and listening assessment instruments. Students do have good days and bad days. Instruction and practice may (ideally) change their skill over time. Both situations may weaken test-retest reli-ability results. Although individual test developers have done considerable work in establishing training proce-dures to support interrater consistency and reliability, these procedures need to be instituted whenever and wherever the assessment of oral communication occurs. Guidelines for choosing appropriate statistical methods to evaluate interrater reliability are included in other NCA (1994) publications.

Future research and development priorities relating to reliability in measures of speaking and listening might include:

• determining the impact of varying topics and communication tasks on the testing process,

• determining the impact of assessor or test admin-istrator on student performances,

• determining the size and diversity of speech sam-ples required for a reliable indication of compe-tence,

• ascertaining test-retest reliabilities of existing in-struments, or

• supporting the institutionalization of methods to enhance inter-rater reliability.

What Measurement Techniques are Presently Available?

Clearly the authors of this volume wish to encourage the development of new measurement techniques or the adaptation of new research methodologies for purposes

of assessing oral communication. However, it is also worthwhile to examine some of the instruments already available. The instruments reviewed in Parts II and III are worth considering for use. In general, these instru-ments range on a continuum from direct (obtrusive) to indirect (unobtrusive) measures. Overall, the most optimal assessment procedures are those that are least intrusive. Naturalistic observation, and even classroom teacher ratings, however, introduce problems of rater bias and problems of consistency.

Indirect tests of speaking ability would alleviate many sources of inconsistency as they assess communication behavior as it naturally occurs within the classroom or context. However, it may be diffi cult to construct such tests. Moreover, indirect tests may be contaminated by extraneous factors like reading ability and “test-wise-ness.” Also, such indirect tests are likely to exert del-eterious “wash back” effects on speech communication instructional practices, and this can lead to a focus on rote knowledge rather than internalized skill.

Direct tests of listening ability present fewer problems involving consistency, particularly when test stimuli are tape-recorded. Even so, measures of listening in conver-sation are elusive. Interactive listening--where the listener is an equal conversational partner who responds and is ever ready to switch into the role of speaker--probably calls on different skills than procedures in which test-takers listen to a tape-recorded conversation, and calls on still other skills from procedures in which test-takers demonstrate their understanding of oral commands.

In tests of speaking ability, use of rating scales pre-dominate. Typically, rating scales are applied to either extended talks or interview situations. It would be use-ful to adapt the use of rating scales to less intrusive, more interactive, communication situations.

Future research and development priorities relating to presently available measurement techniques might include:

• enhancing reliability of naturalistic observation procedures,

• developing more measures of communication in interactive situations, or

• extending performance rating scales to more in-teractive communication.

Are Assessment Instruments Inherently Susceptible to Group and Other Biases?

Consistent group differences in test scores are not, in and of themselves, evidence of test bias. Rather, test



bias can only be ascertained by determining if an in-strument over- or under-predicts a particular group’s performance on some criterion measure. Nevertheless, culture-bound evaluation materials will likely favor one cultural group over another. Such materials may include culture-bound communication contexts (e.g., role-play-ing a business executive), evaluation criteria (e.g., stan-dard English pronunciation and syntax), and test stimuli (e.g., “Point to the grandfather clock.”).

A less obvious source of potential bias against particu-lar cultural groups is the very notion of oral communi-cation assessment. For example, African-American chil-dren may construe the requirements of direct questioning by adults differently than do Anglo-American children. Similarly, socialization patterns among Native American Indians render an oral communication assessment as an anomalous communication contest. In addition to biases against particular cultural groups, it is possible that com-munication assessment procedures may treat particular individuals differentially. Certainly individuals with or-ganic speech defects should not be subjected to the same testing procedures as others. Additionally, certain per-sonality traits may affect the communication assessment process. Most well-known among these is communica-tion apprehension. Will special provisions be made for communication apprehension, or, if not, will the public schools be committed to “remediating” this condition as a part of their responsibility to prepare students for com-munication competency assessment?

Future research and development priorities relating to test bias in procedures for evaluating speaking and listening might include:

• developing criterion measures with which test bias may be determined;

• identifying culture-bound communication con-texts, evaluation criteria, and stimulus materials;

• determining the degree to which oral communica-tion assessment is inherently biased against par-ticular cultural groups; or

• clarifying the status of personality traits vis-à-vis test bias.

Should Communication Competence Be Assessed?

One reason to assess communication competence, in addition to certifying profi ciency among individual stu-dents, is to evaluate oral communication instruction. Of course, if our supposition of negligible communication

instruction at some schools is accurate, then this motive is obviated. Another reason for assessment, therefore, is to encourage and guide the development of communi-cation studies instruction. Testing tends to legitimize a teaching fi eld, and test specifi cations may “wash back” and stimulate instructional practice. Thus, important questions pertain to effects of communication assess-ment on teachers’ and administrators’ attitudes toward the legitimacy of communication studies, effects on cur-ricular innovation, and effects on classroom practices.

One reason not to assess, and a potentially negative effect of any testing program, is the deterioration of student attitudes. Partly this is a function of the ends to which test results are put. If the results are used in benefi cial ways for students as well as for teachers, then oral competency assessment is in order. In any event, it is worthwhile investigating whether the potential ben-efi ts of evaluating communication skills are offset by negative attitudinal outcomes.

Because large scale assessment of oral competency is just becoming widespread, limited information is avail-able concerning its effects on institutional allocations. What are the costs of oral communication testing in terms of instructional hours, personnel hours expended, and dollars spent? It is indeed likely that many adminis-trators do not encourage large scale direct measurement of speech communication competency because they fear it will be too costly. Such cost estimates must be weighed against presently unquantifi able benefi ts and utility to students, teachers, and administrators. NCA has clearly stated, through the delineation of assessment instrument criteria, that the students must either speak or listen (NCA, 1993). While the time and effort neces-sary to conduct effective speaking and listening assess-ment may seem daunting, the effort is well worth the benefi ts and outcome.

In summary, we propose the following research and development priorities relating to the utility and advis-ability of procedures for assessing oral communication competencies and speaking and listening:

• ascertaining whether measures are sensitive for purposes of assessing instructional impact;

• determining effects and value of assessment pro-grams for students, teachers, and administrators;

• determining the curricular and instructional re-sults of assessment programs;

• identifying the ends toward which test results are put; and



bias can only be ascertained by determining if an in-strument over- or under-predicts a particular group’s performance on some criterion measure. Nevertheless, culture-bound evaluation materials will likely favor one cultural group over another. Such materials may include culture-bound communication contexts (e.g., role-play-ing a business executive), evaluation criteria (e.g., stan-dard English pronunciation and syntax), and test stimuli (e.g., “Point to the grandfather clock.”).

A less obvious source of potential bias against particu-lar cultural groups is the very notion of oral communi-cation assessment. For example, African-American chil-dren may construe the requirements of direct questioning by adults differently than do Anglo-American children. Similarly, socialization patterns among Native American Indians render an oral communication assessment as an anomalous communication contest. In addition to biases against particular cultural groups, it is possible that com-munication assessment procedures may treat particular individuals differentially. Certainly individuals with or-ganic speech defects should not be subjected to the same testing procedures as others. Additionally, certain per-sonality traits may affect the communication assessment process. Most well-known among these is communica-tion apprehension. Will special provisions be made for communication apprehension, or, if not, will the public schools be committed to “remediating” this condition as a part of their responsibility to prepare students for com-munication competency assessment?

Future research and development priorities relating to test bias in procedures for evaluating speaking and listening might include:

• developing criterion measures with which test bias may be determined;

• identifying culture-bound communication con-texts, evaluation criteria, and stimulus materials;

• determining the degree to which oral communica-tion assessment is inherently biased against par-ticular cultural groups; or

• clarifying the status of personality traits vis-à-vis test bias.

Should Communication Competence Be Assessed?

One reason to assess communication competence, in addition to certifying profi ciency among individual stu-dents, is to evaluate oral communication instruction. Of course, if our supposition of negligible communication

instruction at some schools is accurate, then this motive is obviated. Another reason for assessment, therefore, is to encourage and guide the development of communi-cation studies instruction. Testing tends to legitimize a teaching fi eld, and test specifi cations may “wash back” and stimulate instructional practice. Thus, important questions pertain to effects of communication assess-ment on teachers’ and administrators’ attitudes toward the legitimacy of communication studies, effects on cur-ricular innovation, and effects on classroom practices.

One reason not to assess, and a potentially negative effect of any testing program, is the deterioration of student attitudes. Partly this is a function of the ends to which test results are put. If the results are used in benefi cial ways for students as well as for teachers, then oral competency assessment is in order. In any event, it is worthwhile investigating whether the potential ben-efi ts of evaluating communication skills are offset by negative attitudinal outcomes.

Because large scale assessment of oral competency is just becoming widespread, limited information is avail-able concerning its effects on institutional allocations. What are the costs of oral communication testing in terms of instructional hours, personnel hours expended, and dollars spent? It is indeed likely that many adminis-trators do not encourage large scale direct measurement of speech communication competency because they fear it will be too costly. Such cost estimates must be weighed against presently unquantifi able benefi ts and utility to students, teachers, and administrators. NCA has clearly stated, through the delineation of assessment instrument criteria, that the students must either speak or listen (NCA, 1993). While the time and effort neces-sary to conduct effective speaking and listening assess-ment may seem daunting, the effort is well worth the benefi ts and outcome.

In summary, we propose the following research and development priorities relating to the utility and advis-ability of procedures for assessing oral communication competencies and speaking and listening:

• ascertaining whether measures are sensitive for purposes of assessing instructional impact;

• determining effects and value of assessment pro-grams for students, teachers, and administrators;

• determining the curricular and instructional re-sults of assessment programs;

• identifying the ends toward which test results are put; and



• ascertaining effects of evaluating students’ com-petencies on their attitudes toward communica-tion.

Conclusion

In spite of considerable advances made since previ-ous editions of this volume, much work in the area of oral competency assessment remains to be done. Parts II and III review some available instruments and, in do-ing so, point out what is available and what is missing. Some gaps exist in general measures of communication

competency as well as in other communication contexts. Speech communication professionals are presently at work fi lling these gaps by engaging in research on in-strument development, tests of validity and reliability, and the effects of the test process itself. The results of these efforts will be of immense benefi t to students and to the communication discipline. As professionals com-mitted to the importance of oral communication and to the effective education of our nation’s students, we have an opportunity through assessment to have a major im-pact on the priorities we value.



• ascertaining effects of evaluating students’ com-petencies on their attitudes toward communica-tion.

Conclusion

In spite of considerable advances made since previ-ous editions of this volume, much work in the area of oral competency assessment remains to be done. Parts II and III review some available instruments and, in do-ing so, point out what is available and what is missing. Some gaps exist in general measures of communication

competency as well as in other communication contexts. Speech communication professionals are presently at work fi lling these gaps by engaging in research on in-strument development, tests of validity and reliability, and the effects of the test process itself. The results of these efforts will be of immense benefi t to students and to the communication discipline. As professionals com-mitted to the importance of oral communication and to the effective education of our nation’s students, we have an opportunity through assessment to have a major im-pact on the priorities we value.



ReferencesAllen, T. (2002). Charting a communication pathway: Using

assessment to guide curriculum development in a re-vital-ized general education plan. Communication Education, 51, 1, 26-39.

Association Trends (June, 1997). Graduates are not prepared to work in business. p. 4.

Backlund, P., Booth, J., Moore, M., Parks, A., & VanRheenen, D. (1982). A national survey of state practices in speaking and listening assessment. Communication Education, 31, 126-129.

Barnes, D. (1980). Situated speech strategies: Aspects of the monitoring of oracy. Educational Research, 32, 123-131.

Cronbach, L. J. (1988). Five perspectives on validity argu-ment. In H. Wainer (ed.), Test validity (pp. 3-17). Hillsdale, NJ: Erlbaum.

Diamond, R. (August 1, 1997). Curriculum reform needed if students are to master core skills. The Chronicle of Higher Education, p. B7.

Dickson, W. P., & Patterson, J. H. (1979). Evaluating refer-ential communication games for teaching speaking and lis-tening skills. Communication Education, 30, 11-21.

Hay, E. A. (1994). Alternative assessment in speech commu-nication. Journal of the Association for Communication Administration, 1, 1-9.

Hecker, D. (November, 2005). Occupation employment pro-jections to 2014. Monthly Labor Review, 70-101.

Herman, J. L., Aschbacher, P. R., & Winters, L. (1992). A practical guide to alternative assessment. Alexandria, VA: Association for Supervision and Curriculum Develop-ment.

Johnston, P. H. (1992). Constructive evaluation of literate ac-tivity. New York: Longman.

Joint Task Force on Assessment of the International Reading Association and National Council of Teachers of English (in press). Standards for the assessment of reading and writing. Urbana, IL: National Council of Teachers Eng-lish.

Kenny, S. (2002). Making the rhetoric a reality. Speech pre-sented at the Boyer Reinvention Center Conference. Palo Alto, CA.

Lundsteen, S. W. (1979). Listening: Its impact on reading and the language arts. Urbana, IL: ERIC and National Council of Teachers of English.

Mead, N. A. (1982). Assessment of listening and speaking performance. Paper presented at the National Symposium on Education Research, Washington, DC.

Morreale, S. P., Gomez, A. M., & Shockley-Zalabak, P. S. (1992, November). Oral communication competency as-sessment and cultural diversity: Ethnicity and students at-risk. Paper presented at the meeting of the National Com-munication Association, Chicago

Morreale, S. Moore, M., Surges-Tatum, Webster, L. (2007). The competent speaker, 2nd ed. Washington, DC: National Communication Association.

Morreale, S., Osborn, M., & Pearson, J. (2000). Why com-munication is important: A rationale for the centrality of the study of communication. Journal of the Association for Communication Administration, 29, 1-25.

Murane, R. and Levy, F. (1996). Teaching the new basic skills. New York, NY: Free Press.

National Communication Association (1994). NCA summer conference on collegiate oral communication competency: Published proceedings. Washington, DC; Author.

National Communication Association (1993). The National Communication Association’s criteria for the assessment of oral communication. Washington, DC: Author.

National Education Goals Panel. (1992). The national educa-tion goals report: Building a nation of learners. Washing-ton, DC. Author.

Piche, G. L., Rubin, D. L., & Turner, L. J. (1980). Training for referential communication accuracy in writing. Re-search in the Teaching of English, 14, 309-318.

Rubin, D. L. (1981). Using performance rating scales in large scale assessments of speaking profi ciency. In R. Stiggins (ed.) Perspective on Oral Communication Assessment in the 80’s. Portland, OR: Northwest Regional Educational Laboratory.

Rubin, D. L. (1992). Cultural bias undermines assessment. Personnel Journal, 71(5), 47-52.

Rubin, R. B. (1982). Assessing speaking and listening skills at the college level: The Communication Competency Assess-ment Instrument. Communication Education, 31, 19-32.

Rubin, R. B. (1985). The validity of the Communication Competency Assessment Instrument. Communication Monographs, 52, 173-185.

Stiggins, J., Backlund, P., & Bridgeford, N. (1985). Avoiding bias in the assessment of communication skills. Communi-cation Education, 34, 136-141.

Valadez, C. L. (1989). Problems of bias and test-wiseness in measuring oral communication and problem-solving skills through multiple choice items. In W. E. Tomes (ed.), Test-wiseness: Cultural orientation and job related factors in the design of multiple choice test questions (pp. 17-27). Alexandria, VA: Assessment Council of the International Personnel Management Association.

VanderVeen, A. (2006). College Board standards for college success: Reading, writing, speaking, listening, and media literacy. New York: The College Board.

Winsor, J.L., Curtis, D.B., and Stephens, R.D. (1997). Na-tional Preferences in business and communication educa-tion: A survey update. Journal for the Association of Com-munication Administration, 3, 170-179.

Wolf, D., Bixby, J., Glenn, J., & Gardner, H. (1991). To use their minds well: Investigating new forms of student as-sessment. Review of Research in Education, 17, 31-74.

Work Week. (December 29, 1998). Wall Street Journal, p. A1.Yule, G. (1997. Referential communication tasks. Mahwah,

NJ: Lawrence Erlbaum.



ReferencesAllen, T. (2002). Charting a communication pathway: Using

assessment to guide curriculum development in a re-vital-ized general education plan. Communication Education, 51, 1, 26-39.

Association Trends (June, 1997). Graduates are not prepared to work in business. p. 4.

Backlund, P., Booth, J., Moore, M., Parks, A., & VanRheenen, D. (1982). A national survey of state practices in speaking and listening assessment. Communication Education, 31, 126-129.

Barnes, D. (1980). Situated speech strategies: Aspects of the monitoring of oracy. Educational Research, 32, 123-131.

Cronbach, L. J. (1988). Five perspectives on validity argu-ment. In H. Wainer (ed.), Test validity (pp. 3-17). Hillsdale, NJ: Erlbaum.

Diamond, R. (August 1, 1997). Curriculum reform needed if students are to master core skills. The Chronicle of Higher Education, p. B7.

Dickson, W. P., & Patterson, J. H. (1979). Evaluating refer-ential communication games for teaching speaking and lis-tening skills. Communication Education, 30, 11-21.

Hay, E. A. (1994). Alternative assessment in speech commu-nication. Journal of the Association for Communication Administration, 1, 1-9.

Hecker, D. (November, 2005). Occupation employment pro-jections to 2014. Monthly Labor Review, 70-101.

Herman, J. L., Aschbacher, P. R., & Winters, L. (1992). A practical guide to alternative assessment. Alexandria, VA: Association for Supervision and Curriculum Develop-ment.

Johnston, P. H. (1992). Constructive evaluation of literate ac-tivity. New York: Longman.

Joint Task Force on Assessment of the International Reading Association and National Council of Teachers of English (in press). Standards for the assessment of reading and writing. Urbana, IL: National Council of Teachers Eng-lish.

Kenny, S. (2002). Making the rhetoric a reality. Speech pre-sented at the Boyer Reinvention Center Conference. Palo Alto, CA.

Lundsteen, S. W. (1979). Listening: Its impact on reading and the language arts. Urbana, IL: ERIC and National Council of Teachers of English.

Mead, N. A. (1982). Assessment of listening and speaking performance. Paper presented at the National Symposium on Education Research, Washington, DC.

Morreale, S. P., Gomez, A. M., & Shockley-Zalabak, P. S. (1992, November). Oral communication competency as-sessment and cultural diversity: Ethnicity and students at-risk. Paper presented at the meeting of the National Com-munication Association, Chicago

Morreale, S. Moore, M., Surges-Tatum, Webster, L. (2007). The competent speaker, 2nd ed. Washington, DC: National Communication Association.

Morreale, S., Osborn, M., & Pearson, J. (2000). Why com-munication is important: A rationale for the centrality of the study of communication. Journal of the Association for Communication Administration, 29, 1-25.

Murane, R. and Levy, F. (1996). Teaching the new basic skills. New York, NY: Free Press.

National Communication Association (1994). NCA summer conference on collegiate oral communication competency: Published proceedings. Washington, DC; Author.

National Communication Association (1993). The National Communication Association’s criteria for the assessment of oral communication. Washington, DC: Author.

National Education Goals Panel. (1992). The national educa-tion goals report: Building a nation of learners. Washing-ton, DC. Author.

Piche, G. L., Rubin, D. L., & Turner, L. J. (1980). Training for referential communication accuracy in writing. Re-search in the Teaching of English, 14, 309-318.

Rubin, D. L. (1981). Using performance rating scales in large scale assessments of speaking profi ciency. In R. Stiggins (ed.) Perspective on Oral Communication Assessment in the 80’s. Portland, OR: Northwest Regional Educational Laboratory.

Rubin, D. L. (1992). Cultural bias undermines assessment. Personnel Journal, 71(5), 47-52.

Rubin, R. B. (1982). Assessing speaking and listening skills at the college level: The Communication Competency Assess-ment Instrument. Communication Education, 31, 19-32.

Rubin, R. B. (1985). The validity of the Communication Competency Assessment Instrument. Communication Monographs, 52, 173-185.

Stiggins, J., Backlund, P., & Bridgeford, N. (1985). Avoiding bias in the assessment of communication skills. Communi-cation Education, 34, 136-141.

Valadez, C. L. (1989). Problems of bias and test-wiseness in measuring oral communication and problem-solving skills through multiple choice items. In W. E. Tomes (ed.), Test-wiseness: Cultural orientation and job related factors in the design of multiple choice test questions (pp. 17-27). Alexandria, VA: Assessment Council of the International Personnel Management Association.

VanderVeen, A. (2006). College Board standards for college success: Reading, writing, speaking, listening, and media literacy. New York: The College Board.

Winsor, J.L., Curtis, D.B., and Stephens, R.D. (1997). Na-tional Preferences in business and communication educa-tion: A survey update. Journal for the Association of Com-munication Administration, 3, 170-179.

Wolf, D., Bixby, J., Glenn, J., & Gardner, H. (1991). To use their minds well: Investigating new forms of student as-sessment. Review of Research in Education, 17, 31-74.

Work Week. (December 29, 1998). Wall Street Journal, p. A1.Yule, G. (1997. Referential communication tasks. Mahwah,

NJ: Lawrence Erlbaum.



Background and Discussion

Standardized assessment reigns as a signifi cant factor in today’s pre-school through secondary (P-12) educa-tional policies. In particular, elementary and high school curricular decisions regarding communication programs are often driven by two closely linked movements. The fi rst is the expectation that instruction should be stan-dards based. In other words, certain knowledge and per-formance should be taught to all students. The second movement is test-based accountability suggesting that school effectiveness should be primarily determined by student performance on those standards (Barton, 2004). Thus, large scale assessments need to be administered to measure student progress. These movements are not new, but have taken on added relevance due to deepen-ing concern regarding school performance.

Political, social, and business leaders have indicated their interest in making certain that our public school system is upholding its obligation to prepare students well. Education is viewed as an expensive investment and, as a result, it must demonstrate how well it is ex-pending that investment. No longer are America’s lead-ers accepting education’s word that students are learn-ing, and if not, that the schools just need more money. Rather, education is being asked to prove their outcomes through data-driven assessment. The implication is that if schools want more money, they must demonstrate results.

Certain disciplines such as history, math and the hard sciences easily adapted to these movements. As-sessments of students knowing the dates of historical events, adding, subtracting, and naming the bones in the body were already in place. Data collaborated if the stu-dents knew the material or not.

Other subjects created problems for standardized as-sessment. Literature, for example, can test on student recollection of story events, but has more diffi culty test-ing student comprehension of meanings. The arts have similar problems with standardized assessments of stu-dent attitude and appreciation. Communication scholars argue similarly that since there are so many variables affecting the communication process, standardized as-sessments are nearly impossible.

Failure for the communication discipline to address standards-based assessment trends or to “ride out the fad” leads to signifi cant consequences. Many P-12

communication programs have already been cut or in-corporated into other programs over the past 10 years since their schools had to shift resources. It is not that communication educators do not assess. Local, indi-vidualized assessment in communication courses is well documented. Rather, the reluctance to develop and practice standardized assessment throughout the disci-pline challenges communication’s ability to maintain a prominent spot in P-12 education.

The purpose of this chapter, therefore, is to renew the discussions and considerations of standardized as-sessment in communication to ensure the continued health of communication instruction at P-12 levels. To accomplish this goal, the chapter reviews trends in for-mal P-12 standards-based instruction and assessment measures. It additionally identifi es a variety of exter-nal assessment devices available to school systems and academic departments who wish to conduct large scale assessments in communication. Finally, some conclu-sions will be drawn regarding ways to make large scale assessments more common in P-12 education.

Standards-Based Instruction

Standards based instruction typically identifi es mini-mal levels of student knowledge, performance behav-iors, and attitude in specifi c content areas of each dis-cipline. A standard sets a level of accomplishment that all students are expected to meet or exceed. Many P-12 standards utilize descriptors that indicate if students are beginning, meeting, or exceeding the goals.

Congress passed the Goals 2000: Educate America Act in 1994 in order to encourage the nation’s schools to implement standards-based reform (National Confer-ence of State Legislatures, 2006). Therefore, through-out the 90’s, states and national learned societies spent considerable effort identifying, labeling, and explaining what each student should academically know and be able to do.

The National Communication Association responded in 1998 by defi ning their expectations of specifi c com-munication skills that P-12 students should master. NCA’s publication of this work is included on their web site, www.natcom.org. NCA developed standards for speaking, listening, and media literacy for students in P-12. For the 20 standards, the association has established competency statements in the areas of knowledge, at-

Part II: Oral Communication Assessment Instruments for P-12




Standardized assessment reigns as a signifi cant factor in today’s pre-school through secondary (P-12) educa-tional policies. In particular, elementary and high school curricular decisions regarding communication programs are often driven by two closely linked movements. The fi rst is the expectation that instruction should be stan-dards based. In other words, certain knowledge and per-formance should be taught to all students. The second movement is test-based accountability suggesting that school effectiveness should be primarily determined by student performance on those standards (Barton, 2004). Thus, large scale assessments need to be administered to measure student progress. These movements are not new, but have taken on added relevance due to deepen-ing concern regarding school performance.

Political, social, and business leaders have indicated their interest in making certain that our public school system is upholding its obligation to prepare students well. Education is viewed as an expensive investment and, as a result, it must demonstrate how well it is ex-pending that investment. No longer are America’s lead-ers accepting education’s word that students are learn-ing, and if not, that the schools just need more money. Rather, education is being asked to prove their outcomes through data-driven assessment. The implication is that if schools want more money, they must demonstrate results.

Certain disciplines such as history, math and the hard sciences easily adapted to these movements. As-sessments of students knowing the dates of historical events, adding, subtracting, and naming the bones in the body were already in place. Data collaborated if the stu-dents knew the material or not.

Other subjects created problems for standardized as-sessment. Literature, for example, can test on student recollection of story events, but has more diffi culty test-ing student comprehension of meanings. The arts have similar problems with standardized assessments of stu-dent attitude and appreciation. Communication scholars argue similarly that since there are so many variables affecting the communication process, standardized as-sessments are nearly impossible.

Failure for the communication discipline to address standards-based assessment trends or to “ride out the fad” leads to signifi cant consequences. Many P-12

communication programs have already been cut or in-corporated into other programs over the past 10 years since their schools had to shift resources. It is not that communication educators do not assess. Local, indi-vidualized assessment in communication courses is well documented. Rather, the reluctance to develop and practice standardized assessment throughout the disci-pline challenges communication’s ability to maintain a prominent spot in P-12 education.

The purpose of this chapter, therefore, is to renew the discussions and considerations of standardized as-sessment in communication to ensure the continued health of communication instruction at P-12 levels. To accomplish this goal, the chapter reviews trends in for-mal P-12 standards-based instruction and assessment measures. It additionally identifi es a variety of exter-nal assessment devices available to school systems and academic departments who wish to conduct large scale assessments in communication. Finally, some conclu-sions will be drawn regarding ways to make large scale assessments more common in P-12 education.

Standards-Based Instruction

Standards based instruction typically identifi es mini-mal levels of student knowledge, performance behav-iors, and attitude in specifi c content areas of each dis-cipline. A standard sets a level of accomplishment that all students are expected to meet or exceed. Many P-12 standards utilize descriptors that indicate if students are beginning, meeting, or exceeding the goals.

Congress passed the Goals 2000: Educate America Act in 1994 in order to encourage the nation’s schools to implement standards-based reform (National Confer-ence of State Legislatures, 2006). Therefore, through-out the 90’s, states and national learned societies spent considerable effort identifying, labeling, and explaining what each student should academically know and be able to do.

The National Communication Association responded in 1998 by defi ning their expectations of specifi c com-munication skills that P-12 students should master. NCA’s publication of this work is included on their web site, www.natcom.org. NCA developed standards for speaking, listening, and media literacy for students in P-12. For the 20 standards, the association has established competency statements in the areas of knowledge, at-

Part II: Oral Communication Assessment Instruments for P-12



titudes, and behaviors, along with grade-level teaching activities aligned with the standards and competencies. In addition, an assessment guidelines program is being produced for use with the standards and competencies. (NCA Speaking, Listening and Media Standards and Competency Statements for P-12 Education, 1998a)

Other disciplines, through their own learned societ-ies, created similar content guidelines since they were usually considered the authoritative source for what stu-dents should know and be able to do. The hope was that when state boards of education created local or state-wide standards, they would refer to the expertise of the learned society. This was not always the case. A recent review of state standards revealed all 50 states plus the District of Columbia utilized a large variety of termi-nology and structure for their standards. In essence, each decided they needed to “invent their own wheel” (Jennings, et al, 2006).

Justifi cation for the individualization comes from the belief in each state’s uniqueness of their cultural and eco-

nomic base. Most legislators and education policy mak-ers in the United States believe that standards should be locally created due to these needs. Most states worked through their boards of education and utilized commit-tees of P-12 faculty, parents, prominent business people, and other interested parties to develop their documents. Therefore, how much a learned society was utilized in the process was decided by the committee’s awareness of the society, their ability to seek out standards created by the society, and their willingness to conform their students’ expectations to an external or national guide.

Communication Standards

The National Communication Association designed 20 specifi c K-12 communication learning standards within four major categories: Communication fundamentals, speaking, listening, and media. Each category is subdi-vided into knowledge, behavioral, and attitudinal indica-tors. The NCA standards provide specifi c identifi ers of what K-12 students should be able to do. For example:

FUNDAMENTALS OF EFFECTIVE COMMUNICATION

Standard 2

Competent communicators. . . .

Knowledge Behaviors Attitudes

2.1 describe their shifting roles in a variety of communication settings

2.7 use their experiences and knowledge to interpret messages

2.14 respect the diversity of communication strategies used by individuals

Communication skills are listed in each of the stan-dards sets established by the 51 states and Washington, D.C., education units. In an analysis of state communi-cation standards, Jennings, Long, Nollinger, and Sank found a range of 7 to 27 communication standards for P-12 students. Maryland, New Hampshire and North Dakota had the fewest communication standards while Kansas, Missouri, Michigan, Washington, D.C., and South Dakota had the most. In a 1998 to 2006 com-parison, states showing the biggest gains were South Dakota (from 10 communication standards in 1998 to 22 standards in 2006) and Utah (from only one in 1998 to 12 in 2006). Overall, from 1998 to 2006, gainers out-paced losers.

The most commonly stated communication stan-dards:

• Able to interpret / evaluate messages (37 times)• Effective delivery (35 times)• Create a variety of oral messages (32 times)• Give constructive feedback (23 times)• Follow directions / respond appropriately (21

times)

Overall, communication standards seem to be rela-tively general and stress speaking, listening, and group work skills. One additional trend that became evident during the study was the growing incorporation of communication skills into the English Language Arts (ELA). Alabama provides an example of the incorpora-tion of generally stated communication skills expected within their English Language Arts curriculum in the following 9th grade ELA standards statement (Alabama State Board of Education website).



titudes, and behaviors, along with grade-level teaching activities aligned with the standards and competencies. In addition, an assessment guidelines program is being produced for use with the standards and competencies. (NCA Speaking, Listening and Media Standards and Competency Statements for P-12 Education, 1998a)

Other disciplines, through their own learned societ-ies, created similar content guidelines since they were usually considered the authoritative source for what stu-dents should know and be able to do. The hope was that when state boards of education created local or state-wide standards, they would refer to the expertise of the learned society. This was not always the case. A recent review of state standards revealed all 50 states plus the District of Columbia utilized a large variety of termi-nology and structure for their standards. In essence, each decided they needed to “invent their own wheel” (Jennings, et al, 2006).

Justifi cation for the individualization comes from the belief in each state’s uniqueness of their cultural and eco-

nomic base. Most legislators and education policy mak-ers in the United States believe that standards should be locally created due to these needs. Most states worked through their boards of education and utilized commit-tees of P-12 faculty, parents, prominent business people, and other interested parties to develop their documents. Therefore, how much a learned society was utilized in the process was decided by the committee’s awareness of the society, their ability to seek out standards created by the society, and their willingness to conform their students’ expectations to an external or national guide.

Communication Standards

The National Communication Association designed 20 specifi c K-12 communication learning standards within four major categories: Communication fundamentals, speaking, listening, and media. Each category is subdi-vided into knowledge, behavioral, and attitudinal indica-tors. The NCA standards provide specifi c identifi ers of what K-12 students should be able to do. For example:

FUNDAMENTALS OF EFFECTIVE COMMUNICATION

Standard 2

Competent communicators. . . .

Knowledge Behaviors Attitudes

2.1 describe their shifting roles in a variety of communication settings

2.7 use their experiences and knowledge to interpret messages

2.14 respect the diversity of communication strategies used by individuals

Communication skills are listed in each of the stan-dards sets established by the 51 states and Washington, D.C., education units. In an analysis of state communi-cation standards, Jennings, Long, Nollinger, and Sank found a range of 7 to 27 communication standards for P-12 students. Maryland, New Hampshire and North Dakota had the fewest communication standards while Kansas, Missouri, Michigan, Washington, D.C., and South Dakota had the most. In a 1998 to 2006 com-parison, states showing the biggest gains were South Dakota (from 10 communication standards in 1998 to 22 standards in 2006) and Utah (from only one in 1998 to 12 in 2006). Overall, from 1998 to 2006, gainers out-paced losers.

The most commonly stated communication stan-dards:

• Able to interpret / evaluate messages (37 times)• Effective delivery (35 times)• Create a variety of oral messages (32 times)• Give constructive feedback (23 times)• Follow directions / respond appropriately (21

times)

Overall, communication standards seem to be rela-tively general and stress speaking, listening, and group work skills. One additional trend that became evident during the study was the growing incorporation of communication skills into the English Language Arts (ELA). Alabama provides an example of the incorpora-tion of generally stated communication skills expected within their English Language Arts curriculum in the following 9th grade ELA standards statement (Alabama State Board of Education website).



(Alabama State Board of Education, 2006)

The same study cited earlier found that the number of states certifying speech teachers has decreased since 1998. Most states now either do not certify in speech or include it as part of their English certifi cation. As a result, the number of communication teacher prepa-ration institutions has declined. Illinois, for example, listed speech teaching majors at six major universities in 1990. Today, only one of those schools is producing speech teachers. A similar trend was found in the other 49 states (Jennings, et al, 2006).

The trends listed above appear to be contradictory. While one mentions the increased desire for P-12 stu-dents to have communication skills, the other demon-strates a demise of formal communication instruction and teacher preparation. One signifi cant reason the contradiction may exist is because the communication discipline has had diffi culty moving from standards-based instruction to standardized assessment. It is clear that formal assessment of communication skills has not changed much over the past 15 years even as many other disciplines have embraced those developments.

Standards-Based Assessment

Assessment is usually defi ned as any method used to determine if stated objectives or standards of performance are being met within a defi ned program or given popula-tion. Assessment is the attempt to determine if someone or something is able to do that which we have determined is important. Assessment that is standards-based refers to formal and informal examination of how well the sub-jects are performing to the provided standards.

Standards-based assessment can be accomplished locally or in large scale assessment. If done locally, the classroom instructor fi nds or designs and imple-ments their chosen assessment device. Communication instructors have typically designed a variety of grade sheets and rubrics that measure student performance of the specifi c assignment. Many of these grading forms combine closed and open responses since instructor feedback needs to be primarily formative. As a result, there is little standardization in local assessments.

Large scale assessments typically contain elements of standardization so they may be utilized over time or over a larger group of students. If the large scale as-sessment is being used for high stakes measurement, it needs to be not only standards-based, but also stan-dardized in design so that statistical comparisons can be made. Standards-based standardized assessment is most commonly practiced in reading, math, and the hard sciences. In speech, a widely distributed pencil and paper test that covers speaking concepts might be a standardized assessment. Or, a test that measures stu-dent memory and listening comprehension might be a standardized assessment.

As mentioned in the introduction, standards-based standardized assessment for speaking and listening skills is not favored by many communication educa-tors. The situational nature of communication where content and strategies are often dictated by the context mandates that we view communication responses as unique. We often agree that what we say and how we say it in one place may not be effective communication in another place. Or, what may be appropriate strategies and content for one person may not be appropriate for another person even if they are speaking in the same place. These beliefs have restricted support for stan-dards-based standardized assessment in communica-tion and, as a result, have had impact on the discipline’s place in evolving educational policy as evidenced in No Child Left Behind.

Relationship of No Child Left Behind to Communication Assessment

The No Child Left Behind Act of 2001 (NCLB) not only expanded federal government expectations on school performance within the standards, but addition-ally linked funding to performance levels of teachers and students. To determine funding, NCLB asks for statistical comparison via testing of all students in the public school population to determine how well they meet the minimum levels of achievement set in speci-fi ed content areas.

The most prominently NCLB benchmarked content areas are reading, math, and the sciences. Communica-tion skills such as problem-solving and critical think-ing are mentioned, but the ability to communicate well through speaking, listening, group interaction, and me-dia literacy are not. While good communication skills might help a student perform better on a high-stakes test, their skills and knowledge of communication con-cepts are not tested.

33. Display self-confi dence in speaking.

Examples: answering questions when called upon, voluntarily asking and answering questions, reading one’s own writing to peers, presenting results from research.



(Alabama State Board of Education, 2006)

The same study cited earlier found that the number of states certifying speech teachers has decreased since 1998. Most states now either do not certify in speech or include it as part of their English certifi cation. As a result, the number of communication teacher prepa-ration institutions has declined. Illinois, for example, listed speech teaching majors at six major universities in 1990. Today, only one of those schools is producing speech teachers. A similar trend was found in the other 49 states (Jennings, et al, 2006).

The trends listed above appear to be contradictory. While one mentions the increased desire for P-12 stu-dents to have communication skills, the other demon-strates a demise of formal communication instruction and teacher preparation. One signifi cant reason the contradiction may exist is because the communication discipline has had diffi culty moving from standards-based instruction to standardized assessment. It is clear that formal assessment of communication skills has not changed much over the past 15 years even as many other disciplines have embraced those developments.

Standards-Based Assessment

Assessment is usually defi ned as any method used to determine if stated objectives or standards of performance are being met within a defi ned program or given popula-tion. Assessment is the attempt to determine if someone or something is able to do that which we have determined is important. Assessment that is standards-based refers to formal and informal examination of how well the sub-jects are performing to the provided standards.

Standards-based assessment can be accomplished locally or in large scale assessment. If done locally, the classroom instructor fi nds or designs and imple-ments their chosen assessment device. Communication instructors have typically designed a variety of grade sheets and rubrics that measure student performance of the specifi c assignment. Many of these grading forms combine closed and open responses since instructor feedback needs to be primarily formative. As a result, there is little standardization in local assessments.

Large scale assessments typically contain elements of standardization so they may be utilized over time or over a larger group of students. If the large scale as-sessment is being used for high stakes measurement, it needs to be not only standards-based, but also stan-dardized in design so that statistical comparisons can be made. Standards-based standardized assessment is most commonly practiced in reading, math, and the hard sciences. In speech, a widely distributed pencil and paper test that covers speaking concepts might be a standardized assessment. Or, a test that measures stu-dent memory and listening comprehension might be a standardized assessment.

As mentioned in the introduction, standards-based standardized assessment for speaking and listening skills is not favored by many communication educa-tors. The situational nature of communication where content and strategies are often dictated by the context mandates that we view communication responses as unique. We often agree that what we say and how we say it in one place may not be effective communication in another place. Or, what may be appropriate strategies and content for one person may not be appropriate for another person even if they are speaking in the same place. These beliefs have restricted support for stan-dards-based standardized assessment in communica-tion and, as a result, have had impact on the discipline’s place in evolving educational policy as evidenced in No Child Left Behind.

Relationship of No Child Left Behind to Communication Assessment

The No Child Left Behind Act of 2001 (NCLB) not only expanded federal government expectations on school performance within the standards, but addition-ally linked funding to performance levels of teachers and students. To determine funding, NCLB asks for statistical comparison via testing of all students in the public school population to determine how well they meet the minimum levels of achievement set in speci-fi ed content areas.

The most prominently NCLB benchmarked content areas are reading, math, and the sciences. Communica-tion skills such as problem-solving and critical think-ing are mentioned, but the ability to communicate well through speaking, listening, group interaction, and me-dia literacy are not. While good communication skills might help a student perform better on a high-stakes test, their skills and knowledge of communication con-cepts are not tested.

33. Display self-confi dence in speaking.

Examples: answering questions when called upon, voluntarily asking and answering questions, reading one’s own writing to peers, presenting results from research.



Due to funding issues written into NCLB, it be-comes a prime motivator for P-12 standards-based standardized teaching and testing. Schools consis-tently identify NCLB as being one of the most sig-nifi cant pressures on their curriculum and expenditure decisions. Since the Act does not connect communica-tion performance to school outcomes and, therefore, funding, there is reduced motivation to devote re-sources to communication instruction. In fact, school performance requirements set by NCLB may even be disincentives for teaching speaking and listening. At least one prominent suburban Chicago school lost a freshman required-speech course that annually en-rolled over 700 students so that the school could spend more time teaching reading.

Meanwhile, there has been little national interest in creating standardized assessments in communication. The result is a shortage of standards-based standardized communication assessments for P-12. In fact, a com-parative review of the large scale assessments that were listed in the previous version of this volume discovered that many are either out of date or no longer available. The next segment summarizes those developments.

Assessing P-12 Communication

The National Communication Association has estab-lished guidelines for communication assessment that may inform P-12 assessment. The complete guidelines are available through their website, www.natcom.org. The following are the general criteria for assessing oral communication (NCA, 1998b):

1. Assessment of oral communication should view competence in oral communication as a gestalt of several interacting dimensions. At a minimum, all assessments of oral communication should in-clude an assessment of knowledge (understand-ing communication process, comprehension of the elements, rules, and dynamics of a communi-cation event, awareness of what is appropriate in a communication situation), an assessment of skills (the possession of a repertoire of skills and the actual performance of skills), and an evaluation of the individual’s attitude toward communication (e.g., value placed on oral communication, appre-hension, reticence, willingness to communicate, readiness to communicate).

2. Because oral communication is an interactive and social process, assessment should consider the judgment of a trained assessor as well as the impressions of others involved in the communica-

tion act (audience, interviewer, other group mem-bers, conversant), and may include the self-report of the individual being assessed.

3. Assessment of oral communication should clearly distinguish speaking and listening from reading and writing. While some parts of the assessment process may include reading and writing, a major portion of the assessment of oral communication should require speaking and listening. Directions from the assessor and re sponses by the individual being assessed should be in the oral/aural mode.

4. Assessment of oral communication should be sensitive to the effects of relevant physical and psychological disabilities on the assessment of competence. (e.g., with appropriate aids in sig-nal reception, a hearing impaired person can be a competent empathic listener.)

5. Assessment of oral communication should be based in part on atomistic/analytic data collected and on a holistic impression.

P-12 Large Scale Standards-Based Standardized Assessments

The previous edition of this volume reported that were no comprehensive assessment instruments for grades P-12 that meet all of the standards set by the National Communication Association. This analysis yielded the same result. In fact, the number of standard-ized communication assessments decreased since the second edition.

Several reviews and lists of communication assess-ments are available. The most comprehensive review came from the National Postsecondary Education Co-operative in their NPEC Sourcebook on Assessment: Defi nitions and Assessment Methods for Communica-tion, Leadership, Information Literacy, Quantitative Reasoning, and Quantitative Skills (June, 2005). The re-view can be found on the Internet at http://nces.ed.gov.

Another helpful resource that includes an extensive resource list is the Bibliography of Assessment Alterna-tives: Oral Communication. Created by the Northwest Regional Educational Laboratory in Portland, Oregon (June 1998), this resource can be found on the Internet at www.nwrel.org.

The earlier edition of this volume also identifi ed Ru-bin and Mead’s (1984) Large Scale Assessment of Oral Communication Skills: Kindergarten through Grade 12, Arter’s (1998) Assessing Communication Competence in Speaking and Listening, and Educational Testing Service’s (1991) Measures of Speech/Communication.



Due to funding issues written into NCLB, it be-comes a prime motivator for P-12 standards-based standardized teaching and testing. Schools consis-tently identify NCLB as being one of the most sig-nifi cant pressures on their curriculum and expenditure decisions. Since the Act does not connect communica-tion performance to school outcomes and, therefore, funding, there is reduced motivation to devote re-sources to communication instruction. In fact, school performance requirements set by NCLB may even be disincentives for teaching speaking and listening. At least one prominent suburban Chicago school lost a freshman required-speech course that annually en-rolled over 700 students so that the school could spend more time teaching reading.

Meanwhile, there has been little national interest in creating standardized assessments in communication. The result is a shortage of standards-based standardized communication assessments for P-12. In fact, a com-parative review of the large scale assessments that were listed in the previous version of this volume discovered that many are either out of date or no longer available. The next segment summarizes those developments.

Assessing P-12 Communication

The National Communication Association has estab-lished guidelines for communication assessment that may inform P-12 assessment. The complete guidelines are available through their website, www.natcom.org. The following are the general criteria for assessing oral communication (NCA, 1998b):

1. Assessment of oral communication should view competence in oral communication as a gestalt of several interacting dimensions. At a minimum, all assessments of oral communication should in-clude an assessment of knowledge (understand-ing communication process, comprehension of the elements, rules, and dynamics of a communi-cation event, awareness of what is appropriate in a communication situation), an assessment of skills (the possession of a repertoire of skills and the actual performance of skills), and an evaluation of the individual’s attitude toward communication (e.g., value placed on oral communication, appre-hension, reticence, willingness to communicate, readiness to communicate).

2. Because oral communication is an interactive and social process, assessment should consider the judgment of a trained assessor as well as the impressions of others involved in the communica-

tion act (audience, interviewer, other group mem-bers, conversant), and may include the self-report of the individual being assessed.

3. Assessment of oral communication should clearly distinguish speaking and listening from reading and writing. While some parts of the assessment process may include reading and writing, a major portion of the assessment of oral communication should require speaking and listening. Directions from the assessor and re sponses by the individual being assessed should be in the oral/aural mode.

4. Assessment of oral communication should be sensitive to the effects of relevant physical and psychological disabilities on the assessment of competence. (e.g., with appropriate aids in sig-nal reception, a hearing impaired person can be a competent empathic listener.)

5. Assessment of oral communication should be based in part on atomistic/analytic data collected and on a holistic impression.

P-12 Large Scale Standards-Based Standardized Assessments

The previous edition of this volume reported that were no comprehensive assessment instruments for grades P-12 that meet all of the standards set by the National Communication Association. This analysis yielded the same result. In fact, the number of standard-ized communication assessments decreased since the second edition.

Several reviews and lists of communication assess-ments are available. The most comprehensive review came from the National Postsecondary Education Co-operative in their NPEC Sourcebook on Assessment: Defi nitions and Assessment Methods for Communica-tion, Leadership, Information Literacy, Quantitative Reasoning, and Quantitative Skills (June, 2005). The re-view can be found on the Internet at http://nces.ed.gov.

Another helpful resource that includes an extensive resource list is the Bibliography of Assessment Alterna-tives: Oral Communication. Created by the Northwest Regional Educational Laboratory in Portland, Oregon (June 1998), this resource can be found on the Internet at www.nwrel.org.

The earlier edition of this volume also identifi ed Ru-bin and Mead’s (1984) Large Scale Assessment of Oral Communication Skills: Kindergarten through Grade 12, Arter’s (1998) Assessing Communication Competence in Speaking and Listening, and Educational Testing Service’s (1991) Measures of Speech/Communication.



Five common aspects of communication became evident in the review of existing standards-based stan-dardized assessments: listening, communication com-petency, learning aptitude through communication, nonverbal intelligence, and communication skills of non-native speakers. The fi rst two appear to be disci-pline based while the last three use communication skills to determine learning aptitude, general intelligence, or language acquisition. All reviewed assessments attempt to measure a single focus rather than comprehensive communication skills. Thus, if a state or school wishes to do comprehensive, standardized assessments of their communication standards, they need to design their own instrument or combine a variety of commercial in-struments. These fi ve aspects will now be discussed fol-lowed by reviews of individual instruments now avail-able for P-12 assessment of communication.

Listening

Listening appears to have generated the most large scale assessments designed by communication research-ers. Listening research from the 1960’s through the 1980’s led to the creation of a number of well-known assessments. Perhaps the most familiar, the Watson-Barker Listening Test, was developed in 1986 and re-vised in 1996 and 2000. A comprehensive and descrip-tive summary of listening assessments and resources can be found on the International Listening Association (ILA) web site, www.listen.org.

Most listening tests use an auditory stimulus where the subject, after listening to the stimulus, responds to a series of related questions. Tests mostly measure how well the subject listens to varying types of listening sit-uations and how well they recall elements of ideas and their ability to draw inferences.

A preliminary review before purchase is recom-mended since some of the instruments were designed as early as 1955. As a result, tests that have not been updated recently may include social bias or measure memory rather than listening ability.

Communication Competency

Communication competency assessments most of-ten measure communicator comfort and competency in interpersonal, group, and public situations. Wiemann’s 1977 Communicative Competence Scale (CCS) and the Communication Competency Assessment Instrument – High School (CCAI-HS), published by Spectra, Inc. in 1993 were commonly cited instruments in the 1990s, but no longer appear to be commercially available.

Spectra Publishers still offer other general communi-cation inventories. These assessments typically evalu-ate student behaviors that might affect their learning performance in the classroom. Areas such as listening, speaking, non-verbal and group work can all be mea-sured to see how well the students communicate in the educational setting.

Learning Aptitude

Harcourt Assessment and CTB McGraw-Hill host a wide variety of well known learning aptitude large scale assessments that utilize communication skills to measure student interest and ability to learn. Harcourt utilizes the Wechsler Intelligence Scales and has designed the WIAT-II to assess problem-solving abilities by evaluating the process. The WIAT-II includes sub-tests on oral language and listening comprehension. Learning aptitude tests continue to be created and updated regularly.

Nonverbal Intelligence

Nonverbal intelligence tests provide standardized and norm-referenced assessments of general intelligence with entirely nonverbal administration and response formats. They are considered a type of learning apti-tude test, but workable also for non-English speakers or others who have language, hearing impairments, or are from different cultural backgrounds. These tests pro-vide diagnostic information relevant to common learn-ing problems associated with various disabilities.

Communication Skills of Non-Native English Speakers

Achievement testing for English language learners has traditionally assessed the extent to which the learner has acquired vocabulary and use of the language. In-terestingly, ESL and foreign language disciplines often utilize similar standards as the communication disci-pline. They are focused on student competencies of fl u-ency in speaking situations, congruency of verbal and non-verbal messages, and overall effectiveness of the message. Considering these shared concerns, the ESL, foreign language and communication disciplines can contribute to each other’s understandings of assessment challenges.

Conclusion

The communication discipline, with the support of the National Communication Association, has embraced the concept of standards based instruction. Not only are the P-12 standards for speaking, listening and media lit-



Five common aspects of communication became evident in the review of existing standards-based stan-dardized assessments: listening, communication com-petency, learning aptitude through communication, nonverbal intelligence, and communication skills of non-native speakers. The fi rst two appear to be disci-pline based while the last three use communication skills to determine learning aptitude, general intelligence, or language acquisition. All reviewed assessments attempt to measure a single focus rather than comprehensive communication skills. Thus, if a state or school wishes to do comprehensive, standardized assessments of their communication standards, they need to design their own instrument or combine a variety of commercial in-struments. These fi ve aspects will now be discussed fol-lowed by reviews of individual instruments now avail-able for P-12 assessment of communication.

Listening

Listening appears to have generated the most large scale assessments designed by communication research-ers. Listening research from the 1960’s through the 1980’s led to the creation of a number of well-known assessments. Perhaps the most familiar, the Watson-Barker Listening Test, was developed in 1986 and re-vised in 1996 and 2000. A comprehensive and descrip-tive summary of listening assessments and resources can be found on the International Listening Association (ILA) web site, www.listen.org.

Most listening tests use an auditory stimulus where the subject, after listening to the stimulus, responds to a series of related questions. Tests mostly measure how well the subject listens to varying types of listening sit-uations and how well they recall elements of ideas and their ability to draw inferences.

A preliminary review before purchase is recom-mended since some of the instruments were designed as early as 1955. As a result, tests that have not been updated recently may include social bias or measure memory rather than listening ability.

Communication Competency

Communication competency assessments most of-ten measure communicator comfort and competency in interpersonal, group, and public situations. Wiemann’s 1977 Communicative Competence Scale (CCS) and the Communication Competency Assessment Instrument – High School (CCAI-HS), published by Spectra, Inc. in 1993 were commonly cited instruments in the 1990s, but no longer appear to be commercially available.

Spectra Publishers still offer other general communi-cation inventories. These assessments typically evalu-ate student behaviors that might affect their learning performance in the classroom. Areas such as listening, speaking, non-verbal and group work can all be mea-sured to see how well the students communicate in the educational setting.

Learning Aptitude

Harcourt Assessment and CTB McGraw-Hill host a wide variety of well known learning aptitude large scale assessments that utilize communication skills to measure student interest and ability to learn. Harcourt utilizes the Wechsler Intelligence Scales and has designed the WIAT-II to assess problem-solving abilities by evaluating the process. The WIAT-II includes sub-tests on oral language and listening comprehension. Learning aptitude tests continue to be created and updated regularly.

Nonverbal Intelligence

Nonverbal intelligence tests provide standardized and norm-referenced assessments of general intelligence with entirely nonverbal administration and response formats. They are considered a type of learning apti-tude test, but workable also for non-English speakers or others who have language, hearing impairments, or are from different cultural backgrounds. These tests pro-vide diagnostic information relevant to common learn-ing problems associated with various disabilities.

Communication Skills of Non-Native English Speakers

Achievement testing for English language learners has traditionally assessed the extent to which the learner has acquired vocabulary and use of the language. In-terestingly, ESL and foreign language disciplines often utilize similar standards as the communication disci-pline. They are focused on student competencies of fl u-ency in speaking situations, congruency of verbal and non-verbal messages, and overall effectiveness of the message. Considering these shared concerns, the ESL, foreign language and communication disciplines can contribute to each other’s understandings of assessment challenges.

Conclusion

The communication discipline, with the support of the National Communication Association, has embraced the concept of standards based instruction. Not only are the P-12 standards for speaking, listening and media lit-



eracy well articulated, they refl ect concerns from both education and business. As the standards became more known, many states incorporated them into their local guidelines. There is little surprise that American educa-tion and business agree that quality communication is inherent to a well-functioning society.

Despite this wide acceptance of the value of com-munication, the discipline has encountered diffi culty in maintaining previous standardized assessments much less creating new ones. National-level discussions in-variably lead to a belief that it “can’t be done.” Rubin and Mead commented in 1984 that “despite this initial impetus to evaluate communication competencies, de-spite this view that developing assessment procedures presents no insurmountable technical obstacles, and de-spite some concrete suggestions of pertinent measure-ment instruments, attempts to implement large-scale as-sessment of speaking and listening skills have not been forthcoming” (p.3). Four years later, O’Leary and Fen-

ton came to the same conclusion (1998). In 2006, there remains no signifi cant change.

Failure to design and incorporate standardized as-sessments in P-12 communication education has cre-ated and will continue to create severe challenges for communication programs in schools concerned about meeting No Child Left Behind requirements. The evi-dence already exists. Research indicates that the num-ber of states certifying new communication teachers has drastically declined as the discipline became more incorporated into English Language Arts. The number of P-12 teachers with oral communication coursework is also declining.

Accountability in education is real and is pressuring the communication discipline to respond. Rather than considering it a fad that will eventually pass, it would be better for the discipline to advocate serious review of the feasibility and desirability of standardized assess-ment of communication.



eracy well articulated, they refl ect concerns from both education and business. As the standards became more known, many states incorporated them into their local guidelines. There is little surprise that American educa-tion and business agree that quality communication is inherent to a well-functioning society.

Despite this wide acceptance of the value of com-munication, the discipline has encountered diffi culty in maintaining previous standardized assessments much less creating new ones. National-level discussions in-variably lead to a belief that it “can’t be done.” Rubin and Mead commented in 1984 that “despite this initial impetus to evaluate communication competencies, de-spite this view that developing assessment procedures presents no insurmountable technical obstacles, and de-spite some concrete suggestions of pertinent measure-ment instruments, attempts to implement large-scale as-sessment of speaking and listening skills have not been forthcoming” (p.3). Four years later, O’Leary and Fen-

ton came to the same conclusion (1998). In 2006, there remains no signifi cant change.

Failure to design and incorporate standardized as-sessments in P-12 communication education has cre-ated and will continue to create severe challenges for communication programs in schools concerned about meeting No Child Left Behind requirements. The evi-dence already exists. Research indicates that the num-ber of states certifying new communication teachers has drastically declined as the discipline became more incorporated into English Language Arts. The number of P-12 teachers with oral communication coursework is also declining.

Accountability in education is real and is pressuring the communication discipline to respond. Rather than considering it a fad that will eventually pass, it would be better for the discipline to advocate serious review of the feasibility and desirability of standardized assess-ment of communication.



ReferencesAlabama Department of Education (1999). Classroom im-

provement publications. English Language Arts. Retrieved September 20, 2006 from web site: www.alsde.edu.

Arter, Judith A. (1998). Assessing communication compe-tence in speaking and listening: A consumer’s guide. As-sessment Resource Library, Northwest Regional Educa-tional Laboratory, Portland, Oregon.

Barton, P. (2005). Unfi nished business: More measured ap-proaches in standards-based reform. Educational Testing Service, Princeton, NJ.

Educational Testing Service (1991), Praxis Series: Speech Communication (0220). Princeton, NJ. Retrieved Septem-ber 20, 2006 from Web site: www.ets.org.

Jennings, D., Long, L., Nollinger, K., & Sank, L. (April, 2006). Meeting the needs of No Child Left Behind through secondary speech courses and standardized assessments: National research results. Paper presented at the meeting of the Central States Communication Association, India-napolis.

National Communication Association. (1998a). National K-12 Standards for Speaking, Listening and Media Literacy. Washington, D.C: Author.

National Communication Association. (1998b). Guidelines for Communication Assessment. Washington, D.C.: Au-thor.

National Postsecondary Education Cooperative. (2005). NPEC sourcebook on assessment: Defi nitions and assess-ment methods for communication, leadership, information literacy, quantitative reasons, and quantitative skills.

Northwest Regional Educational Laboratory. (1998). Bibli-ography of assessment alternatives: Oral communication.

Rubin, D. L. & Mead, N.A. (1984). Large scale assessment of oral communication skills: Kindergarten through grade 12. Urbana, IL: ERIC Clearinghouse on Reading and Com-munication Skills.

Rubin, R. B., Welch, S.A., & Buerkel, R. (1995). Perfor-mance-based assessment of high school speech instruc-tion. Communication Education, 44, 30-39.



ReferencesAlabama Department of Education (1999). Classroom im-

provement publications. English Language Arts. Retrieved September 20, 2006 from web site: www.alsde.edu.

Arter, Judith A. (1998). Assessing communication compe-tence in speaking and listening: A consumer’s guide. As-sessment Resource Library, Northwest Regional Educa-tional Laboratory, Portland, Oregon.

Barton, P. (2005). Unfi nished business: More measured ap-proaches in standards-based reform. Educational Testing Service, Princeton, NJ.

Educational Testing Service (1991), Praxis Series: Speech Communication (0220). Princeton, NJ. Retrieved Septem-ber 20, 2006 from Web site: www.ets.org.

Jennings, D., Long, L., Nollinger, K., & Sank, L. (April, 2006). Meeting the needs of No Child Left Behind through secondary speech courses and standardized assessments: National research results. Paper presented at the meeting of the Central States Communication Association, India-napolis.

National Communication Association. (1998a). National K-12 Standards for Speaking, Listening and Media Literacy. Washington, D.C: Author.

National Communication Association. (1998b). Guidelines for Communication Assessment. Washington, D.C.: Au-thor.

National Postsecondary Education Cooperative. (2005). NPEC sourcebook on assessment: Defi nitions and assess-ment methods for communication, leadership, information literacy, quantitative reasons, and quantitative skills.

Northwest Regional Educational Laboratory. (1998). Bibli-ography of assessment alternatives: Oral communication.

Rubin, D. L. & Mead, N.A. (1984). Large scale assessment of oral communication skills: Kindergarten through grade 12. Urbana, IL: ERIC Clearinghouse on Reading and Com-munication Skills.

Rubin, R. B., Welch, S.A., & Buerkel, R. (1995). Perfor-mance-based assessment of high school speech instruc-tion. Communication Education, 44, 30-39.



TABLE: ORAL COMMUNICATION ASSESSMENT INSTRUMENTS FOR P-12

LISTENING

Title Source Context Population Administration Page

Advocacy/Inquiry Skill Inventory

Spectra Publishing Listening Business Unknown 26

Learning to Listen HRDQ. www.hrdq.com

Listening College, business

Unknown 27

Listener Prefer-ence Profi le


Listening Assess-ment for ITBS, Forms K, L & TAP and ITED

The Iowa Tests

Riverside Publishing

Listening Grade 3-12 Oral presentation, respond on ques-tion 7 response sheets

30

Watson-Barker Listening Test

Innolect, Inc., Spectra Publishing

Listening H.S., col-lege, adult

Audio & video tape; respond on question & response sheets

32

COMMUNICATION SKILLS - COMPETENCIES


Communication Competency Assessment Instrument,

High School


Public speak-ing, inter-personal

Grade 9-12 Subjects present talk & engage in guided discus-sions with tester

34

Communication Competency Self-Report Question-naire

R.B. Rubin. (1995) Commu-nication Mono-graphs, 52, 173-185


College 19 item response form. Not recom-mended for high stakes testing

36

Competent Speaker Speech Evaluation Form, 2nd Edition

Sherwyn Mor-reale, National Communication Association

Public speaking

Various 8 item open ended form with descriptors. Not recommended for high stakes testing

38



TABLE: ORAL COMMUNICATION ASSESSMENT INSTRUMENTS FOR P-12

LISTENING


Advocacy/Inquiry Skill Inventory


Learning to Listen HRDQ. www.hrdq.com

Listening College, business

Unknown 27

Listener Prefer-ence Profi le


Listening Assess-ment for ITBS, Forms K, L & TAP and ITED

The Iowa Tests

Riverside Publishing

Listening Grade 3-12 Oral presentation, respond on ques-tion 7 response sheets

30



Listening H.S., col-lege, adult

Audio & video tape; respond on question & response sheets

32

COMMUNICATION SKILLS - COMPETENCIES


Communication Competency Assessment Instrument,

High School



Grade 9-12 Subjects present talk & engage in guided discus-sions with tester

34

Communication Competency Self-Report Question-naire

R.B. Rubin. (1995) Commu-nication Mono-graphs, 52, 173-185


College 19 item response form. Not recom-mended for high stakes testing

36

Competent Speaker Speech Evaluation Form, 2nd Edition

Sherwyn Mor-reale, National Communication Association

Public speaking

Various 8 item open ended form with descriptors. Not recommended for high stakes testing

38



District 214 Speaker Evaluation Form

Illinois Township District 214

Public Speaking

Various 12 item rubric 40

Massachusetts Speaking Assess-ment Criteria

Massachusetts Department of Education

Public speaking

K-12 Individualized presentation of a speech with four competencies

41

Ogden Speaking Rubric

Ogden Elemen-tary Schools, Chicago Public Schools

Public speaking

K-8 5 item rubric 42

Oral Profi ciency/ Second Language Acquisition

Fairfax County, Virginia Public Schools

Public speaking

9-12 5 item rubric 43

Speaking and Listening Assess-ment Project

Illinois State University, Nor-mal, IL

Oral presenta-tions

K-12 Multi-item rubrics individualized by grade level and activity

44

EDUCATIONAL ACHIEVEMENT


Basic Skills Test Harcourt Assessment

Speaking, writing, processing

Business – clerical

Oral discussion, responses on test booklets with scoring key

45

Joliet 3-Minute Speech and Language Screen

Harcourt Assessment

Speaking P-9 Standardized, individualized screening

46



District 214 Speaker Evaluation Form

Illinois Township District 214

Public Speaking

Various 12 item rubric 40

Massachusetts Speaking Assess-ment Criteria

Massachusetts Department of Education

Public speaking

K-12 Individualized presentation of a speech with four competencies

41


Ogden Elemen-tary Schools, Chicago Public Schools

Public speaking

K-8 5 item rubric 42

Oral Profi ciency/ Second Language Acquisition

Fairfax County, Virginia Public Schools

Public speaking

9-12 5 item rubric 43

Speaking and Listening Assess-ment Project

Illinois State University, Nor-mal, IL

Oral presenta-tions

K-12 Multi-item rubrics individualized by grade level and activity

44

EDUCATIONAL ACHIEVEMENT


Basic Skills Test Harcourt Assessment

Speaking, writing, processing

Business – clerical

Oral discussion, responses on test booklets with scoring key

45

Joliet 3-Minute Speech and Language Screen

Harcourt Assessment

Speaking P-9 Standardized, individualized screening

46



Advocacy/Inquiry Skill Inventory (AISI)

Author / Developers: Kittie W. Watson and Larry L. Barker.

Source: Spectra Publishers, Innolect Inc., PO Box 3099, Tega Cay, SC 29708. Tel: 803.396.8500 / Fax: 803.396.8502, http://www.innolectinc.com/products.html.

Context of communication behavior for which intended: Assess preferences for using advocacy and inquiry communication skills.

Academic level for which intended: Varied – designed for occupational applications rather than educational.

Time required: Not indicated.

Description of test: Identifi es four habitual listening responses: People, action, content and time-oriented.

Description of administration and scoring: Self-scoring answer form, preference grid and interpretation sheet provide immediate feedback.

Norm referencing/criterion referencing populations: Adult, career oriented. No other details supplied other than it has been widely used and fi eld tested.

Tests for cultural bias (ethnicity): Not stated.

Validity

Content (face, intrinsic, representativeness): Not fully stated. Advertising says that the form has been used for thousands of respondents.

Predictive: Not stated.

Construct: Not stated.

Reliability

Split-half: Not stated.

Internal consistency: Not stated.

Evaluative reactions

Practicality and usefulness: The scale is easily used; written for general consumption and set up for self-review.

Clarity of instructions: Directions are self-instructive; materials are easy to understand.

Identifi cation of a range of responses or degree of competence: Not available to the reviewer.

Primary strengths: Ease of use; general availability and cost (packet of 10, $50.00 or packet of 100, $400).

Primary weaknesses: Designed primarily for business applications; P-12 or other educational application may require adaptation.

Overall adequacy: Questionable or limited application for P-12.

Reviewer: Douglas Jennings.



Advocacy/Inquiry Skill Inventory (AISI)

Author / Developers: Kittie W. Watson and Larry L. Barker.

Source: Spectra Publishers, Innolect Inc., PO Box 3099, Tega Cay, SC 29708. Tel: 803.396.8500 / Fax: 803.396.8502, http://www.innolectinc.com/products.html.

Context of communication behavior for which intended: Assess preferences for using advocacy and inquiry communication skills.


Time required: Not indicated.

Description of test: Identifi es four habitual listening responses: People, action, content and time-oriented.

Description of administration and scoring: Self-scoring answer form, preference grid and interpretation sheet provide immediate feedback.

Norm referencing/criterion referencing populations: Adult, career oriented. No other details supplied other than it has been widely used and fi eld tested.


Validity

Content (face, intrinsic, representativeness): Not fully stated. Advertising says that the form has been used for thousands of respondents.



Reliability




Practicality and usefulness: The scale is easily used; written for general consumption and set up for self-review.

Clarity of instructions: Directions are self-instructive; materials are easy to understand.

Identifi cation of a range of responses or degree of competence: Not available to the reviewer.

Primary strengths: Ease of use; general availability and cost (packet of 10, $50.00 or packet of 100, $400).

Primary weaknesses: Designed primarily for business applications; P-12 or other educational application may require adaptation.

Overall adequacy: Questionable or limited application for P-12.




Learning to Listen 2nd Edition

Author / Developers: HRDQ research and development team. First edition published in 1998; second edition in 2004.

Source: HRDQ, 2002 Renaissance Boulevard #100, King of Prussia, PA 19406-2756. Phone: 610.279.2002 or 800.633.4533. Fax: 610.279.0524. http://www.hrdq.com/products/learntolisten.htm.

Context of communication behavior for which intended: Listening skills.

Academic level for which intended: Varied – designed for all ages and levels.

Time required: 60 minutes; optional 1 ½ hour workshop available.

Description of test: Focuses on listening behaviors and skills allowing participants to self-assess needs and reme-dial actions. Determines listening effectiveness in three dimensions which were not articulated. Designed to be used as group or individual evaluation. Choice of fi ve responses for each of the 30 statements (almost always to almost never).

Description of administration and scoring: Presents 30 statements about listening behavior during one-on-one conversations on CD-ROM. Includes content, technical information including reliability, as well as validity and normative data.

Norm referencing/criterion referencing populations: Adult, career-oriented.


Validity

Content (face, intrinsic, representativeness): Not stated in promotional materials.



Reliability




Practicality and usefulness: Unknown.

Clarity of instructions: Sample items included defi nitions of key terms; choice of fi ve responses allowed respon-dent a wide range.

Identifi cation of a range of responses or degree of competence: Not stated.

Primary strengths: Claims to measure skills rather than listening styles. Ease of use; general availability.

Primary weaknesses: Designed primarily for business applications; P-12 educational application may require ad-aptation. Insuffi cient information to determine validity. Has respondents self-assess – determining their own level and quality of listening skills.

Overall adequacy: Limited application for P-12, but provides reinforcement of the importance of good listening skills especially in occupational areas.






Source: HRDQ, 2002 Renaissance Boulevard #100, King of Prussia, PA 19406-2756. Phone: 610.279.2002 or 800.633.4533. Fax: 610.279.0524. http://www.hrdq.com/products/learntolisten.htm.








Validity




Reliability













Listener Preference Profi le

Author / Developers: Kittie W. Watson, Larry L. Barker, and James B. Weaver.

Source Spectra Publishers, Innolect Inc., PO Box 3099, Tega Cay, SC 29708. Tel: 803.396.8500 / Fax: 803.396.8502, http://www.innolectinc.com/products.html.

Context of communication behavior for which intended: Essential listening skills in business.


Time required: 15-20 minutes.

Description of test: Identifi es four habitual listening responses: People, action, content and time-oriented. Results may indicate no listening preference (listening avoidance), one listening preference, or multiple listening prefer-ences.

Description of administration and scoring: 20 item, self-scoring answer form, preference grid and interpretation sheet provide immediate feedback. Responses on the LPP range from 1 (always) to 5 (never). Items are arranged such that Items 1-5 represent a People-Oriented Listener, Items 6-10 represent an Action-Oriented Listener, Items 11-15 indicate a Content-Oriented Listener and Items 16-20 indicate a Time-Oriented Listener.

Norm referencing/criterion referencing populations: Adult, career oriented. No other details supplied other than it has been widely used and fi eld tested. Earlier reviews suggested that 40 percent of respondents have two or more listening preferences.

Tests for cultural bias (ethnicity): Gender differences have been documented in listening preferences. Female respondents report stronger people and content listener preferences, while males report action and time listener preferences.

Validity

Content (face, intrinsic, representativeness): Conclusions are supported from research on 56 respondents during three administrations. The analysis suggests distinctively different patterns of listening styles associated with three predominant personality types. Further research defi nes each listening preference by highlighting linkages among listener preferences and three constructs of empathy.



Reliability




Practicality and usefulness: The scale is easily used; written for general consumption.

Clarity of instructions: Instructions are clear.


Primary strengths: Ease of use; general availability and cost (packet of 10, $50.00; other quantities available.

Primary weaknesses: Designed primarily for business applications; P-12 educational application may require ad-aptation. Data supporting validity are not readily available.



Listener Preference Profi le

Author / Developers: Kittie W. Watson, Larry L. Barker, and James B. Weaver.


Context of communication behavior for which intended: Essential listening skills in business.



Description of test: Identifi es four habitual listening responses: People, action, content and time-oriented. Results may indicate no listening preference (listening avoidance), one listening preference, or multiple listening prefer-ences.

Description of administration and scoring: 20 item, self-scoring answer form, preference grid and interpretation sheet provide immediate feedback. Responses on the LPP range from 1 (always) to 5 (never). Items are arranged such that Items 1-5 represent a People-Oriented Listener, Items 6-10 represent an Action-Oriented Listener, Items 11-15 indicate a Content-Oriented Listener and Items 16-20 indicate a Time-Oriented Listener.

Norm referencing/criterion referencing populations: Adult, career oriented. No other details supplied other than it has been widely used and fi eld tested. Earlier reviews suggested that 40 percent of respondents have two or more listening preferences.

Tests for cultural bias (ethnicity): Gender differences have been documented in listening preferences. Female respondents report stronger people and content listener preferences, while males report action and time listener preferences.

Validity

Content (face, intrinsic, representativeness): Conclusions are supported from research on 56 respondents during three administrations. The analysis suggests distinctively different patterns of listening styles associated with three predominant personality types. Further research defi nes each listening preference by highlighting linkages among listener preferences and three constructs of empathy.



Reliability




Practicality and usefulness: The scale is easily used; written for general consumption.



Primary strengths: Ease of use; general availability and cost (packet of 10, $50.00; other quantities available.

Primary weaknesses: Designed primarily for business applications; P-12 educational application may require ad-aptation. Data supporting validity are not readily available.




Additional References

Watson, K. W., Barker, L. & Weaver, J (1992). Development and validation of the Listener Preference Profi le. New Orleans: Tulane University. (ERIC Document Reproductions Service No. ED 348.703).

Reviewer: Douglas Jennings, Judith Dallinger.





Watson, K. W., Barker, L. & Weaver, J (1992). Development and validation of the Listener Preference Profi le. New Orleans: Tulane University. (ERIC Document Reproductions Service No. ED 348.703).

Reviewer: Douglas Jennings, Judith Dallinger.



Listening Assessment for TAP/ITED

Author / Developers: O.M. Haugh, & D. P. Scannel.

Source: Riverside Publishing Company, 425 Spring Lake Drive, Itasca, IL 60143-2079. Telephone: 800.323.9540; Outside US: 1.630.467.7000; Fax: 1.630.467.7192. http://www.riverpub.com.

Context of communication behavior for which intended: Listening.

Academic level for which intended: Grades 3 to 12.


Description of test: Group administered test for listening. Assesses strengths and weaknesses in listening develop-ment of individuals so that effective instruction can be planned. Part of the Iowa Tests. Objectives assessed include: Literal Meaning (short-term memory of details about persons, objects, time, and place); Inferential Meaning (word meaning in context, sequential relations, and drawing conclusions); Speakers Point of View and Purpose (main point of lecture): Differences Between Fact and Opinion; and Detecting Bias.

Description of administration and scoring: 95 items that measure literal meaning, inferential meaning, following directions, visual relationships, numerical/spatial/temporal relations, and the speaker’s purpose, point of view or style. Developmental standard scores, national percentile ranks, local percentile ranks, stanines, and a number of special norms are available. Individual and group raw score information is also provided.

Norm referencing/criterion referencing populations: TP and ITED have separate tests for grades K-3 and 3-12. Normed with the Iowa Tests of Educational Development and Tests of Achievement and Profi ciency. Grade equiva-lents, standard scores, and percentile ranks are available.

Tests for cultural bias (ethnicity): All items have been reviewed with statistical analysis to determine items with differential performance.

Validity

Content (face, intrinsic, representativeness): Items on the various levels of the test relate to common listening ac-tivities. Presentation of items by the teacher is positive because the speaker is familiar, but this may be a limiting factor in generalization of scores across situations.

Predictive: Not evaluated.

Construct: The items directly relate to the listening skills used in school and are categorized according to four lev-els of cognition: knowledge/information, comprehension, application/analysis, and synthesis/evaluation. Criterion-referenced information appears to be directly related to listening and comprehension.

Reliability

Split-half: Not evaluated.

Internal consistency: KR-20 reliability coeffi cients ranged from .83 to .85 for the various levels in a fall adminis-tration and from .82 to .85 in a spring administration.


Practicality and usefulness: The test is easily administered and is one of the few listening tests that directly relates to educational settings.


Identifi cation of a range of responses or degree of competence: Complete analysis is available.



Listening Assessment for TAP/ITED

Author / Developers: O.M. Haugh, & D. P. Scannel.

Source: Riverside Publishing Company, 425 Spring Lake Drive, Itasca, IL 60143-2079. Telephone: 800.323.9540; Outside US: 1.630.467.7000; Fax: 1.630.467.7192. http://www.riverpub.com.


Academic level for which intended: Grades 3 to 12.


Description of test: Group administered test for listening. Assesses strengths and weaknesses in listening develop-ment of individuals so that effective instruction can be planned. Part of the Iowa Tests. Objectives assessed include: Literal Meaning (short-term memory of details about persons, objects, time, and place); Inferential Meaning (word meaning in context, sequential relations, and drawing conclusions); Speakers Point of View and Purpose (main point of lecture): Differences Between Fact and Opinion; and Detecting Bias.

Description of administration and scoring: 95 items that measure literal meaning, inferential meaning, following directions, visual relationships, numerical/spatial/temporal relations, and the speaker’s purpose, point of view or style. Developmental standard scores, national percentile ranks, local percentile ranks, stanines, and a number of special norms are available. Individual and group raw score information is also provided.

Norm referencing/criterion referencing populations: TP and ITED have separate tests for grades K-3 and 3-12. Normed with the Iowa Tests of Educational Development and Tests of Achievement and Profi ciency. Grade equiva-lents, standard scores, and percentile ranks are available.

Tests for cultural bias (ethnicity): All items have been reviewed with statistical analysis to determine items with differential performance.

Validity

Content (face, intrinsic, representativeness): Items on the various levels of the test relate to common listening ac-tivities. Presentation of items by the teacher is positive because the speaker is familiar, but this may be a limiting factor in generalization of scores across situations.

Predictive: Not evaluated.

Construct: The items directly relate to the listening skills used in school and are categorized according to four lev-els of cognition: knowledge/information, comprehension, application/analysis, and synthesis/evaluation. Criterion-referenced information appears to be directly related to listening and comprehension.

Reliability

Split-half: Not evaluated.

Internal consistency: KR-20 reliability coeffi cients ranged from .83 to .85 for the various levels in a fall adminis-tration and from .82 to .85 in a spring administration.


Practicality and usefulness: The test is easily administered and is one of the few listening tests that directly relates to educational settings.


Identifi cation of a range of responses or degree of competence: Complete analysis is available.



Primary strengths: Easily administered norm-referenced measure that provides individual and group averages.

Primary weaknesses: The test has some memory-oriented segments which may not be adequate measures of listen-ing skills.

Overall adequacy: A well-made norm-referenced test.

Reviewers: Ray Fenton; updated by Douglas Jennings.



Primary strengths: Easily administered norm-referenced measure that provides individual and group averages.

Primary weaknesses: The test has some memory-oriented segments which may not be adequate measures of listen-ing skills.

Overall adequacy: A well-made norm-referenced test.

Reviewers: Ray Fenton; updated by Douglas Jennings.




Revised Video Version 2001

Author / Developers: Kittie W. Watson, Larry L. Barker, Charles V. Roberts, and John D. Roberts.



Academic level for which intended: Designed for college students, but acceptable for high school students.

Time required: 30 or 60 minutes depending on chosen format.

Description of test: This is a fi ve part listening test presented on a VHS video tape, self-scoring answer sheets and scoring key. Each part is designed to test a particular type of listening comprehension ability. Part one presents ques-tions and statements to measure listener skill in interpreting message content. Part two tests understanding meaning in conversations by presenting video dialogues. Part three involves listening to a series of brief lectures and tests remembering lecture information. Part four uses questions and/or statements to measure the listener’s skill in inter-preting emotional meaning. Part fi ve measures a listener’s ability to follow instructions.

Description of administration and scoring: Parts one, two and four are designed to measure the listening skills used in brief listening situations. Parts three and fi ve are designed to measure the listening skills used in longer listening situations. Evaluating individual section scores allows test takers to assess specifi c areas where improve-ment may be necessary. General scoring guidelines are presented for characterizing scores from “Very Poor” to “Excellent.”

Norm referencing/criterion referencing populations: Adult, career-oriented. No other details supplied other than it has been widely used and fi eld tested.

Tests for cultural bias (ethnicity): Male and female, black, white, and Asian speakers are included in the source materials. Females score slightly higher on the measure than males.

Validity

Content (face, intrinsic, representativeness): Situations are common to high-school students. Results from early versions of the test were submitted to item analysis. Those items selected were designed to discriminate among effective and ineffective test takers. Items identifi ed as too easy or too diffi cult were eliminated from the fi nal ver-sion.

Predictive: Not stated for K-12 levels. Guide states that data from 65,000 college and adult subjects has been col-lected.

Construct: The instrument is considered as the classic listening test and has been used by a large number of com-munication researchers.

Reliability

Split-half: Not stated for K-12.

Internal consistency: Not stated for K-12. Reliability coeffi cients were determined for the alternate forms of the test using the Kuder Richardson reliability formula. Signifi cant positive correlations were found between Form A (r = >.71) and Form B (r = .68).


Practicality and usefulness: A very practical and easy to administer test that is consistent with various methods of teaching listening skills. Instrument focuses on listening in typical communication settings.




Revised Video Version 2001

Author / Developers: Kittie W. Watson, Larry L. Barker, Charles V. Roberts, and John D. Roberts.



Academic level for which intended: Designed for college students, but acceptable for high school students.

Time required: 30 or 60 minutes depending on chosen format.

Description of test: This is a fi ve part listening test presented on a VHS video tape, self-scoring answer sheets and scoring key. Each part is designed to test a particular type of listening comprehension ability. Part one presents ques-tions and statements to measure listener skill in interpreting message content. Part two tests understanding meaning in conversations by presenting video dialogues. Part three involves listening to a series of brief lectures and tests remembering lecture information. Part four uses questions and/or statements to measure the listener’s skill in inter-preting emotional meaning. Part fi ve measures a listener’s ability to follow instructions.

Description of administration and scoring: Parts one, two and four are designed to measure the listening skills used in brief listening situations. Parts three and fi ve are designed to measure the listening skills used in longer listening situations. Evaluating individual section scores allows test takers to assess specifi c areas where improve-ment may be necessary. General scoring guidelines are presented for characterizing scores from “Very Poor” to “Excellent.”

Norm referencing/criterion referencing populations: Adult, career-oriented. No other details supplied other than it has been widely used and fi eld tested.

Tests for cultural bias (ethnicity): Male and female, black, white, and Asian speakers are included in the source materials. Females score slightly higher on the measure than males.

Validity

Content (face, intrinsic, representativeness): Situations are common to high-school students. Results from early versions of the test were submitted to item analysis. Those items selected were designed to discriminate among effective and ineffective test takers. Items identifi ed as too easy or too diffi cult were eliminated from the fi nal ver-sion.

Predictive: Not stated for K-12 levels. Guide states that data from 65,000 college and adult subjects has been col-lected.

Construct: The instrument is considered as the classic listening test and has been used by a large number of com-munication researchers.

Reliability

Split-half: Not stated for K-12.

Internal consistency: Not stated for K-12. Reliability coeffi cients were determined for the alternate forms of the test using the Kuder Richardson reliability formula. Signifi cant positive correlations were found between Form A (r = >.71) and Form B (r = .68).


Practicality and usefulness: A very practical and easy to administer test that is consistent with various methods of teaching listening skills. Instrument focuses on listening in typical communication settings.



Clarity of instructions: Instructions are clear and precise and designed to be used by individuals without previous testing experience.

Identifi cation of a range of responses or degree of competence: National averages and defi ned ranges of compe-tence are provided in the instruction booklet. Sub-scores are easily determined.

Primary strengths: One of the few listening tests that have been updated (2001). Test is easy to use and has wide availability. The WBLT is currently being administered in over 40 states and 10 different countries. Video tape for-mat allows general classroom use.

Primary weaknesses: Some portions of the test rely on memory rather than listening ability. The test guide states that a K-6 listening assessment instrument still needs to be created.

Overall adequacy: The Watson-Barker test is still considered the classic listening test and the recent video updat-ing will help the test maintain that standing. While somewhat linked to memory, tests (and, thus, some segments of Watson-Barker) that rely on student memory may not be true indicators of one’s listening ability.


Barker, L.L., & Watson, K. W. (200). Listen Up!. New York: St. Martin Press. New York.

Roberts, C. V. (1998). The validation of listening tests: Cutting the Gordian Knot. Journal of the International Lis-tening Association, 2. 1-19.

Vander Kooi, Daryl (1998). The Predictive Value of the Watson-Barker Listening Test. International Listening As-sociation.




Clarity of instructions: Instructions are clear and precise and designed to be used by individuals without previous testing experience.

Identifi cation of a range of responses or degree of competence: National averages and defi ned ranges of compe-tence are provided in the instruction booklet. Sub-scores are easily determined.

Primary strengths: One of the few listening tests that have been updated (2001). Test is easy to use and has wide availability. The WBLT is currently being administered in over 40 states and 10 different countries. Video tape for-mat allows general classroom use.

Primary weaknesses: Some portions of the test rely on memory rather than listening ability. The test guide states that a K-6 listening assessment instrument still needs to be created.

Overall adequacy: The Watson-Barker test is still considered the classic listening test and the recent video updat-ing will help the test maintain that standing. While somewhat linked to memory, tests (and, thus, some segments of Watson-Barker) that rely on student memory may not be true indicators of one’s listening ability.


Barker, L.L., & Watson, K. W. (200). Listen Up!. New York: St. Martin Press. New York.

Roberts, C. V. (1998). The validation of listening tests: Cutting the Gordian Knot. Journal of the International Lis-tening Association, 2. 1-19.

Vander Kooi, Daryl (1998). The Predictive Value of the Watson-Barker Listening Test. International Listening As-sociation.




Communication Competency Assessment Instrument – High School

Author / Developers: R. B. Rubin.


Context of communication behavior for which intended: General speaking and interaction.

Academic level for which intended: Versions are available for high school (CCAI-HS) and college students (CCAI).

Time required: 15 to 30 minutes per student.

Description of test: Oral communication assessment is based on communication competencies identifi ed by the National Communication Association. Students present a three-minute impromptu talk and engage in a guided conversation with the teacher/rater. Skills assessed include use of understandable pronunciation, appropriate tone of voice, clarity, distinguishing between information and persuasive messages, presenting thesis and main points clearly and concisely, expressing and defending a point of view, recognizing when others do not understand.

Description of administration and scoring: Students are scored on fi ve-point criterion referenced scales relating degrees of competence.

Norm referencing/criterion referencing populations: Norms are based on the NCA guidelines provided in Speak-ing and Listening Competencies for K-12 Students.

Tests for cultural bias (ethnicity): CCAI performance by a group of college students found no signifi cant differ-ence in attaining a satisfactory score (3.00) by gender or minority group membership.

Validity

Content (face, intrinsic, representativeness): Individual competencies are related to important skills for success, identify by NCA. Speech communication faculty members were able to sort individual questions into the main NCA competencies.

Predictive: Not reviewed.

Construct: Not reviewed.

Reliability

Split-half: Split-half reliability for the CCAI-HS was .66.

Internal consistency: Alpha reliability for the CCAI-HS was .77. Four graduate students attained .80 interrater reli-ability after two hours of training.


Practicality and usefulness: Commonly used instrument.


Identifi cation of a range of responses or degree of competence: Scoring ranges have high face validity.

Primary strengths: Ease of use; general availability and appropriateness for high school use.

Primary weaknesses: Materials are somewhat dated (1994). Scales are not designed as rubrics, but remain open to some interpretation by the rater.

Overall adequacy: Appropriate application to high school speaking situations.



Communication Competency Assessment Instrument – High School



Context of communication behavior for which intended: General speaking and interaction.

Academic level for which intended: Versions are available for high school (CCAI-HS) and college students (CCAI).

Time required: 15 to 30 minutes per student.

Description of test: Oral communication assessment is based on communication competencies identifi ed by the National Communication Association. Students present a three-minute impromptu talk and engage in a guided conversation with the teacher/rater. Skills assessed include use of understandable pronunciation, appropriate tone of voice, clarity, distinguishing between information and persuasive messages, presenting thesis and main points clearly and concisely, expressing and defending a point of view, recognizing when others do not understand.

Description of administration and scoring: Students are scored on fi ve-point criterion referenced scales relating degrees of competence.

Norm referencing/criterion referencing populations: Norms are based on the NCA guidelines provided in Speak-ing and Listening Competencies for K-12 Students.

Tests for cultural bias (ethnicity): CCAI performance by a group of college students found no signifi cant differ-ence in attaining a satisfactory score (3.00) by gender or minority group membership.

Validity

Content (face, intrinsic, representativeness): Individual competencies are related to important skills for success, identify by NCA. Speech communication faculty members were able to sort individual questions into the main NCA competencies.



Reliability

Split-half: Split-half reliability for the CCAI-HS was .66.

Internal consistency: Alpha reliability for the CCAI-HS was .77. Four graduate students attained .80 interrater reli-ability after two hours of training.




Identifi cation of a range of responses or degree of competence: Scoring ranges have high face validity.

Primary strengths: Ease of use; general availability and appropriateness for high school use.

Primary weaknesses: Materials are somewhat dated (1994). Scales are not designed as rubrics, but remain open to some interpretation by the rater.

Overall adequacy: Appropriate application to high school speaking situations.



Additional ReferencesRubin, R. B. (1994). The CCAI-HS: A new assessment tool. The Speech Communication Teacher, 8(2), 14-15.Rubin, R. B. (1985). Validity of the Communication Competency Assessment Instrument. Communication Monographs, 52,

73-185.

Reviewer: Neil O’Leary; updated by Douglas Jennings



Additional ReferencesRubin, R. B. (1994). The CCAI-HS: A new assessment tool. The Speech Communication Teacher, 8(2), 14-15.Rubin, R. B. (1985). Validity of the Communication Competency Assessment Instrument. Communication Monographs, 52,

73-185.

Reviewer: Neil O’Leary; updated by Douglas Jennings



Communication Competency Self-Report Questionnaire


Address: Dr. Rebecca B. Rubin, c/o School of Communication Studies, Kent State University P.O. Box 5190, Kent, OH 44242; [email protected].

Source: Referenced in Communication Monographs, 52, 173-185.

Context of communication behavior for which intended: Overall self-impression of communication competence (various situations).

Academic level for which intended: Primarily designed for college, but useable in upper level high school situa-tions.


Description of test: The CCSR includes 19 statements related to an individual’s perception of their public speaking, interaction, and listening abilities. Respondents indicate on a fi ve-point Likert-type scale the degree to which each statement describes their abilities.

Description of administration and scoring: The negatively worded items are reversed, and the 19 times are to-taled. The article does not offer an interpretation of the scores. Means for each of the 19 items are provided.

Norm referencing/criterion referencing populations: College students (N=41) were used in developing the in-strument.

Tests for cultural bias (ethnicity): Information not available.

Validity

Content (face, intrinsic, representativeness): The CCSR was developed as a parallel to the items on the Communi-cation Competency Assessment Instrument, which refl ects the 19 functional communication competencies devel-oped by NCA members.


Construct: Only two of the items on the CCSR were signifi cantly correlated with the CCAI. Subject tended to over-estimate their own abilities. Instructor rating of the respondents tended to be closer to the respondents’ self-ratings than did the ratings of unknown observers.

Reliability

Split-half: Not reviewed.

Internal consistency: An alpha coeffi cient of .87 was reported.


Practicality and usefulness: The 19 items provide information about the subject’s perception of communication abilities in several situations. As a pre/post measure for a course or major, it may help to identify specifi c changes in perceptions.

Clarity of instructions: Instructions are very clear.

Identifi cation of a range of responses or degree of competence: Administration and scoring are highly standard-ized.

Primary strengths: The CCSR helps subjects consider how their communication may change in particular situa-tions. It is very quick and easy to administer.



Communication Competency Self-Report Questionnaire


Address: Dr. Rebecca B. Rubin, c/o School of Communication Studies, Kent State University P.O. Box 5190, Kent, OH 44242; [email protected].

Source: Referenced in Communication Monographs, 52, 173-185.

Context of communication behavior for which intended: Overall self-impression of communication competence (various situations).

Academic level for which intended: Primarily designed for college, but useable in upper level high school situa-tions.


Description of test: The CCSR includes 19 statements related to an individual’s perception of their public speaking, interaction, and listening abilities. Respondents indicate on a fi ve-point Likert-type scale the degree to which each statement describes their abilities.

Description of administration and scoring: The negatively worded items are reversed, and the 19 times are to-taled. The article does not offer an interpretation of the scores. Means for each of the 19 items are provided.

Norm referencing/criterion referencing populations: College students (N=41) were used in developing the in-strument.

Tests for cultural bias (ethnicity): Information not available.

Validity

Content (face, intrinsic, representativeness): The CCSR was developed as a parallel to the items on the Communi-cation Competency Assessment Instrument, which refl ects the 19 functional communication competencies devel-oped by NCA members.


Construct: Only two of the items on the CCSR were signifi cantly correlated with the CCAI. Subject tended to over-estimate their own abilities. Instructor rating of the respondents tended to be closer to the respondents’ self-ratings than did the ratings of unknown observers.

Reliability




Practicality and usefulness: The 19 items provide information about the subject’s perception of communication abilities in several situations. As a pre/post measure for a course or major, it may help to identify specifi c changes in perceptions.

Clarity of instructions: Instructions are very clear.

Identifi cation of a range of responses or degree of competence: Administration and scoring are highly standard-ized.

Primary strengths: The CCSR helps subjects consider how their communication may change in particular situa-tions. It is very quick and easy to administer.



Primary weaknesses: As validity studies indicate, self-perception may not be an accurate measure of communica-tion competence. This particular instrument has not been very thoroughly tested. It was primarily designed to study the CCAI.

Overall adequacy: While it would be diffi cult to generalize too much from this instrument, it would serve as a good beginning for considering communication competence. When coupled with other measures, it might be very effective.

Reviewer: Ellen A. Hay



Primary weaknesses: As validity studies indicate, self-perception may not be an accurate measure of communica-tion competence. This particular instrument has not been very thoroughly tested. It was primarily designed to study the CCAI.

Overall adequacy: While it would be diffi cult to generalize too much from this instrument, it would serve as a good beginning for considering communication competence. When coupled with other measures, it might be very effective.




The Competent Speaker Speech Evaluation Form, 2nd Edition

Authors/Developers: S.P. Morreale, M.R. Moore, D.S. Tatum & L. Webster

Source: National Communication Association, 1765 N Street, NW, Washington, D.C., 20036, www.natcom.org

Context or communication behavior for which intended: Public Speaking

Academic level for which intended: College

Time required: Training requires approximately two hours. Administration of the instrument requires the length of the assigned speech plus an additional 10 minutes.

Description: This instrument presents a standardized form and approach for evaluating public speaking behaviors. It is based upon eight competencies, four related to preparation and four related to delivery. The second edition in-cludes a holistic form as well the original atomistic form. In the newest, both electronic and non-electronic sources are included in the evaluation of supporting material. The manual includes a discussion of each competency and an explanation of how each would be demonstrated at excellent, satisfactory, and unsatisfactory levels along with detailed instruction on the preparation of a training video.

Description of scoring: After receiving training, an evaluator rates student speeches on each of the eight competen-cies. Points are assigned. A rating of unsatisfactory receives one point, satisfactory receives two points, and excel-lent receives three points. Scores range from 8-24. Users can set their own standards for determining competency. As explained in the training manual, scoring can also be adjusted to facilitate attention to particular competencies.

Norm referencing/criterion referencing populations: Twelve speech communication educators, ten gradu-ate teaching assistants and three community college instructors provided referencing scores for the three anchor speeches along with the other sample speeches from a training tape.

Cultural/gender bias: Two studies were conducted to investigate bias. In one study minority students rated the sample speeches. No differences were found between their ratings and the ones reported by the speech communi-cation educators. When the instrument was used in a speech class with 260 students, no signifi cant differences in scores were found among students from different cultural groups. No signifi cant differences were reported between men and women.

Validity

Concurrent: A negative correlation was reported between this instrument and the six public speaking items on the Personal Report of Communication Apprehension, indicating that better public speakers had lower apprehension. A positive correlation was reported with the seven public speaking items on the Communication Competency As-sessment Instrument.

Face/Content: Developers conducted an extensive literature review when determining appropriate competencies and criteria. A panel of 11 communication educators were involved in the fi nal determination.

Reliability

Interrater reliability: High inter-rater reliability between .76-.84 was reported after training of evaluators.

Evaluative Reactions

Practicality and usefulness: The developers suggest many possible uses for this instrument. Using this form to evaluate all student speeches and maintaining records of this use will be invaluable for departments that must dem-onstrate that students are learning to speak in public. It is currently used by a number of colleges and universities in their institutional assessment process.




Authors/Developers: S.P. Morreale, M.R. Moore, D.S. Tatum & L. Webster





Description: This instrument presents a standardized form and approach for evaluating public speaking behaviors. It is based upon eight competencies, four related to preparation and four related to delivery. The second edition in-cludes a holistic form as well the original atomistic form. In the newest, both electronic and non-electronic sources are included in the evaluation of supporting material. The manual includes a discussion of each competency and an explanation of how each would be demonstrated at excellent, satisfactory, and unsatisfactory levels along with detailed instruction on the preparation of a training video.




Validity



Reliability






Clarity of instructions: Competencies are clear as are rating procedures. Description of Rasch Analysis is not entirely clear but not really necessary to the use of the instrument.

Degree of standardization in administration and evaluation of responses: Each competency is fully explained to allow for optimal standardization when different evaluators use the instrument. The second edition includes more detailed instruction on creating a training tape and establishing inter-reliability. In-structions are also included for various scoring options.

Identifi cation of a range of responses or degrees of competence: Assigning the label of “competent” is done on a scale.

Primary strengths: This is a clear, simple way to bring uniformity to speech evaluations.

Primary weaknesses: The instrument may not highlight thinking, reasoning and evidence as substantial components of speech preparation and presentation. As we focus more on the connection between critical thinking and communication, this link may need to be strengthened. More space for written feedback is also needed.

Overall adequacy: The Competent Speaker will be a valuable tool for many department. It could be par-ticularly effective for institutions which offer multiple sections of the public speaking course because it allows for standardized evaluation of the speeches.













District 214 Speaking Rubric

Author / Developers: Illinois Township District 214.

Source: Chicago Public Schools via their intranet site: intranet.cps.k12.il.us/Assessments/Ideas_and_Rubrics/Ru-bric_Bank/SpeakingRubrics.pdf.

Context of communication behavior for which intended: Public speaking skills.

Academic level for which intended: Middle and high school students.

Time required: Varies according to assignment.

Description of test: This is a rubric rather than a test. Instructors can adapt the scales to evaluate various middle school and high school oral presentations.

Description of administration and scoring: The form identifi es 12 public speaking competencies including ex-pressiveness, clarity, audibility, eye contact, bodily movement, organization-order, focus, transitions, reasoning, support and elaboration, adaptation to audience, and integration. Each scale has six points, but not all points are defi ned specifi cally. A score of 1 is assigned when the attribute being measured is absent. Scores of 2-3 are assigned when the attribute is developing. Scores of 4-5 indicate adequate performance. A score of 6 is assigned when the attribute is fully developed.

Norm referencing/criterion referencing populations: Not stated.

Tests for cultural bias (ethnicity): Not evaluated.

Validity

Content (face, intrinsic, representativeness): Not available.

Predictive: Not available.

Construct: Not available.

Reliability

Split-half: Not available.

Internal consistency: Not available.


Practicality and usefulness: The scales have been used district-wide.

Clarity of instructions: Form is self-explanatory.

Identifi cation of a range of responses or degree of competence: Raters use a six-item response form. Behavioral differences differentiating speaker performance for each rating category are partially provided. Inter-rater reliability is uncertain due to the partial defi nitions of desired behaviors.

Primary strengths: Utilization of 12 competencies which coincide with those defi ned by the National Communica-tion Association. Form is relatively easy to use and is available via the web site.

Primary weaknesses: Scoring differentiation is only defi ned for some of the rating items.

Overall adequacy: Appropriate application to middle school and high school speaking situations for common classroom assignments. Not appropriate for standardized assessments for comparisons between groups of people.

Reviewer: Douglas Jennings



District 214 Speaking Rubric

Author / Developers: Illinois Township District 214.

Source: Chicago Public Schools via their intranet site: intranet.cps.k12.il.us/Assessments/Ideas_and_Rubrics/Ru-bric_Bank/SpeakingRubrics.pdf.


Academic level for which intended: Middle and high school students.


Description of test: This is a rubric rather than a test. Instructors can adapt the scales to evaluate various middle school and high school oral presentations.

Description of administration and scoring: The form identifi es 12 public speaking competencies including ex-pressiveness, clarity, audibility, eye contact, bodily movement, organization-order, focus, transitions, reasoning, support and elaboration, adaptation to audience, and integration. Each scale has six points, but not all points are defi ned specifi cally. A score of 1 is assigned when the attribute being measured is absent. Scores of 2-3 are assigned when the attribute is developing. Scores of 4-5 indicate adequate performance. A score of 6 is assigned when the attribute is fully developed.



Validity




Reliability




Practicality and usefulness: The scales have been used district-wide.


Identifi cation of a range of responses or degree of competence: Raters use a six-item response form. Behavioral differences differentiating speaker performance for each rating category are partially provided. Inter-rater reliability is uncertain due to the partial defi nitions of desired behaviors.

Primary strengths: Utilization of 12 competencies which coincide with those defi ned by the National Communica-tion Association. Form is relatively easy to use and is available via the web site.






Massachusetts Speaking Assessment Criteria

Author / Developers: Massachusetts Department of Education Assessment of Basic Skills, Speaking Assessment Rating Guide.

Source: Massachusetts Department of Education. 350 Main Street, Malden, MA 02148-5023, 781-338-3000.

Also available from Chicago Public Schools via their Intranet site: intranet.cps.k12.il.us/Assessments/Ideas_and_Rubrics/Rubric_Bank/SpeakingRubrics.pdf.


Academic level for which intended: High school students.


Description of test: This is a multi-scale rubric rather than a test. Instructors can adapt the scales to evaluate various public speaking assignments.

Description of administration and scoring: The form identifi es four public speaking competencies and scales in-cluding content, delivery, organization and language. Raters have a choice of four categories to describe the student performance within each competency: Superior, adequate, minimal, and inadequate..



Validity




Reliability





Clarity of instructions: Form is self-explanatory and includes indicators and descriptors of desired behavior.

Identifi cation of a range of responses or degree of competence: Raters use a four-item scale. Behavioral differ-ences differentiating speaker performance for each rating category are provided.

Primary strengths: Ease of use; general availability and appropriateness for high school use. Scales are rubrics.

Primary weaknesses: The limited number of competencies and scales limits the range of coverage and may lose some of the competencies identifi ed for P-12 students.

Overall adequacy: Appropriate application to high school speaking situations for common classroom assignments. Not appropriate for standardized assessments for comparisons between groups of people.




Massachusetts Speaking Assessment Criteria

Author / Developers: Massachusetts Department of Education Assessment of Basic Skills, Speaking Assessment Rating Guide.

Source: Massachusetts Department of Education. 350 Main Street, Malden, MA 02148-5023, 781-338-3000.

Also available from Chicago Public Schools via their Intranet site: intranet.cps.k12.il.us/Assessments/Ideas_and_Rubrics/Rubric_Bank/SpeakingRubrics.pdf.


Academic level for which intended: High school students.



Description of administration and scoring: The form identifi es four public speaking competencies and scales in-cluding content, delivery, organization and language. Raters have a choice of four categories to describe the student performance within each competency: Superior, adequate, minimal, and inadequate..



Validity




Reliability






Identifi cation of a range of responses or degree of competence: Raters use a four-item scale. Behavioral differ-ences differentiating speaker performance for each rating category are provided.


Primary weaknesses: The limited number of competencies and scales limits the range of coverage and may lose some of the competencies identifi ed for P-12 students.






Author / Developers: Ogden Elementary School, Chicago, IL.

Source: Chicago Public Schools via their Intranet site: intranet.cps.k12.il.us/Assessments/Ideas_and_Rubrics/Rubric_Bank/SpeakingRubrics.pdf.


Academic level for which intended: Elementary students.


Description of test: This is a holistic rubric rather than a test. Instructors can adapt the scales to evaluate various elementary oral presentations.

Description of administration and scoring: The form identifi es fi ve degrees of competencies within fi ve areas.



Validity




Reliability





Clarity of instructions: Form is very simple and self-explanatory.

Identifi cation of a range of responses or degree of competence: Raters use a fi ve-item response form. Behavioral differences differentiating speaker performance for each rating category are provided.

Primary strengths: Ease of use; general availability and appropriateness for elementary students.

Primary weaknesses: Form uses only a few communication variables.

Overall adequacy: Appropriate application to elementary speaking situations for common classroom assignments. Not appropriate for standardized assessments for comparisons between groups of people.





Author / Developers: Ogden Elementary School, Chicago, IL.

Source: Chicago Public Schools via their Intranet site: intranet.cps.k12.il.us/Assessments/Ideas_and_Rubrics/Rubric_Bank/SpeakingRubrics.pdf.


Academic level for which intended: Elementary students.


Description of test: This is a holistic rubric rather than a test. Instructors can adapt the scales to evaluate various elementary oral presentations.

Description of administration and scoring: The form identifi es fi ve degrees of competencies within fi ve areas.



Validity




Reliability





Clarity of instructions: Form is very simple and self-explanatory.

Identifi cation of a range of responses or degree of competence: Raters use a fi ve-item response form. Behavioral differences differentiating speaker performance for each rating category are provided.

Primary strengths: Ease of use; general availability and appropriateness for elementary students.

Primary weaknesses: Form uses only a few communication variables.

Overall adequacy: Appropriate application to elementary speaking situations for common classroom assignments. Not appropriate for standardized assessments for comparisons between groups of people.




Oral Profi ciency/Second Language Acquisition

Author / Developers: Fairfax County Public Schools

Source: Perlman, Carole. The CPS Performance Assessment Idea Book, November 1994. Chicago Public Schools, 1819 W. Pershing Rd., Chicago, IL 60609


Academic level for which intended: High school students and ESL students.



Description of administration and scoring: The form identifi es fi ve public speaking competencies and scales including comprehension, communication, fl uency, structure, and vocabulary. Raters have a choice of six rating categories which are defi ned within each scale.



Validity




Reliability






Identifi cation of a range of responses or degree of competence: Raters use a four- or six-item scale. Behavioral differences differentiating speaker performance for each rating category are provided.


Primary weaknesses: Form uses only a few communication variables. The limited number of competencies and scales limits the range of coverage and may lose some of the competencies identifi ed for P-12 students.





Oral Profi ciency/Second Language Acquisition

Author / Developers: Fairfax County Public Schools

Source: Perlman, Carole. The CPS Performance Assessment Idea Book, November 1994. Chicago Public Schools, 1819 W. Pershing Rd., Chicago, IL 60609


Academic level for which intended: High school students and ESL students.



Description of administration and scoring: The form identifi es fi ve public speaking competencies and scales including comprehension, communication, fl uency, structure, and vocabulary. Raters have a choice of six rating categories which are defi ned within each scale.



Validity




Reliability






Identifi cation of a range of responses or degree of competence: Raters use a four- or six-item scale. Behavioral differences differentiating speaker performance for each rating category are provided.


Primary weaknesses: Form uses only a few communication variables. The limited number of competencies and scales limits the range of coverage and may lose some of the competencies identifi ed for P-12 students.





Speaking and Listening Assessment Project

Author / Developers: Illinois Speech and Theatre Association and the School of Communication, Illinois State University.

Source: Douglas K. Jennings, School of Communication, Illinois State University. 309.438.2872. [email protected]. Speaking and Listening Assessment Project web site: http://lilt.ilstu.edu/cslaml/.

Context of communication behavior for which intended: Various speaking and listening skills.

Academic level for which intended: P-12 students.


Description of test: This is a series of rubrics that were designed by P-12 teachers to be used in a variety of class speaking/listening situations.

Description of administration and scoring: Each rubric is grade-specifi c from pre-school to high school and is designed to assess skills and competencies identifi ed for that specifi c grade level. Raters use a 3 point or four point scale that identifi es specifi c behaviors related to the presenter’s degree of competence.

Norm referencing/criterion referencing populations: Developed by P-12 teachers throughout Illinois and adapted throughout United States.


Validity

Content (face, intrinsic, representativeness): Forms have been tested in a large number of classrooms.



Reliability




Practicality and usefulness: Specifi c indicators for each criterion item are clearly identifi ed as to what behaviors the presenter has to accomplish to earn a specifi c score.


Identifi cation of a range of responses or degree of competence: Raters use a three or four item response form based on the grade level. Preschool through second grade use a three-item scale and grade three and up use a four-item scale. Behavioral expectations of speaker performance for each rating category are fully provided.

Primary strengths: Utilization of fi ve to 25 competencies which coincide with those defi ned by the National Commu-nication Association and most state speech associations. Forms and rubrics are grade specifi c and demonstrate growth in sophistication and knowledge as they progress. Form is relatively easy to use and is available via the Web site.






Speaking and Listening Assessment Project

Author / Developers: Illinois Speech and Theatre Association and the School of Communication, Illinois State University.

Source: Douglas K. Jennings, School of Communication, Illinois State University. 309.438.2872. [email protected]. Speaking and Listening Assessment Project web site: http://lilt.ilstu.edu/cslaml/.

Context of communication behavior for which intended: Various speaking and listening skills.

Academic level for which intended: P-12 students.


Description of test: This is a series of rubrics that were designed by P-12 teachers to be used in a variety of class speaking/listening situations.

Description of administration and scoring: Each rubric is grade-specifi c from pre-school to high school and is designed to assess skills and competencies identifi ed for that specifi c grade level. Raters use a 3 point or four point scale that identifi es specifi c behaviors related to the presenter’s degree of competence.

Norm referencing/criterion referencing populations: Developed by P-12 teachers throughout Illinois and adapted throughout United States.


Validity

Content (face, intrinsic, representativeness): Forms have been tested in a large number of classrooms.



Reliability




Practicality and usefulness: Specifi c indicators for each criterion item are clearly identifi ed as to what behaviors the presenter has to accomplish to earn a specifi c score.


Identifi cation of a range of responses or degree of competence: Raters use a three or four item response form based on the grade level. Preschool through second grade use a three-item scale and grade three and up use a four-item scale. Behavioral expectations of speaker performance for each rating category are fully provided.

Primary strengths: Utilization of fi ve to 25 competencies which coincide with those defi ned by the National Commu-nication Association and most state speech associations. Forms and rubrics are grade specifi c and demonstrate growth in sophistication and knowledge as they progress. Form is relatively easy to use and is available via the Web site.






Basic Skills Test

Author / Developers: William W. Ruch, Allen N. Shub, Sheryl Moinat, and David Dye.

Source: Harcourt Assessment, http://harcourtassessment.com

Context of communication behavior for which intended: Screen applicants for their ability to perform essential offi ce-related functions.

Academic level for which intended: 4th grade.

Time required: Five minutes per subtest, 30 minutes for the complete battery.

Description of test: A series of brief subtests that measure skills and abilities essential for clerical work perfor-mance such as reading and language, computation, problem-solving, memory, reading and the ability to follow written and oral directions.

Description of administration and scoring: Not stated.

Norm referencing/criterion referencing populations: Developed based on nationwide task analysis involving almost 13,000 clerical workers.


Validity

Content (face, intrinsic, representativeness): Not reviewed.



Reliability


Internal consistency: Not reviewed.


Practicality and usefulness: Not stated.

Clarity of instructions: Not reviewed.

Identifi cation of a range of responses or degree of competence: Not stated

Primary strengths: Uncertain of use despite the statement regarding 4th grade application.

Primary weaknesses: Appropriate for clerical functions only.

Overall adequacy: Appropriate for clerical functions only.




Basic Skills Test

Author / Developers: William W. Ruch, Allen N. Shub, Sheryl Moinat, and David Dye.

Source: Harcourt Assessment, http://harcourtassessment.com

Context of communication behavior for which intended: Screen applicants for their ability to perform essential offi ce-related functions.

Academic level for which intended: 4th grade.

Time required: Five minutes per subtest, 30 minutes for the complete battery.

Description of test: A series of brief subtests that measure skills and abilities essential for clerical work perfor-mance such as reading and language, computation, problem-solving, memory, reading and the ability to follow written and oral directions.

Description of administration and scoring: Not stated.

Norm referencing/criterion referencing populations: Developed based on nationwide task analysis involving almost 13,000 clerical workers.


Validity

Content (face, intrinsic, representativeness): Not reviewed.



Reliability


Internal consistency: Not reviewed.


Practicality and usefulness: Not stated.

Clarity of instructions: Not reviewed.

Identifi cation of a range of responses or degree of competence: Not stated

Primary strengths: Uncertain of use despite the statement regarding 4th grade application.

Primary weaknesses: Appropriate for clerical functions only.

Overall adequacy: Appropriate for clerical functions only.




Joliet 3-Minute Speech and Language Screen (Revised)

Author / Developers: Mary C. Kinzler and Constance Cowing Johnson.

Source: Harcourt Assessment. Http://harcourtassessment.com

Context of communication behavior for which intended: Identify clients with suspected language problems such as phonology, grammar, and semantics.

Academic level for which intended: Preschool Test: 2 ½ to 4 ½ years.

Time required: three minutes per student.

Description of test: An individually administered screening measure of vocabulary, grammar, phonology, voice, and fl uency designed to identify students who may need additional assessment and review for speech and language services. The test includes eight vocabulary plates for each grade (24) and 10 evoked sentences. Students are asked to point to illustrations of objects for the listening/vocabulary assessment and to repeat 10 sentences. The teacher or Speech and Hearing Specialist assesses each student relative to specifi c grade level criteria and identifi es students falling below the criterion score as defi cient.

Description of administration and scoring: A scoring sheet is provided and errors are recorded for each tested item. Some sentences allow for identifi cation of up to three specifi c phonological errors. Error scores are compared to suggested cutoff scores. Directions indicate that students may be retested.

Norm referencing/criterion referencing populations: The test was standardized on 2,587 students in grades K, 2, and 5 in eight public and parochial schools in Joliet, Illinois. A randomly selected subsample of 586 was used to establish cutoff scores. Students were screened to assure that there were no hearing disorders and that they were mainstreamed for the greater part of the school day.

Tests for cultural bias (ethnicity): Some differences were found. There is extended discussion of nonstandard dialects, but only some discussion of acceptable and unacceptable deviations from common pronunciation.

Validity

Content (face, intrinsic, representativeness): Activities are clearly related to school and home communication be-haviors. There is emphasis on interpretation and generation of speech.

Predictive: Pass/Fail categorizations were compared to mean differences in PPVT and Carrow Elicited Language Inventory scores for grade and ethnic groups. Gender was found to be a signifi cant variable related to vocabulary but not grammar.


Reliability


Internal consistency: No signifi cant difference was found in the performance of students over a 4-week period. No inter-relater reliability is provided.


Practicality and usefulness: Highly useful for screening students in early grades. The manual provides the basis for a discussion of fl uency and listening vocabulary with parents.


Identifi cation of a range of responses or degree of competence: Recommended criteria for classifi cation appear to be adequate.



Joliet 3-Minute Speech and Language Screen (Revised)

Author / Developers: Mary C. Kinzler and Constance Cowing Johnson.

Source: Harcourt Assessment. Http://harcourtassessment.com

Context of communication behavior for which intended: Identify clients with suspected language problems such as phonology, grammar, and semantics.

Academic level for which intended: Preschool Test: 2 ½ to 4 ½ years.

Time required: three minutes per student.

Description of test: An individually administered screening measure of vocabulary, grammar, phonology, voice, and fl uency designed to identify students who may need additional assessment and review for speech and language services. The test includes eight vocabulary plates for each grade (24) and 10 evoked sentences. Students are asked to point to illustrations of objects for the listening/vocabulary assessment and to repeat 10 sentences. The teacher or Speech and Hearing Specialist assesses each student relative to specifi c grade level criteria and identifi es students falling below the criterion score as defi cient.

Description of administration and scoring: A scoring sheet is provided and errors are recorded for each tested item. Some sentences allow for identifi cation of up to three specifi c phonological errors. Error scores are compared to suggested cutoff scores. Directions indicate that students may be retested.

Norm referencing/criterion referencing populations: The test was standardized on 2,587 students in grades K, 2, and 5 in eight public and parochial schools in Joliet, Illinois. A randomly selected subsample of 586 was used to establish cutoff scores. Students were screened to assure that there were no hearing disorders and that they were mainstreamed for the greater part of the school day.

Tests for cultural bias (ethnicity): Some differences were found. There is extended discussion of nonstandard dialects, but only some discussion of acceptable and unacceptable deviations from common pronunciation.

Validity

Content (face, intrinsic, representativeness): Activities are clearly related to school and home communication be-haviors. There is emphasis on interpretation and generation of speech.

Predictive: Pass/Fail categorizations were compared to mean differences in PPVT and Carrow Elicited Language Inventory scores for grade and ethnic groups. Gender was found to be a signifi cant variable related to vocabulary but not grammar.


Reliability


Internal consistency: No signifi cant difference was found in the performance of students over a 4-week period. No inter-relater reliability is provided.


Practicality and usefulness: Highly useful for screening students in early grades. The manual provides the basis for a discussion of fl uency and listening vocabulary with parents.


Identifi cation of a range of responses or degree of competence: Recommended criteria for classifi cation appear to be adequate.



Primary strengths: Ease of administration and high face validity.

Primary weaknesses: Lack of information on interrater reliability and correlation with other communication skills.

Overall adequacy: A useful measure for the early-grades classroom teacher interested in a quick measure of oral language skills in the classroom.

Reviewer: Ray J. Fenton



Primary strengths: Ease of administration and high face validity.

Primary weaknesses: Lack of information on interrater reliability and correlation with other communication skills.

Overall adequacy: A useful measure for the early-grades classroom teacher interested in a quick measure of oral language skills in the classroom.

Reviewer: Ray J. Fenton




Since the last edition of this compilation of instru-ments, the assessment landscape has changed consider-ably. A decade ago, most colleges and universities were just beginning the process of developing assessment plans. For many, it was a challenge to articulate clear goal statements that carefully defi ned the knowledge, abilities, and dispositions that students should possess at the completion of their studies. It was even more daunting to select or design assessment strategies that would measure student achievement, and then respond to the fi ndings of these assessments with the neces-sary modifi cation of curriculum and pedagogy. While once considered a trend, assessment is now viewed as an integral component of higher education. Accredit-ing agencies and legislatures have continued to expect proof that colleges and universities are meeting their stated objectives, and as their comfort with assessment has grown, faculty and administrators are regularly us-ing assessment results to inform their work.

The demand for accountability, however, has con-tinued to escalate. Just as the No Child Left Behind regulations have resulted in increased assessment at the K-12 level, Congress, state legislatures, state and fed-eral departments of education, and regional accrediting agencies remain concerned about the quality of higher education. The U.S. Department of Education’s Com-mission on the Future of Higher Education is just one example of the ongoing demand that colleges and uni-versities demonstrate that they are providing rigorous and appropriate instruction (Field, 2006). In the ETS report, A Culture of Evidence: Postsecondary Assess-ment and Learning Outcomes, Dwyer, Millett & Payne (2006) call for “a comprehensive national system for determining the nature and extent of college learn-ing” (p.1). The National Center for Public Policy and Higher Education has developed a model for state-by-state comparisons of colleges and universities (Miller & Ewell, 2005).

As communication educators, we need to continue to fi ne tune our assessment approaches so that we can not only meet these increased demands but also so we have the necessary information to continue to improve study in the discipline of communication. W. Robert Connors of the Teagle Foundation noted in his January 2006 address to college deans, “First, in academia we

value knowledge. As scholars, we insist that it is better to know more than to know less. How can we as teach-ers reject an opportunity to know more about how our students are learning? Second, well-designed assess-ment, skillfully used, makes it possible to teach better. And that’s the important point, the positive point: We can teach better, and students can learn better, if their learning is systematically assessed.”

A brief Web search of oral communication assess-ment programs reveals that institutions have developed a variety of strategies for monitoring the impact of com-munication studies within the general education context and within specifi c departmental programs. Many of the measures noted in this text are included in these assessment plans. Currently, most assessment of oral communication focuses on the public speaking context. The oral presentations of students are regularly evalu-ated using an established rubric such as The Compe-tent Speaker. Often, the students’ level of apprehension is also monitored through pre-post administration of the Personal Report of Communication Apprehension (PRCA-24). Self-reports of improvement are another important component of many plans. Students and fac-ulty are surveyed about their perceptions of improve-ment. While these are commendable efforts, the next generation of assessment will need to expand upon these practices with more sophisticated and detailed strategies. The questions that we ask need to be more precise and the mechanism we employ need to be more targeted. This third edition of Large Scale Assessment can be of assistance to those interested in developing these more precise and targeted plans. The instruments can help us monitor the impact of instruction not only in the realm of public speaking, but also listening, inter-personal, small group, and intercultural contexts. Most can be easily included in classroom and programmatic assessment plans, and can also help us study other learn-ing opportunities such as international experiences and co-curricular participation.

Identifying and Selecting Measures

Possible instruments to include in this publication were identifi ed through a review of several relevant database such as Academic Search Premier (EBSCO), ComAbstracts, and Communication and Mass Media Complete. A call was published in Spectra, and com-

Part III: Oral Communication Assessment Instruments for Higher Education




Since the last edition of this compilation of instru-ments, the assessment landscape has changed consider-ably. A decade ago, most colleges and universities were just beginning the process of developing assessment plans. For many, it was a challenge to articulate clear goal statements that carefully defi ned the knowledge, abilities, and dispositions that students should possess at the completion of their studies. It was even more daunting to select or design assessment strategies that would measure student achievement, and then respond to the fi ndings of these assessments with the neces-sary modifi cation of curriculum and pedagogy. While once considered a trend, assessment is now viewed as an integral component of higher education. Accredit-ing agencies and legislatures have continued to expect proof that colleges and universities are meeting their stated objectives, and as their comfort with assessment has grown, faculty and administrators are regularly us-ing assessment results to inform their work.

The demand for accountability, however, has con-tinued to escalate. Just as the No Child Left Behind regulations have resulted in increased assessment at the K-12 level, Congress, state legislatures, state and fed-eral departments of education, and regional accrediting agencies remain concerned about the quality of higher education. The U.S. Department of Education’s Com-mission on the Future of Higher Education is just one example of the ongoing demand that colleges and uni-versities demonstrate that they are providing rigorous and appropriate instruction (Field, 2006). In the ETS report, A Culture of Evidence: Postsecondary Assess-ment and Learning Outcomes, Dwyer, Millett & Payne (2006) call for “a comprehensive national system for determining the nature and extent of college learn-ing” (p.1). The National Center for Public Policy and Higher Education has developed a model for state-by-state comparisons of colleges and universities (Miller & Ewell, 2005).

As communication educators, we need to continue to fi ne tune our assessment approaches so that we can not only meet these increased demands but also so we have the necessary information to continue to improve study in the discipline of communication. W. Robert Connors of the Teagle Foundation noted in his January 2006 address to college deans, “First, in academia we

value knowledge. As scholars, we insist that it is better to know more than to know less. How can we as teach-ers reject an opportunity to know more about how our students are learning? Second, well-designed assess-ment, skillfully used, makes it possible to teach better. And that’s the important point, the positive point: We can teach better, and students can learn better, if their learning is systematically assessed.”

A brief Web search of oral communication assess-ment programs reveals that institutions have developed a variety of strategies for monitoring the impact of com-munication studies within the general education context and within specifi c departmental programs. Many of the measures noted in this text are included in these assessment plans. Currently, most assessment of oral communication focuses on the public speaking context. The oral presentations of students are regularly evalu-ated using an established rubric such as The Compe-tent Speaker. Often, the students’ level of apprehension is also monitored through pre-post administration of the Personal Report of Communication Apprehension (PRCA-24). Self-reports of improvement are another important component of many plans. Students and fac-ulty are surveyed about their perceptions of improve-ment. While these are commendable efforts, the next generation of assessment will need to expand upon these practices with more sophisticated and detailed strategies. The questions that we ask need to be more precise and the mechanism we employ need to be more targeted. This third edition of Large Scale Assessment can be of assistance to those interested in developing these more precise and targeted plans. The instruments can help us monitor the impact of instruction not only in the realm of public speaking, but also listening, inter-personal, small group, and intercultural contexts. Most can be easily included in classroom and programmatic assessment plans, and can also help us study other learn-ing opportunities such as international experiences and co-curricular participation.

Identifying and Selecting Measures

Possible instruments to include in this publication were identifi ed through a review of several relevant database such as Academic Search Premier (EBSCO), ComAbstracts, and Communication and Mass Media Complete. A call was published in Spectra, and com-

Part III: Oral Communication Assessment Instruments for Higher Education



pilations such as Communication Research Measures: A Sourcebook (Rubin, Palmgreen & Sypher, 2004) and The Sourcebook of Nonverbal Measures: Going Beyond Words (Manusov, 2005) were also reviewed. Hundreds of measures were identifi ed from the aforementioned search. The next task was to select those instruments most appropriate to this publication. In making this de-cision, the following criteria were applied:

• Adherence to the NCA guidelines for assess-ment.

• Suitability/adaptability to the instructional set-ting.

• Appropriateness to a college/university popula-tion as opposed to more specialized populations in health care, family counseling, education, busi-ness, etc.

• Availability of measure from a publisher, docu-ment service, or through the library of a major research institution.

• Timeliness of measure, ensuring that it refl ects the most contemporary research on the construct/concept under consideration.

Every effort was made to ensure the inclusion of measures that would tap the broad spectrum of the dis-cipline and also be suited to many different types of stu-dents in a variety of educational settings.

Using Measures

The instruments are only one element of an assess-ment program. Good assessment fi rst depends on clear statements of the outcomes that are expected from the learning experience. To determine if the goals are re-alized, assessment should be planned to gather data from many different sources over an extended period of time. The instruments included in this publication can be combined with each other and with institutional sur-veys, examinations, and portfolios to form a “snapshot” of student learning. Once conclusions can be drawn, then the strengths and weaknesses of the total learn-ing experience can be identifi ed and appropriately ad-dressed.

Through its Web site, the National Communication Association (www.natcom.org/assessment) offers a number of resources that are helpful to those interested in further refi ning their assessment plans and strategies. The site provides competency statements, philosophi-cal grounding, and guidelines and templates for depart-

ment assessment plans along with criteria for evaluat-ing assessment choices. These resources should be drawn upon in formulating or redesigning assessment approaches. They will provide a context for the use of these measures.

Continuing Challenges for the Future

Even with the progress that has been made in devel-oping and implementing assessment programs, com-munication educators still need to participate in further enhancing assessment related to our discipline. First, there has been increased pressure to develop discipline-based standardized measures. For example, the ETS re-port recommends “a series of expert panels to address the issue of what constitutes adequate knowledge and preparation within broad major fi elds” (Dwyer Millett & Payne, 2006, p.20). Once we have a set of expecta-tions we then need to focus our attention on designing and validating strategies that will measure effectiveness in realizing these outcomes.

Second, communication educators should contribute to the ongoing discussion of essential college outcomes. Oral communication profi ciency, group communica-tion abilities and listening are regularly identifi ed as important lifelong skills for all college graduates, but assessment of these skills is sidestepped. For example, the American Association of Colleges and Universities publication Liberal Education Outcomes identifi es oral communication competency and teamwork as essen-tial outcomes but notes that there is no data related to achievement in these areas (Schneider & Miller, 2005). The National Center for Public Policy and Higher Ed-ucation recommends that states use four of the ACT WorkKeys assessments but does not include the tests of listening and teamwork (Miller & Ewell, 2005). The ETS report acknowledges the importance of these “soft skills,” but does not recommend their assessment (Dw-yer, Millett & Payne, 2006). Oral communication com-petency in its many dimensions is an integral element in effective adult functioning and should not be ignored in national assessment initiatives. While the measures in this publication may serve as a foundation for develop-ing assessment strategies, more research is needed.

Clearly, while we have made strides in assessing oral communication at the department and institutional lev-els, there are now national conversations in which we should participate. The challenge in the next decade is to engage in these discussions.



pilations such as Communication Research Measures: A Sourcebook (Rubin, Palmgreen & Sypher, 2004) and The Sourcebook of Nonverbal Measures: Going Beyond Words (Manusov, 2005) were also reviewed. Hundreds of measures were identifi ed from the aforementioned search. The next task was to select those instruments most appropriate to this publication. In making this de-cision, the following criteria were applied:

• Adherence to the NCA guidelines for assess-ment.

• Suitability/adaptability to the instructional set-ting.

• Appropriateness to a college/university popula-tion as opposed to more specialized populations in health care, family counseling, education, busi-ness, etc.

• Availability of measure from a publisher, docu-ment service, or through the library of a major research institution.

• Timeliness of measure, ensuring that it refl ects the most contemporary research on the construct/concept under consideration.

Every effort was made to ensure the inclusion of measures that would tap the broad spectrum of the dis-cipline and also be suited to many different types of stu-dents in a variety of educational settings.

Using Measures

The instruments are only one element of an assess-ment program. Good assessment fi rst depends on clear statements of the outcomes that are expected from the learning experience. To determine if the goals are re-alized, assessment should be planned to gather data from many different sources over an extended period of time. The instruments included in this publication can be combined with each other and with institutional sur-veys, examinations, and portfolios to form a “snapshot” of student learning. Once conclusions can be drawn, then the strengths and weaknesses of the total learn-ing experience can be identifi ed and appropriately ad-dressed.

Through its Web site, the National Communication Association (www.natcom.org/assessment) offers a number of resources that are helpful to those interested in further refi ning their assessment plans and strategies. The site provides competency statements, philosophi-cal grounding, and guidelines and templates for depart-

ment assessment plans along with criteria for evaluat-ing assessment choices. These resources should be drawn upon in formulating or redesigning assessment approaches. They will provide a context for the use of these measures.

Continuing Challenges for the Future

Even with the progress that has been made in devel-oping and implementing assessment programs, com-munication educators still need to participate in further enhancing assessment related to our discipline. First, there has been increased pressure to develop discipline-based standardized measures. For example, the ETS re-port recommends “a series of expert panels to address the issue of what constitutes adequate knowledge and preparation within broad major fi elds” (Dwyer Millett & Payne, 2006, p.20). Once we have a set of expecta-tions we then need to focus our attention on designing and validating strategies that will measure effectiveness in realizing these outcomes.

Second, communication educators should contribute to the ongoing discussion of essential college outcomes. Oral communication profi ciency, group communica-tion abilities and listening are regularly identifi ed as important lifelong skills for all college graduates, but assessment of these skills is sidestepped. For example, the American Association of Colleges and Universities publication Liberal Education Outcomes identifi es oral communication competency and teamwork as essen-tial outcomes but notes that there is no data related to achievement in these areas (Schneider & Miller, 2005). The National Center for Public Policy and Higher Ed-ucation recommends that states use four of the ACT WorkKeys assessments but does not include the tests of listening and teamwork (Miller & Ewell, 2005). The ETS report acknowledges the importance of these “soft skills,” but does not recommend their assessment (Dw-yer, Millett & Payne, 2006). Oral communication com-petency in its many dimensions is an integral element in effective adult functioning and should not be ignored in national assessment initiatives. While the measures in this publication may serve as a foundation for develop-ing assessment strategies, more research is needed.

Clearly, while we have made strides in assessing oral communication at the department and institutional lev-els, there are now national conversations in which we should participate. The challenge in the next decade is to engage in these discussions.



ReferencesConnors, W.R. (2006). From foxes to hedgehogs. Taking

liberal education to the next level --valuing our values. Keynote Address to the Association of College Academic Deans, Washington D.C., January 27, 2006.

Dwyer, C.A., Millett, C.M. & Payne, D.G. (2006). A culture of evidence: Postsecondary assessment and learning out-comes. Princeton, N.J: Educational Testing Service.

Field, K. (2006). Members of federal panel on higher edu-cation express optimism after closed-door discussions of draft report The Chronicle of Higher Education’s Daily Report for Subscribers, June 29, 2006.

Manusov, V.L. (2005). The sourcebook of nonverbal mea-sures: Going beyond words.Mahway, N.J.: Lawrence Erl-baum Associates.

Miller, M.A. & Ewell, P.T. (2005). Measuring up on college-level learning. Washington, D.C.: The National Center for Public Policy and Higher Education.

Rubin, R.B. Palmgreen, P. & Sypher, H.E. (2004). Commu-nication research measures:A sourcebook. Mahway, N.J.: Lawrence Erlbaum Associates.

Schneider, C. & Miller, R. (2005). Liberal education out-comes: A preliminary report of student achievement in col-lege. Washington, D.C.: American Association of Colleges and Universities.



ReferencesConnors, W.R. (2006). From foxes to hedgehogs. Taking

liberal education to the next level --valuing our values. Keynote Address to the Association of College Academic Deans, Washington D.C., January 27, 2006.

Dwyer, C.A., Millett, C.M. & Payne, D.G. (2006). A culture of evidence: Postsecondary assessment and learning out-comes. Princeton, N.J: Educational Testing Service.

Field, K. (2006). Members of federal panel on higher edu-cation express optimism after closed-door discussions of draft report The Chronicle of Higher Education’s Daily Report for Subscribers, June 29, 2006.

Manusov, V.L. (2005). The sourcebook of nonverbal mea-sures: Going beyond words.Mahway, N.J.: Lawrence Erl-baum Associates.

Miller, M.A. & Ewell, P.T. (2005). Measuring up on college-level learning. Washington, D.C.: The National Center for Public Policy and Higher Education.

Rubin, R.B. Palmgreen, P. & Sypher, H.E. (2004). Commu-nication research measures:A sourcebook. Mahway, N.J.: Lawrence Erlbaum Associates.

Schneider, C. & Miller, R. (2005). Liberal education out-comes: A preliminary report of student achievement in col-lege. Washington, D.C.: American Association of Colleges and Universities.



TABLE: ORAL COMMUNICATION ASSESSMENT INSTRUMENTS FOR HIGHER EDUCATION

DIMENSIONS OF INTERPERSONAL INTERACTION


Affective Communi-cation Test

Journal of Personality & Social Psychology, 39

Interpersonal College/Adult 13 items, Likert-type scale

55

Argumentiveness Scale

Journal of Personality As-sessment, 46

Interpersonal College/Adult 20 items Likert-type scale

57

Cognitive Communi-cation Competence Scale

Communication Quar-terly, 43


61

Cognitive Flexibility Scale

Psychology Reports, 76 Interpersonal College/Adult 12 items, Likert-type scale

63

College Self-Expres-sion Scale

Behavior Therapy, 5 Interpersonal College/Adult 50 items Likert-type scale

65

Communication-Based Emotional Support Scale

Communication Research Reports, 13


67

Conversational Sen-sitivity Scale

Human Communication Research, 14


77

Conversational Skills Rating Scale

NCA Interpersonal College/Adult 25 items self or other report

79

Decision-Making Collaboration Scale



81

Interaction Involve-ment Scale

Communication Mono-graphs, 49


83

Interpersonal Com-munication Compe-tency Scale



89

Interpersonal Com-munication Satisfac-tion Inventory

Human Communication, 4

Interpersonal College/Adult 15 minutes, 19 rating statements

91

Relationship Assess-ment Scale

Journal of Marriage and Family, 50


105

Relational Communi-cation Scale



106



TABLE: ORAL COMMUNICATION ASSESSMENT INSTRUMENTS FOR HIGHER EDUCATION

DIMENSIONS OF INTERPERSONAL INTERACTION


Affective Communi-cation Test



55

Argumentiveness Scale

Journal of Personality As-sessment, 46


57

Cognitive Communi-cation Competence Scale

Communication Quar-terly, 43


61


Psychology Reports, 76 Interpersonal College/Adult 12 items, Likert-type scale

63

College Self-Expres-sion Scale

Behavior Therapy, 5 Interpersonal College/Adult 50 items Likert-type scale

65




67

Conversational Sen-sitivity Scale



77

Conversational Skills Rating Scale

NCA Interpersonal College/Adult 25 items self or other report

79




81

Interaction Involve-ment Scale



83

Interpersonal Com-munication Compe-tency Scale



89

Interpersonal Com-munication Satisfac-tion Inventory

Human Communication, 4

Interpersonal College/Adult 15 minutes, 19 rating statements

91

Relationship Assess-ment Scale

Journal of Marriage and Family, 50


105

Relational Communi-cation Scale



106




Revised Self-Disclo-sure Scale



112

Socio-Communica-tive Style & Orienta-tion

Communication for Teachers, 2002


116

Social Skills Inven-tory



118

Verbal Aggressive-ness Scale



126

COMMUNICATION COMPETENCY


Communication Competency As-sessment Instru-ment-Revised

Triton Video Comm. Comp.

College/Adult Present speech, view video, conver-sation

71

Self-perceived Communication Competence Scale

Communication Education, 5

Comm. Comp.

College/Adult 12 items, Estimate percentage

114

PUBLIC SPEAKING


Competent Speaker

NCA Speaking College/Adult Rubric 73

Public Speak-ing Competen-cy Instrument

Communica-tion Research Reports, 67

Speaking College/Adult Rubric 102

GROUP COMMUNICATION


Relational Satisfaction Scale

Communication Studies, 52

Group inter-action

College/Adult 12 items, Likert-type scale

108

WorkKeys for Education Teamwork

ACT-Iowa City Group inter-action

H.S./College/Adult

12 video scenarios 136




Revised Self-Disclo-sure Scale



112

Socio-Communica-tive Style & Orienta-tion

Communication for Teachers, 2002


116

Social Skills Inven-tory



118

Verbal Aggressive-ness Scale



126

COMMUNICATION COMPETENCY


Communication Competency As-sessment Instru-ment-Revised

Triton Video Comm. Comp.

College/Adult Present speech, view video, conver-sation

71

Self-perceived Communication Competence Scale


Comm. Comp.

College/Adult 12 items, Estimate percentage

114

PUBLIC SPEAKING


Competent Speaker

NCA Speaking College/Adult Rubric 73

Public Speak-ing Competen-cy Instrument


Speaking College/Adult Rubric 102

GROUP COMMUNICATION



Communication Studies, 52

Group inter-action

College/Adult 12 items, Likert-type scale

108

WorkKeys for Education Teamwork

ACT-Iowa City Group inter-action

H.S./College/Adult

12 video scenarios 136



LISTENING


Kentucky Lis-tening Compre-hension Test

[email protected]

Listening H.S./College/Adult

Four-part listening and activities

93

Learning to Listen

HRDQ. www.hrdq.com

Listening College/ business

Unknown 95

Listening Styles Inventory

Journal of Busi-ness & Technical Communication

Listening College/Adult 10 items, Likert-type scale

96

Watson-Barker Revised Listen-ing Test-C&D

Innolect, Inc. Listening College/Adult Five-part video 128

Willingness to Listen Scale

International Journal of Listen-ing, 12

Listening College/Adult 36 items, estimate percentage

132

WorkKeys for Education Lis-tening

ACT-Iowa City Listening College/Adult Eight-part audio tape

134

DIMENSIONS OF INTERCULTURAL INTERACTION


Intercultural Sensitivity Scale

Human Com-munication Research, 3

Intercultural College/Adult 24 items, Likert-type scale

85

Intercultural Willingness to Communicate


Intercultural College/Adult 12 items, estimate percentage

87

PRICA/PRECA Communica-tion Research Reports, 14


99

Speech Evalua-tion Instrument

Journal of Lan-guage & Social Psychology

Intercultural College/Adult 30 items, Semantic Differential

120



LISTENING


Kentucky Lis-tening Compre-hension Test

[email protected]

Listening H.S./College/Adult

Four-part listening and activities

93

Learning to Listen

HRDQ. www.hrdq.com

Listening College/ business

Unknown 95


Journal of Busi-ness & Technical Communication

Listening College/Adult 10 items, Likert-type scale

96

Watson-Barker Revised Listen-ing Test-C&D

Innolect, Inc. Listening College/Adult Five-part video 128


International Journal of Listen-ing, 12

Listening College/Adult 36 items, estimate percentage

132

WorkKeys for Education Lis-tening

ACT-Iowa City Listening College/Adult Eight-part audio tape

134

DIMENSIONS OF INTERCULTURAL INTERACTION



Human Com-munication Research, 3


85

Intercultural Willingness to Communicate


Intercultural College/Adult 12 items, estimate percentage

87

PRICA/PRECA Communica-tion Research Reports, 14


99

Speech Evalua-tion Instrument

Journal of Lan-guage & Social Psychology

Intercultural College/Adult 30 items, Semantic Differential

120



COMMUNICATION APPREHENSION


Communication Anxiety Inven-tory

Communication Quarterly, 34

Apprehension College/Adult 20-41 items, Likert-type scale

67

Personal Report of Communica-tion Apprehen-sion


Apprehension College/Adult 24 items, Likert-type scale

97

Receiver Appre-hension Test

Speech Teach-er, 24


103

Reticence Scale Communication Quarterly, 45


110

Stroop Test for Public Speaking Apprehension


Apprehension College/Adult Response time to stimuli

122

Willingness to Communicate


Apprehension College/Adult 20 items, estimate percentage

130

INSTRUCTIONAL COMMUNICATION


Class Appre-hension About Participation Scale


Instructional College/Adult 20 items, Likert-type scale

59

Connected Classroom Cli-mate Inventory



75

Teacher Clarity Short Inventory



124



COMMUNICATION APPREHENSION


Communication Anxiety Inven-tory


Apprehension College/Adult 20-41 items, Likert-type scale

67

Personal Report of Communica-tion Apprehen-sion



97

Receiver Appre-hension Test

Speech Teach-er, 24


103

Reticence Scale Communication Quarterly, 45


110



Apprehension College/Adult Response time to stimuli

122

Willingness to Communicate


Apprehension College/Adult 20 items, estimate percentage

130

INSTRUCTIONAL COMMUNICATION


Class Appre-hension About Participation Scale



59

Connected Classroom Cli-mate Inventory



75

Teacher Clarity Short Inventory



124



Affective Communication Test

Author / Developers: H.S. Friedman, L.M. Prince, R.E. Riggio, and M.R. DiMatteo

Address: Howard S. Friedman, Department of Psychology, University of California, Riverside, CA 92521

Source: Friedman,H.S., Prince, L.M., Riggio, R.E. & DiMatteo, M.R. (1980). Understanding and assessing nonver-bal expressiveness: The affective communication test. Journal of Personality and Social Psychology, 39, 333-51.

Context or communication behavior for which intended: Nonverbal expressiveness during interpersonal interac-tion

Academic level for which intended: College/Adult

Time required: 5-10 minutes

Description: This self-report measure includes 13 items. Respondents indicate on a nine-point Likert-type scale (-4 to +4), the degree to which each statement is applicable to them.

Description of scoring: Scoring for 6 of the items is reversed and then a total score is computed. Scores ranged from 28-114.

Norm referencing/criterion referencing populations: Various groups of college students were used in developing and testing this measure.

Tests for cultural/gender bias: Women tend to score slightly higher than men.

Validity

Predictive: ACT scores highly correlated with perceptions by others as to the individual’s expressiveness. High ACT scores were correlated with individuals who frequently engage in behavior intended to “move, inspire, and captivate others” (p. 337).

Face/Content: The article discusses conceptualization of the nonverbal expressiveness construct and past research in this area.

Construct: ACT correlated in the predicted manner with other measures of personality such as extroversion, neu-roticism, social desirability, anxiety, self-esteem and self-monitoring.

Reliability

Test/retest: After a two-week interval, a test-retest correlation of .90 was reported.

Internal consistency: A reliability co-effi cient of .77 was reported.


Practicality and usefulness: This might be useful on a pre/post course basis as an indicator of behavioral change.

Clarity of instructions: Directions are very clear.

Degree of standardization in administration and evaluation of responses: The measure is highly standardized.

Identifi cation of a range of responses or degrees of competence: Interpretative ranges are not provided in the article.

Primary strengths: This measure has undergone very thorough validation studies and offers a unique focus on the nonverbal dimension of interpersonal interaction.

Primary weaknesses: Users need to avoid over generalizing from this measure since it focuses on a fairly specifi c construct.



Affective Communication Test

Author / Developers: H.S. Friedman, L.M. Prince, R.E. Riggio, and M.R. DiMatteo

Address: Howard S. Friedman, Department of Psychology, University of California, Riverside, CA 92521

Source: Friedman,H.S., Prince, L.M., Riggio, R.E. & DiMatteo, M.R. (1980). Understanding and assessing nonver-bal expressiveness: The affective communication test. Journal of Personality and Social Psychology, 39, 333-51.

Context or communication behavior for which intended: Nonverbal expressiveness during interpersonal interac-tion



Description: This self-report measure includes 13 items. Respondents indicate on a nine-point Likert-type scale (-4 to +4), the degree to which each statement is applicable to them.

Description of scoring: Scoring for 6 of the items is reversed and then a total score is computed. Scores ranged from 28-114.

Norm referencing/criterion referencing populations: Various groups of college students were used in developing and testing this measure.

Tests for cultural/gender bias: Women tend to score slightly higher than men.

Validity

Predictive: ACT scores highly correlated with perceptions by others as to the individual’s expressiveness. High ACT scores were correlated with individuals who frequently engage in behavior intended to “move, inspire, and captivate others” (p. 337).

Face/Content: The article discusses conceptualization of the nonverbal expressiveness construct and past research in this area.

Construct: ACT correlated in the predicted manner with other measures of personality such as extroversion, neu-roticism, social desirability, anxiety, self-esteem and self-monitoring.

Reliability

Test/retest: After a two-week interval, a test-retest correlation of .90 was reported.



Practicality and usefulness: This might be useful on a pre/post course basis as an indicator of behavioral change.



Identifi cation of a range of responses or degrees of competence: Interpretative ranges are not provided in the article.

Primary strengths: This measure has undergone very thorough validation studies and offers a unique focus on the nonverbal dimension of interpersonal interaction.

Primary weaknesses: Users need to avoid over generalizing from this measure since it focuses on a fairly specifi c construct.



Overall adequacy: When used in conjunction with other measures, the ACT could be a good indicator of interper-sonal effectiveness.

Additional ReferencesFriedman, H. S., Riggio, R. E., & Casella, D. F. (1988). Nonverbal skill, personal charisma, and initial attraction. Personality

and Social Psychology Bulletin, 14, 203-211.Riggio, R.E. & Woll, S.B. (1984) The role of nonverbal cues and physical attractiveness in the selection of dating partners.

Journal of Social and Personal Relationships, 1: 3, 347-357.

Tucker, J.S. & Friedman, H.S. (1993) Sex differences in nonverbal expressiveness: Emotional expression, personality, and impressions. Journal of Nonverbal Behavior, 17:2, 103-117




Overall adequacy: When used in conjunction with other measures, the ACT could be a good indicator of interper-sonal effectiveness.

Additional ReferencesFriedman, H. S., Riggio, R. E., & Casella, D. F. (1988). Nonverbal skill, personal charisma, and initial attraction. Personality

and Social Psychology Bulletin, 14, 203-211.Riggio, R.E. & Woll, S.B. (1984) The role of nonverbal cues and physical attractiveness in the selection of dating partners.

Journal of Social and Personal Relationships, 1: 3, 347-357.

Tucker, J.S. & Friedman, H.S. (1993) Sex differences in nonverbal expressiveness: Emotional expression, personality, and impressions. Journal of Nonverbal Behavior, 17:2, 103-117




The Argumentativeness Scale

Author / Developers: D.A. Infante and A.S. Rancer

Address: Dominic Infante, Kent State University, School of Speech, Kent, OH 42242

Source: Infante, D.A. & Rancer, A.S. (1982) A conceptualization and measure of argumentativeness. Journal of Personality Assessment, 46, 72-80.

Context or communication behavior for which intended: Measures the tendency in individuals to approach or avoid arguments.


Time required: 15 minutes

Description of test: Respondents react to 20 statements on a fi ve point Likert-type scale noting how well that state-ment describes their attitudes and behaviors in argumentative situations.

Description of scoring: Two subscales are calculated. Ten designated items are totaled to produce a score of the tendency to approach argumentative situations. Ten other items relate to the tendency to avoid arguments. The ar-gumentative trait is calculated by subtracting the avoid subscale from the approach subscale. A mean score of 4.44 was reported for the generalized argumentative trait. Ranges for high, low, and normal argumentativeness are given (Infante, 1988).

Norm referencing/criterion referencing populations: College and university students were used as subjects for the various studies conducted on this instrument.

Culture/Gender Bias: When U.. and Japanese subjects were compared, slight cultural variations were identifi ed, but construct validity across cultures was satisfactory (Suzuki & Rancer, 1994). Reported gender differences may be attributed to a social desirability bias (Nicotera, 1996).

Validity

Predictive: High and moderate correlations were found between subject responses and the responses of a friend as to that subject’s predisposition to argue.

Concurrent: Argumentativeness as measured by this scale was correlated slightly or moderately with measures of apprehension, willingness to communicate and predispositions of verbal behavior.

Face/Content: The article describes prior research and theory that contributed to scale development.

Construct: Subjects high in argumentativeness were found to be more likely to engage in argumentative situa-tions.

Reliability

Test/retest: A R=.91 between test and retest scores indicates high stability.

Internal consistency: A Cronbach alpha of .91 was reported for the approach items. A Cronbach alpha of .86 was reported for the avoid items.


Practicality and usefulness: This measures a fairly narrow predisposition but might be useful in helping to assess the success of instruction in argumentation, persuasion and assertiveness.

Clarity of instructions: Very clear instructions were provided.



The Argumentativeness Scale

Author / Developers: D.A. Infante and A.S. Rancer


Source: Infante, D.A. & Rancer, A.S. (1982) A conceptualization and measure of argumentativeness. Journal of Personality Assessment, 46, 72-80.

Context or communication behavior for which intended: Measures the tendency in individuals to approach or avoid arguments.



Description of test: Respondents react to 20 statements on a fi ve point Likert-type scale noting how well that state-ment describes their attitudes and behaviors in argumentative situations.

Description of scoring: Two subscales are calculated. Ten designated items are totaled to produce a score of the tendency to approach argumentative situations. Ten other items relate to the tendency to avoid arguments. The ar-gumentative trait is calculated by subtracting the avoid subscale from the approach subscale. A mean score of 4.44 was reported for the generalized argumentative trait. Ranges for high, low, and normal argumentativeness are given (Infante, 1988).

Norm referencing/criterion referencing populations: College and university students were used as subjects for the various studies conducted on this instrument.

Culture/Gender Bias: When U.. and Japanese subjects were compared, slight cultural variations were identifi ed, but construct validity across cultures was satisfactory (Suzuki & Rancer, 1994). Reported gender differences may be attributed to a social desirability bias (Nicotera, 1996).

Validity

Predictive: High and moderate correlations were found between subject responses and the responses of a friend as to that subject’s predisposition to argue.

Concurrent: Argumentativeness as measured by this scale was correlated slightly or moderately with measures of apprehension, willingness to communicate and predispositions of verbal behavior.

Face/Content: The article describes prior research and theory that contributed to scale development.

Construct: Subjects high in argumentativeness were found to be more likely to engage in argumentative situa-tions.

Reliability

Test/retest: A R=.91 between test and retest scores indicates high stability.

Internal consistency: A Cronbach alpha of .91 was reported for the approach items. A Cronbach alpha of .86 was reported for the avoid items.


Practicality and usefulness: This measures a fairly narrow predisposition but might be useful in helping to assess the success of instruction in argumentation, persuasion and assertiveness.

Clarity of instructions: Very clear instructions were provided.



Degree of standardization in administration and evaluation of responses: Scoring of the instrument is straightforward and consistent.

Identifi cation of a range of responses or degrees of competence: High, moderate and low ranges have been identifi ed.

Primary strengths: This is a very effi cient, straightforward measure which might be used at the begin-ning and ending of a course to assess changes in predisposition.

Overall adequacy: With increased emphasis on preparing students to argue effectively both in oral and written forms, this measure would be an appropriate pre-post assessment of attitude change as argumenta-tive ability develops.

Additional ReferencesDeWine, S., Nicotera, A.M. & Parry, D. (1991) Argumentativeness and aggressiveness: The fl ip side of gentle persuasion. Man-

agement Communication Quarterly, 4:3, 386-411.Dowling, R.E. & Flint, L.J. (1990) The argumentativeness scale: Problems and promises. Communication Studies, 41:2, 183-

198.Infante, D.A. (1988). Arguing Constructively. Prospect Heights, IL: Waveland Press.Neer, M.R. (1994). Argumentative fl exibility as a factor infl uencing message responsestyle to argumentative and aggressive

arguers. Argumentation & Advocacy, 31, 17-33.Nicotera, A.M. (1996). An assessment of the argumentativeness scale for social desirability bias. Communication Reports, 9:1,

23-36.Suzuki, S. & Rancer, A.S. (1994). Argumentativeness and verbal aggressiveness: Testing for conceptual and measurement

equivalence across cultures. Communication Monographs: 61: 3, 256-279.




Degree of standardization in administration and evaluation of responses: Scoring of the instrument is straightforward and consistent.

Identifi cation of a range of responses or degrees of competence: High, moderate and low ranges have been identifi ed.

Primary strengths: This is a very effi cient, straightforward measure which might be used at the begin-ning and ending of a course to assess changes in predisposition.

Overall adequacy: With increased emphasis on preparing students to argue effectively both in oral and written forms, this measure would be an appropriate pre-post assessment of attitude change as argumenta-tive ability develops.

Additional ReferencesDeWine, S., Nicotera, A.M. & Parry, D. (1991) Argumentativeness and aggressiveness: The fl ip side of gentle persuasion. Man-


198.Infante, D.A. (1988). Arguing Constructively. Prospect Heights, IL: Waveland Press.Neer, M.R. (1994). Argumentative fl exibility as a factor infl uencing message responsestyle to argumentative and aggressive







Class Apprehension about Participation Scale (CAPS)

Author / Developers: Michael R. Neer

Address: Department of Communication Studies, University of Missouri, 5100 Rockhill Road, Kansas City, MO 64110

Source: Neer, M. R. (1987). The development of an instrument to measure classroom apprehension. Communica-tion Education, 36, 154-166.

Context of communication behavior for which intended: communication apprehension


Time required: 20 to 30 minutes

Description: A self-report inventory of 20 Likert-type statements designed to measure level of classroom apprehen-sion and to identify observable behaviors. The dimension called communication participation consists of 10 items which generally index approach/avoidance behaviors associated with communication apprehension, and the dimen-sion called communication confi dence is made up of 10 items which include general nervousness and level of fear of being evaluated during class discussion.

Description of administration and scoring: Both instruments have instructions included at the beginning, so they can be administered in large settings if necessary. Items scores are summed for each of the two dimensions, or added together to an overall summary score.

Norm referencing/criterion referencing populations: Studies were conducted on 324 undergraduates enrolled in a basic public speaking course. The mean was 52.68 with a standard deviation of 15.61.

Tests for cultural bias (ethnicity): Both males and females were included in the validity study. Jaasma (1997) has subsequently found that females are more apprehensive than males. Olaniran & Stewart (1996, 1994) have investi-gated cultural differences in classroom apprehension.

Validity

Predictive: CAPS scores correlated with self reports of 11 or 12 instructional practices, which help students feel more comfortable about participating in class.

Concurrent: Overall CAPS scores correlated at l7i with the PRCA-24. Each dimension (confi dence and participa-tion) correlated at .74 with the PRCA-24.

Reliability

Internal consistency: Alpha reliability for overall score .94. For confi dence = .88, and for participation dimension = .91.


Practicality and usefulness: The scales are easily used. It provides information about classroom behaviors and evaluative attitudes which is not available from the PRCA.

Clarity of instructions: The directions are clear.

Identifi cation of a range of responses or degree of competence: Students scores have been divided into low, me-dium and high apprehensives based on mean deviates.

Primary strengths: It has been compared favorably with the PRCA and is easily administered.

Primary weaknesses: The CAPS scale has been used primarily in research of communication variables rather than as an assessment technique.

Overall adequacy: This has good potential to be developed as an assessment technique.



Class Apprehension about Participation Scale (CAPS)

Author / Developers: Michael R. Neer

Address: Department of Communication Studies, University of Missouri, 5100 Rockhill Road, Kansas City, MO 64110

Source: Neer, M. R. (1987). The development of an instrument to measure classroom apprehension. Communica-tion Education, 36, 154-166.

Context of communication behavior for which intended: communication apprehension



Description: A self-report inventory of 20 Likert-type statements designed to measure level of classroom apprehen-sion and to identify observable behaviors. The dimension called communication participation consists of 10 items which generally index approach/avoidance behaviors associated with communication apprehension, and the dimen-sion called communication confi dence is made up of 10 items which include general nervousness and level of fear of being evaluated during class discussion.

Description of administration and scoring: Both instruments have instructions included at the beginning, so they can be administered in large settings if necessary. Items scores are summed for each of the two dimensions, or added together to an overall summary score.

Norm referencing/criterion referencing populations: Studies were conducted on 324 undergraduates enrolled in a basic public speaking course. The mean was 52.68 with a standard deviation of 15.61.

Tests for cultural bias (ethnicity): Both males and females were included in the validity study. Jaasma (1997) has subsequently found that females are more apprehensive than males. Olaniran & Stewart (1996, 1994) have investi-gated cultural differences in classroom apprehension.

Validity

Predictive: CAPS scores correlated with self reports of 11 or 12 instructional practices, which help students feel more comfortable about participating in class.

Concurrent: Overall CAPS scores correlated at l7i with the PRCA-24. Each dimension (confi dence and participa-tion) correlated at .74 with the PRCA-24.

Reliability

Internal consistency: Alpha reliability for overall score .94. For confi dence = .88, and for participation dimension = .91.


Practicality and usefulness: The scales are easily used. It provides information about classroom behaviors and evaluative attitudes which is not available from the PRCA.


Identifi cation of a range of responses or degree of competence: Students scores have been divided into low, me-dium and high apprehensives based on mean deviates.

Primary strengths: It has been compared favorably with the PRCA and is easily administered.

Primary weaknesses: The CAPS scale has been used primarily in research of communication variables rather than as an assessment technique.




Additional References Aitken, J.E. & Neer, M.R. (1993). College student question-asking. The relationship of classroom communication apprehen-

sion and motivation. Southern Communication Journal, 59, 73-81.Jaasma, M. A. (1997) Classroom communication apprehension: Does being male or female make a difference. Communication

Reports. 10, 219-228. Myers, S. A., & Rocca, K. A. (2001). Perceived instructor argumentativeness and verbal aggressiveness in the college class-

room: Effects on student perceptions of climate, apprehension, and state motivation. Western Journal of Communication. 65, 113-137.

Olaniran, B. A. and Stewart, R. A. (1996) Instructional practices and classroom communication apprehension: A cultural expla-nation. Communication Reports. 9,203-203.

Olaniran, B. A. and Stewart, R. A. Communication apprehension and classroom apprehension in Nigerian classrooms. Com-munication Quarterly. 42, 379-389.

Reviewers: Judith Dallinger and Ellen A. Hay



Additional References Aitken, J.E. & Neer, M.R. (1993). College student question-asking. The relationship of classroom communication apprehen-

sion and motivation. Southern Communication Journal, 59, 73-81.Jaasma, M. A. (1997) Classroom communication apprehension: Does being male or female make a difference. Communication

Reports. 10, 219-228. Myers, S. A., & Rocca, K. A. (2001). Perceived instructor argumentativeness and verbal aggressiveness in the college class-

room: Effects on student perceptions of climate, apprehension, and state motivation. Western Journal of Communication. 65, 113-137.

Olaniran, B. A. and Stewart, R. A. (1996) Instructional practices and classroom communication apprehension: A cultural expla-nation. Communication Reports. 9,203-203.

Olaniran, B. A. and Stewart, R. A. Communication apprehension and classroom apprehension in Nigerian classrooms. Com-munication Quarterly. 42, 379-389.

Reviewers: Judith Dallinger and Ellen A. Hay



Cognitive Communication Competence Scale (CCCS)

Author / Developers: Robert L. Duran and Brian H. Spitzberg

Address: Robert L. Duran, University of Hartford, School of Communication, 200 Bloomfi eld Ave., West Hartford, CT 06117. Phone: 860-768-4633. E-mail: [email protected]

Source: Duran, R. L., & Spitzberg, B. H. (1995). Toward the development and validation of a measure of cognitive communication competence. Communication Quarterly, 43(3), 259-275.

Intended Context or Behavior: Cognitions about interpersonal communication

Intended Academic Level: College

Time Required: 20-30 minutes

Text Description: This 22-item instrument (CCCS) measures fi ve categories of cognitive abilities used in interper-sonal conversation: planning, modeling, presence, refl ection, and consequence. The planning cognitions category has fi ve items. The modeling cognitions category has four items. The presence cognitions category has four items. The refl ection cognitions category has fi ve items. The consequence cognitions category has four items.

Scoring Description: Respondents rate on a fi ve-point Likert-type scale how true each of the statements are in terms of what respondents think about communication in social situations. A score of fi ve indicates “Always true of me” and a score of one indicates “Never true of me.” Each category of cognitions is summed. However it is unclear if all categories are summed for a total score of cognitive communication competence.

Norm/Criterion Reference Populations: This instrument is based on a population of college students. There is no information about the range of respondent scores that can be used as a basis for comparison through norms. There is no information about specifi c criteria that respondents are expected to know or be able to do.

Culture and Gender Bias: Information not available.

Validity

Concurrent: Information not available.

Face/Content: Looking at the CCCS instrument on its face, the construct of cognitive communication competence is limited in its operationalization to conversations within interpersonal interaction. However the article provides a detailed description of how to defi ne the content of cognitive communication.

Construct: Three constructs were tested as construct validation variables: communication knowledge, interaction involvement, and self-monitoring. In comparison to communication knowledge, four of the fi ve cognitive catego-ries had signifi cant relationships. The CCCS and Communicative Knowledge Scale are not redundant measures. In comparison to interaction involvement, four of the fi ve cognitive categories had two signifi cant root relationships that were conceptually consistent. The CCCS and the Interaction Involvement Scale are conceptually consistent. In comparison to self-monitoring, two of the fi ve cognitive categories had signifi cant relationships. The CCCS and revised Self-Monitoring Scale are conceptually consistent. All three constructs measure cognitive communication during interaction but do not measure cognitions that occur before or after interaction. Therefore the construct va-lidity of the CCCS is initially supported but more research is needed to refi ne the operationalization of its construct cognitive communication competence.

Predictive: Information not available.

Reliability

Alternate Form: Information not available.

Test/Retest: Information not available.



Cognitive Communication Competence Scale (CCCS)

Author / Developers: Robert L. Duran and Brian H. Spitzberg

Address: Robert L. Duran, University of Hartford, School of Communication, 200 Bloomfi eld Ave., West Hartford, CT 06117. Phone: 860-768-4633. E-mail: [email protected]

Source: Duran, R. L., & Spitzberg, B. H. (1995). Toward the development and validation of a measure of cognitive communication competence. Communication Quarterly, 43(3), 259-275.

Intended Context or Behavior: Cognitions about interpersonal communication



Text Description: This 22-item instrument (CCCS) measures fi ve categories of cognitive abilities used in interper-sonal conversation: planning, modeling, presence, refl ection, and consequence. The planning cognitions category has fi ve items. The modeling cognitions category has four items. The presence cognitions category has four items. The refl ection cognitions category has fi ve items. The consequence cognitions category has four items.

Scoring Description: Respondents rate on a fi ve-point Likert-type scale how true each of the statements are in terms of what respondents think about communication in social situations. A score of fi ve indicates “Always true of me” and a score of one indicates “Never true of me.” Each category of cognitions is summed. However it is unclear if all categories are summed for a total score of cognitive communication competence.



Validity


Face/Content: Looking at the CCCS instrument on its face, the construct of cognitive communication competence is limited in its operationalization to conversations within interpersonal interaction. However the article provides a detailed description of how to defi ne the content of cognitive communication.

Construct: Three constructs were tested as construct validation variables: communication knowledge, interaction involvement, and self-monitoring. In comparison to communication knowledge, four of the fi ve cognitive catego-ries had signifi cant relationships. The CCCS and Communicative Knowledge Scale are not redundant measures. In comparison to interaction involvement, four of the fi ve cognitive categories had two signifi cant root relationships that were conceptually consistent. The CCCS and the Interaction Involvement Scale are conceptually consistent. In comparison to self-monitoring, two of the fi ve cognitive categories had signifi cant relationships. The CCCS and revised Self-Monitoring Scale are conceptually consistent. All three constructs measure cognitive communication during interaction but do not measure cognitions that occur before or after interaction. Therefore the construct va-lidity of the CCCS is initially supported but more research is needed to refi ne the operationalization of its construct cognitive communication competence.


Reliability





Internal Consistency: The planning cognitions category had an alpha reliability of .73. The modeling cognitions category had an alpha reliability of .71. The presence cognitions category had an alpha reliability of .68. The refl ec-tion cognitions category had an alpha reliability of .86. The consequence cognitions category had an alpha reliability of .81.


Practicality and Usefulness: The CCCS instrument could be administered in courses teaching interpersonal com-munication. This might increase student awareness of how they think about communicating before, during, and after interpersonal conversations.

Clarity of Instructions: The CCCS instrument is reprinted with directions as an article appendix. However no instructions for scoring are provided.

Standardization in Administration and Evaluation: There is no description of how this instrument was admin-istered or used for evaluation of students. The instrument is still in development, which would explain the lack of standardization.

Range of Reponses or Degrees of Competence: This instrument does not provide a range of responses that con-stitute various degrees of competence for this scale. There is no description of how to determine anchor points of high, middle, or low scores.

Primary Strengths: This instrument is designed to be easy to complete and take a minimal amount of time to ad-minister.

Primary Weaknesses: This paper and pencil test relies on self-report from respondents. It is possible that respon-dents will show bias in marking socially desirable answers. Another weakness is the need for further instrument testing to develop a holistic measure of cognitive communication competence.

Overall Adequacy: More research is needed to fully develop this instrument before it is used on a larger scale.

Additional ReferencesDuran, R. L., Kelly, L., Schwager, K., Carone, S., & Stevens, S. (1993, April). Toward the development of a measure of cogni-

tive communication competence. Paper presented at the meeting of the Eastern Communication Association, New Haven, CT.

Reviewer: Sherry L. Wien



Internal Consistency: The planning cognitions category had an alpha reliability of .73. The modeling cognitions category had an alpha reliability of .71. The presence cognitions category had an alpha reliability of .68. The refl ec-tion cognitions category had an alpha reliability of .86. The consequence cognitions category had an alpha reliability of .81.


Practicality and Usefulness: The CCCS instrument could be administered in courses teaching interpersonal com-munication. This might increase student awareness of how they think about communicating before, during, and after interpersonal conversations.

Clarity of Instructions: The CCCS instrument is reprinted with directions as an article appendix. However no instructions for scoring are provided.

Standardization in Administration and Evaluation: There is no description of how this instrument was admin-istered or used for evaluation of students. The instrument is still in development, which would explain the lack of standardization.


Primary Strengths: This instrument is designed to be easy to complete and take a minimal amount of time to ad-minister.

Primary Weaknesses: This paper and pencil test relies on self-report from respondents. It is possible that respon-dents will show bias in marking socially desirable answers. Another weakness is the need for further instrument testing to develop a holistic measure of cognitive communication competence.

Overall Adequacy: More research is needed to fully develop this instrument before it is used on a larger scale.

Additional ReferencesDuran, R. L., Kelly, L., Schwager, K., Carone, S., & Stevens, S. (1993, April). Toward the development of a measure of cogni-

tive communication competence. Paper presented at the meeting of the Eastern Communication Association, New Haven, CT.





Author / Developers:: Matthew Martin and Rebecca Rubin

Address:: Matthew Martin, Department of Communication Studies, 130 Armstrong Hall, POB 6293, West Virginia University, Morgantown, WV 26506-6293

Source: Martin, M.M. & Rubin, R.B.(1995). A new measure of cognitive fl exibility. Psychological Reports, 76, 623-626.

Intended context or behavior: Flexibility in responding to different communication situations.

Intended academic level: College/adult

Time required: Ten minutes

Description: On a six-point Likert scale (6-strongly agree; 1=strongly disagree), subjects respond to 12 statements related to their awareness of communication options and their willingness to select appropriate options.

Description of scoring: After four items are reversed for scoring, responses are totaled. A mean of 54.1 was re-ported.

Norm referencing/criterion referencing populations: Undergraduates

Culture/gender bias: No signifi cant differences between men and women were found in initial testing of the mea-sure. Martin & Rubin (1996) reported that males under 21 and females over 55 were less cognitively fl exible.

Validity

Concurrent: Signifi cant correlations were found with the Interaction Involvement Scale, the Unwillingness to Communicate Scale, and the Self-Monitoring Scale. Martin & Anderson (1998) reported that signifi cant correla-tions with the Assertiveness-Responsiveness Measure. Chesebro & Martin (2003) found a signifi cant correlation between conversational sensitivity and cognitive fl exibility.

Construct: “Scores on cognitive fl exibility were positively related to ratings of communication fl exibility and nega-tively related to rigidity”(p.124). Moderate correlations have been found between self-reports of cognitive fl exibility and friends’ perception of fl exibility (Martin & Anderson, 1998, p.5).

Reliability

Test/retest: A correlation of .83 was reported.

Internal consistency: An alpha coeffi cient of .79 was reported. (Chesbro & Martin, 2003)


Practicality and usefulness: This instrument would be particularly useful in the consideration of interpersonal abilities. If instruction is aimed at developing a repertoire of communication responses, this instrument can be used on a pre-post basis to determine if individuals are knowledgeable about and pre-disposed to use their options. Clar-ity of instructions: Directions are simple and clear.

Range of responses or degrees of competence: Ranges of responses have not been reported.

Primary strengths: This is an effi cient measure of fl exibility that would be helpful in demonstrating the concept of fl exibility to students.

Primary weaknesses: It has been used only in the research context.




Author / Developers:: Matthew Martin and Rebecca Rubin

Address:: Matthew Martin, Department of Communication Studies, 130 Armstrong Hall, POB 6293, West Virginia University, Morgantown, WV 26506-6293

Source: Martin, M.M. & Rubin, R.B.(1995). A new measure of cognitive fl exibility. Psychological Reports, 76, 623-626.

Intended context or behavior: Flexibility in responding to different communication situations.


Time required: Ten minutes

Description: On a six-point Likert scale (6-strongly agree; 1=strongly disagree), subjects respond to 12 statements related to their awareness of communication options and their willingness to select appropriate options.

Description of scoring: After four items are reversed for scoring, responses are totaled. A mean of 54.1 was re-ported.


Culture/gender bias: No signifi cant differences between men and women were found in initial testing of the mea-sure. Martin & Rubin (1996) reported that males under 21 and females over 55 were less cognitively fl exible.

Validity

Concurrent: Signifi cant correlations were found with the Interaction Involvement Scale, the Unwillingness to Communicate Scale, and the Self-Monitoring Scale. Martin & Anderson (1998) reported that signifi cant correla-tions with the Assertiveness-Responsiveness Measure. Chesebro & Martin (2003) found a signifi cant correlation between conversational sensitivity and cognitive fl exibility.

Construct: “Scores on cognitive fl exibility were positively related to ratings of communication fl exibility and nega-tively related to rigidity”(p.124). Moderate correlations have been found between self-reports of cognitive fl exibility and friends’ perception of fl exibility (Martin & Anderson, 1998, p.5).

Reliability

Test/retest: A correlation of .83 was reported.

Internal consistency: An alpha coeffi cient of .79 was reported. (Chesbro & Martin, 2003)


Practicality and usefulness: This instrument would be particularly useful in the consideration of interpersonal abilities. If instruction is aimed at developing a repertoire of communication responses, this instrument can be used on a pre-post basis to determine if individuals are knowledgeable about and pre-disposed to use their options. Clar-ity of instructions: Directions are simple and clear.

Range of responses or degrees of competence: Ranges of responses have not been reported.

Primary strengths: This is an effi cient measure of fl exibility that would be helpful in demonstrating the concept of fl exibility to students.

Primary weaknesses: It has been used only in the research context.



Overall adequacy: This measure would be an effective complement to the Richmond/McCroskey socio-communi-cative style measure. It would provide an assessment of fl exibility in addition to the consideration of assertiveness and responsiveness.

Additional ReferencesChesebro, J.L. & Martin, M.M. (2003). The relationship between conversational sensitivity, cognitive fl exibility, verbal aggres-

siveness and indirect interpersonal aggressiveness. Communication Research Reports, 20:2, 143-150. Martin, M.M. & Anderson, C.M. (1998) The cognitive fl exibility scale: Three validity studies. Communication Reports, 11,

1-9. Martin M.M. & Anderson, C.M. (2001) The relationship between cognitive fl exibility and affi nity -seeking strategies. Advances

in Psychological Research, 4, 69-76. Martin, M.M. Anderson, C.M. & Thweatt, L.S. (1998). Individuals’ perceptions of their communication behaviors: A validity

study of the relationship between the cognitive fl exibility scale and the cognitive fl exibility scale with aggressive communi-cation traits. Journal of Social Behavior and Personality, 12, 531-540.




Overall adequacy: This measure would be an effective complement to the Richmond/McCroskey socio-communi-cative style measure. It would provide an assessment of fl exibility in addition to the consideration of assertiveness and responsiveness.

Additional ReferencesChesebro, J.L. & Martin, M.M. (2003). The relationship between conversational sensitivity, cognitive fl exibility, verbal aggres-

siveness and indirect interpersonal aggressiveness. Communication Research Reports, 20:2, 143-150. Martin, M.M. & Anderson, C.M. (1998) The cognitive fl exibility scale: Three validity studies. Communication Reports, 11,

1-9. Martin M.M. & Anderson, C.M. (2001) The relationship between cognitive fl exibility and affi nity -seeking strategies. Advances

in Psychological Research, 4, 69-76. Martin, M.M. Anderson, C.M. & Thweatt, L.S. (1998). Individuals’ perceptions of their communication behaviors: A validity

study of the relationship between the cognitive fl exibility scale and the cognitive fl exibility scale with aggressive communi-cation traits. Journal of Social Behavior and Personality, 12, 531-540.




College Self-Expression Scale

Author / Developers: J.P. Galassi, J.S. DeLo, M.D. Galassi, and S. Bastien

Address: John P. Galassi, School of Education, University of North Carolina, Chapel Hill, NC 27514.

Source: Galassi, J.P., DeLo, J.S., Galassi, M.D.& Bastien, S. (1974). The college self-expression scale: A measure of assertiveness. Behavior Therapy, 5, 165-71.

Intended context or behavior: Assertiveness



Description: “The CSES is a 50 item, self-report measure. It utilizes a fi ve-point Likert-type scale with 21 posi-tively worded items and 29 negatively worded items. The scale attempts to measure three aspects of assertiveness” (p.168).

Description of scoring: “A total score for the scale is obtained by summing all positively worded items, and reverse scoring and summing all negatively worded items. Low scores are indicative of a generalized nonassertive response pattern” (p.168). Mean scores ranging from 117.91 to 133.00 were reported for various populations studied in this article.

Norm referencing/criterion referencing populations: College students and student teachers (n=200) were used in norm referencing this measure.

Cultural/gender bias: Males achieved slightly higher scores than females.

Validity

Predictive: Low to moderate correlations were reported between CSES scores and rating of assertiveness done by the supervisors of the student teachers.

Face/Content: The article discusses this conceptualization of assertiveness.

Construct: The CSES correlated with several appropriate subscales on the 300 item Adjective Check List.

Reliability

Test/retest: Co-effi cients of .89-.90 were established for various groups taking the measure on a test-retest basis.


Practicality and usefulness: CSES would be helpful in measuring changes in student perceptions of assertive-ness.

Clarity of instructions: Directions for completing the measure were very clear.

Degree of standardization in administration and evaluation of responses: Procedures are highly standardized.

Range of responses or degrees of competence: Means are included in the article. Interpretative ranges are not given.

Primary strengths: This is a quick, effi cient means of assessing assertiveness.

Primary weaknesses: There seemed to be some question about one’s self perception of assertiveness in relation to the perceptions of others.

Overall adequacy: It may be helpful in showing development as a result of instruction in this area. Hong & Cooker (1984) indicated that the CSES can measure the impact of assertiveness training.



College Self-Expression Scale

Author / Developers: J.P. Galassi, J.S. DeLo, M.D. Galassi, and S. Bastien

Address: John P. Galassi, School of Education, University of North Carolina, Chapel Hill, NC 27514.

Source: Galassi, J.P., DeLo, J.S., Galassi, M.D.& Bastien, S. (1974). The college self-expression scale: A measure of assertiveness. Behavior Therapy, 5, 165-71.

Intended context or behavior: Assertiveness



Description: “The CSES is a 50 item, self-report measure. It utilizes a fi ve-point Likert-type scale with 21 posi-tively worded items and 29 negatively worded items. The scale attempts to measure three aspects of assertiveness” (p.168).

Description of scoring: “A total score for the scale is obtained by summing all positively worded items, and reverse scoring and summing all negatively worded items. Low scores are indicative of a generalized nonassertive response pattern” (p.168). Mean scores ranging from 117.91 to 133.00 were reported for various populations studied in this article.

Norm referencing/criterion referencing populations: College students and student teachers (n=200) were used in norm referencing this measure.

Cultural/gender bias: Males achieved slightly higher scores than females.

Validity

Predictive: Low to moderate correlations were reported between CSES scores and rating of assertiveness done by the supervisors of the student teachers.

Face/Content: The article discusses this conceptualization of assertiveness.

Construct: The CSES correlated with several appropriate subscales on the 300 item Adjective Check List.

Reliability

Test/retest: Co-effi cients of .89-.90 were established for various groups taking the measure on a test-retest basis.


Practicality and usefulness: CSES would be helpful in measuring changes in student perceptions of assertive-ness.

Clarity of instructions: Directions for completing the measure were very clear.

Degree of standardization in administration and evaluation of responses: Procedures are highly standardized.

Range of responses or degrees of competence: Means are included in the article. Interpretative ranges are not given.

Primary strengths: This is a quick, effi cient means of assessing assertiveness.

Primary weaknesses: There seemed to be some question about one’s self perception of assertiveness in relation to the perceptions of others.

Overall adequacy: It may be helpful in showing development as a result of instruction in this area. Hong & Cooker (1984) indicated that the CSES can measure the impact of assertiveness training.



Additional ReferencesHong, K. & Cooker, P. G. (1984). Assertion training with Korean college students: Effects on self-expression and anxiety. Per-

sonnel and Guidance Journal , 62,353-58. Ramanaiah, N. V. & Deniston, W. M. (1993). Neo personality inventory profi les of assertive and nonassertive persons. Psycho-

logical Reports, 73,336-339.




Additional ReferencesHong, K. & Cooker, P. G. (1984). Assertion training with Korean college students: Effects on self-expression and anxiety. Per-

sonnel and Guidance Journal , 62,353-58. Ramanaiah, N. V. & Deniston, W. M. (1993). Neo personality inventory profi les of assertive and nonassertive persons. Psycho-

logical Reports, 73,336-339.




Communication Anxiety Inventory

Author / Developers: Steven Booth-Butterfi eld and Malloy Gould

Address: Steven Booth-Butterfi eld, Department of Communication Studies, West Virginia University, Morgan-town, WV 26506.

Source: Booth-Butterfi eld, S. & Gould, M. (1986). The communication anxiety inventory: Validation of state and context communication apprehension. Communication Quarterly, 34, 194-205.

Context of communication behavior for which intended: Communication apprehension



Description: Two instruments are available: one called Form Trait and one called Form State. Form trait includes 21, four-point, Likert-type items assessing communication apprehension in three specifi c contexts: dyadic, small group and public speaking. Form Trait is made up of 20, four-point, Likert-type items.

Description of administration and scoring: Both instruments have instructions included at the beginning, so they can be administered in large settings if necessary. Scores for state apprehension, trait apprehension, and trait ap-prehension in particular contexts are calculated by summing appropriate items.

Norm referencing/criterion referencing populations: A series of studies on the Form Trait measure with a total of 754 high school and college students resulted in a mean of 46.90 with a standard deviation of 10.495. That sample re-sulted in means and standard deviations for trait apprehension in particular contexts of: interpersonal: mean=12.996, sd = 3.204; small group: mean = 15.033, sd = 4.235; and public speaking: mean = 18.874, sd = 5.174. On a sample of 163 undergraduate students, the mean for Form State was 44.06 with a standard deviation of 11.07.

Cultural/Gender bias: Both males and females were included in the scale development sample.

Validity

Predictive: Students were involved in an experimental condition in which they participated in instances of com-munication for each of the three contexts. Following their participation, they completed the CAI scales and other state anxiety measures. Results showed that scores on the different sections of Form Trait correlated best with state apprehension scores based on experience in the respective contexts.

Reliability

Split-half: Split-half reliability for Form Trait: .919; Interpersonal Trait: .665; Small Group Trait: .855; Public Speaking Trait: .894; Form State: .921.

Internal consistency: Alpha reliability for Form Trait: .898; Interpersonal Trait: .654; Small Group Trait: .846; Public Speaking Trait: .887; Form State: .912.


Practicality and usefulness: The scales are easily used.


Identifi cation of a range of responses or degree of competence: Not enough data has been collected to stabilize norms.

Primary strengths: It is easily administered and conceptually sound.

Primary weaknesses: The CAI has been used primarily in research of communication variables rather than as an assessment technique.




Communication Anxiety Inventory

Author / Developers: Steven Booth-Butterfi eld and Malloy Gould

Address: Steven Booth-Butterfi eld, Department of Communication Studies, West Virginia University, Morgan-town, WV 26506.

Source: Booth-Butterfi eld, S. & Gould, M. (1986). The communication anxiety inventory: Validation of state and context communication apprehension. Communication Quarterly, 34, 194-205.




Description: Two instruments are available: one called Form Trait and one called Form State. Form trait includes 21, four-point, Likert-type items assessing communication apprehension in three specifi c contexts: dyadic, small group and public speaking. Form Trait is made up of 20, four-point, Likert-type items.

Description of administration and scoring: Both instruments have instructions included at the beginning, so they can be administered in large settings if necessary. Scores for state apprehension, trait apprehension, and trait ap-prehension in particular contexts are calculated by summing appropriate items.

Norm referencing/criterion referencing populations: A series of studies on the Form Trait measure with a total of 754 high school and college students resulted in a mean of 46.90 with a standard deviation of 10.495. That sample re-sulted in means and standard deviations for trait apprehension in particular contexts of: interpersonal: mean=12.996, sd = 3.204; small group: mean = 15.033, sd = 4.235; and public speaking: mean = 18.874, sd = 5.174. On a sample of 163 undergraduate students, the mean for Form State was 44.06 with a standard deviation of 11.07.

Cultural/Gender bias: Both males and females were included in the scale development sample.

Validity

Predictive: Students were involved in an experimental condition in which they participated in instances of com-munication for each of the three contexts. Following their participation, they completed the CAI scales and other state anxiety measures. Results showed that scores on the different sections of Form Trait correlated best with state apprehension scores based on experience in the respective contexts.

Reliability

Split-half: Split-half reliability for Form Trait: .919; Interpersonal Trait: .665; Small Group Trait: .855; Public Speaking Trait: .894; Form State: .921.

Internal consistency: Alpha reliability for Form Trait: .898; Interpersonal Trait: .654; Small Group Trait: .846; Public Speaking Trait: .887; Form State: .912.


Practicality and usefulness: The scales are easily used.




Primary weaknesses: The CAI has been used primarily in research of communication variables rather than as an assessment technique.




Additional References Booth-Butterfi eld, S. (1991) Ethical issues in the treatment of communication apprehension and avoidance. Communication

Education, 40, 172-179.Booth-Butterfi eld, S., Chory, R. & Beynon, W. (1997) Communication apprehension and health communication and behavior.

Communication Quarterly, 45, 235-250.Ralston, S.M., Ambler, R & Scudder, J.N. Reconsidering the impact of racial differences in college public speaking classroom

student communication anxiety. Communication Reports, 4, 43-50.

Reviewer: Judith Dallinger



Additional References Booth-Butterfi eld, S. (1991) Ethical issues in the treatment of communication apprehension and avoidance. Communication

Education, 40, 172-179.Booth-Butterfi eld, S., Chory, R. & Beynon, W. (1997) Communication apprehension and health communication and behavior.

Communication Quarterly, 45, 235-250.Ralston, S.M., Ambler, R & Scudder, J.N. Reconsidering the impact of racial differences in college public speaking classroom

student communication anxiety. Communication Reports, 4, 43-50.





Author / Developers: Keith Weber and Brian Patterson, West Virginia University, Department of Com-munication Studies, Morgantown, WV 26506

Source: Weber, K.D. & Batterson, B.R. (1996) Construction and validation of a communication based emotional support scale. Communication Research Reports, 13:1, 68-76.

Intended context or behavior: Communication of emotional support


Time required: Approximately fi ve minutes

Description: Subjects respond to 13 statements on a fi ve-point Likert-type scale. The statements ask them to assess the communication behavior (speaking and listening) of their romantic partner.

Scoring description: Responses range from one (almost never true) to fi ve (almost always true). Three items are reversed for scoring. Central tendencies and ranges are not reported.

Norm/criterion reference populations: College students

Validity

Face/content: Items were selected from the Interpersonal Competence Questionnaire (Buhrmester, D et al (1988) Five domains of interpersonal competence in peer relationships. Journal of Personality and Social Psychology, 55, 991-1008) and from McCroskey & Richmond’s conceptualization of responsiveness (McCroskey, J.C. & Rich-mond, V.P. (1996) Fundamentals of Human Communication. Illinois: Waveland, Press, Inc.)

Construct: The scale was correlated with measures of relational solidarity and quality as well as measures of “re-ceptivity, similarity, equality, dominance, formality, attributional confi dence, relational depth, relational confl ict and relational support” (Weber, Johnson & Corrigan, 2004, p.322).

Reliability

Internal consistency: A Cronbach alpha of .93 was reported.


Practicality and usefulness: If study of interpersonal communication is designed to increase the repertoire of com-munication behaviors that a student exhibits, this measure could be a viable pre-post measure of a student’s ability to communicate support. The statements could apply to friendships as well as romantic partners.

Clarity of instructions: The instructions are clear.

Standardization in administration and evaluation: Administration can be easily standardized.

Range of responses or degrees of competence: The fi ve-point scale seems adequate.

Primary strengths: This is usable measure that focuses on one important dimension of interpersonal interaction, communicating emotional support.

Primary weaknesses: Median scores and ranges are not reported.

Additional ReferencesAvtgis, T.A. (2002). Marital support giving as a function of attributional confi dence. Communication Research Reports, 17,

357-365.Avtgis, T.A., Martin, M.M. & Rocca, K.A. (2002). Social support and perceived understanding in the brother relationship.

Communication Research Reports, 17, 407-414.




Author / Developers: Keith Weber and Brian Patterson, West Virginia University, Department of Com-munication Studies, Morgantown, WV 26506

Source: Weber, K.D. & Batterson, B.R. (1996) Construction and validation of a communication based emotional support scale. Communication Research Reports, 13:1, 68-76.

Intended context or behavior: Communication of emotional support


Time required: Approximately fi ve minutes

Description: Subjects respond to 13 statements on a fi ve-point Likert-type scale. The statements ask them to assess the communication behavior (speaking and listening) of their romantic partner.

Scoring description: Responses range from one (almost never true) to fi ve (almost always true). Three items are reversed for scoring. Central tendencies and ranges are not reported.


Validity

Face/content: Items were selected from the Interpersonal Competence Questionnaire (Buhrmester, D et al (1988) Five domains of interpersonal competence in peer relationships. Journal of Personality and Social Psychology, 55, 991-1008) and from McCroskey & Richmond’s conceptualization of responsiveness (McCroskey, J.C. & Rich-mond, V.P. (1996) Fundamentals of Human Communication. Illinois: Waveland, Press, Inc.)

Construct: The scale was correlated with measures of relational solidarity and quality as well as measures of “re-ceptivity, similarity, equality, dominance, formality, attributional confi dence, relational depth, relational confl ict and relational support” (Weber, Johnson & Corrigan, 2004, p.322).

Reliability



Practicality and usefulness: If study of interpersonal communication is designed to increase the repertoire of com-munication behaviors that a student exhibits, this measure could be a viable pre-post measure of a student’s ability to communicate support. The statements could apply to friendships as well as romantic partners.


Standardization in administration and evaluation: Administration can be easily standardized.

Range of responses or degrees of competence: The fi ve-point scale seems adequate.

Primary strengths: This is usable measure that focuses on one important dimension of interpersonal interaction, communicating emotional support.

Primary weaknesses: Median scores and ranges are not reported.

Additional ReferencesAvtgis, T.A. (2002). Marital support giving as a function of attributional confi dence. Communication Research Reports, 17,

357-365.Avtgis, T.A., Martin, M.M. & Rocca, K.A. (2002). Social support and perceived understanding in the brother relationship.

Communication Research Reports, 17, 407-414.



Punyanunt-Carter, Narissra M., & Wagner, T. R..(2005). Communication based emotional support differences between profes-sors and teaching assistants. Education, 125:4, 569-574.

Weber, K. Johnson, A., & Corrigan, M. (2004). Communicating emotional support and its relationship to feelings of being understood, trust, and self-disclosure. Communication Research Reports, 21:3, 316-323.




Punyanunt-Carter, Narissra M., & Wagner, T. R..(2005). Communication based emotional support differences between profes-sors and teaching assistants. Education, 125:4, 569-574.

Weber, K. Johnson, A., & Corrigan, M. (2004). Communicating emotional support and its relationship to feelings of being understood, trust, and self-disclosure. Communication Research Reports, 21:3, 316-323.




Communication Competency Assessment Instrument-Revised Edition

Author / Developers: Rebecca B. Rubin

Address: R.B. Rubin, c/o School of Communication Studies, Kent State University, Kent, OH, 44242-0001

Source: Triton Video, 5503 Cates Dr., Greensboro, NC, 27409, 336-854-9430, or [email protected]

Intended context or behavior: Public Speaking, interaction abilities, listening

Academic level for which intended: College students


Description: The CCAI includes three parts. In the fi rst part subjects are given fi ve minutes to prepare a speech on a topic of their choice. The three-minute speech is then presented in a classroom setting to a rater. Students then view a videotape that simulates the fi rst day of a fi ctitious college class. The rater than discusses the tape with the student asking specifi c questions to probe listening comprehension and interaction abilities.

Description of scoring: The raters assess 19 competencies during the speech and interview sessions. The subject is evaluated on a fi ve-point scale for each competency. For each of the 19 competencies, raters are provided with clear descriptions of each of the fi ve levels. Rubin has cautioned that the instrument should be considered as a global measure of communication competence although individual items may indicate strengths and weaknesses.

Norm referencing/criterion referencing populations: Various groups of college students have been used in the development and testing of this instrument.

Cultural/gender bias: Validation data indicates that the test does not appear to be biased on the basis of gender or ethnic differences. In the technical manual, Rubin discusses many of the common forms of bias that should be addressed in rater recruiting, training, and monitoring. Subsequent research suggested that Whites performed sig-nifi cantly higher than did Blacks, Asians or Hispanics (Powell & Avila, 1986).

Validity

Predictive: Previous studies have found that the CCAI was an effective predictor of success for student teachers, and have reported that those with higher CCAI scores were more likely to persist in college. The CCAI has been positively correlated with GPA, high school speaking experiences, and college speaking experiences when the sub-jects were freshmen and sophomores.

Concurrent: Some moderate correlations have been reported between the CCAI and the Communication Compe-tence Self-Report Questionnaire although for the most part subjects seem to overestimate their competence.

Face/Content: The domain of nineteen competencies was drawn from previous taxonomies of functional com-munication abilities. Speech communication experts considered whether the specifi c items on the assessment ac-curately tapped the abilities they were intended to measure.

Reliability

Scoring: Inter-rater reliability coeffi cients have been good. The rating scale contains good descriptions of the varia-tions of behaviors being assessed and consistency was high among raters particularly if they receive training.


Practicality and usefulness: This is one of the few measures available which actually evaluates the communication abilities of the subject rather than relying on a self-report of these abilities. The drawback is that a tremendous time commitment is necessary when using this measure on a large scale.

Clarity of instructions: The directions and probes are very clear. Further research has found that adding a bit more information to each probe can improve CCAI performance.



Communication Competency Assessment Instrument-Revised Edition

Author / Developers: Rebecca B. Rubin

Address: R.B. Rubin, c/o School of Communication Studies, Kent State University, Kent, OH, 44242-0001

Source: Triton Video, 5503 Cates Dr., Greensboro, NC, 27409, 336-854-9430, or [email protected]

Intended context or behavior: Public Speaking, interaction abilities, listening

Academic level for which intended: College students


Description: The CCAI includes three parts. In the fi rst part subjects are given fi ve minutes to prepare a speech on a topic of their choice. The three-minute speech is then presented in a classroom setting to a rater. Students then view a videotape that simulates the fi rst day of a fi ctitious college class. The rater than discusses the tape with the student asking specifi c questions to probe listening comprehension and interaction abilities.

Description of scoring: The raters assess 19 competencies during the speech and interview sessions. The subject is evaluated on a fi ve-point scale for each competency. For each of the 19 competencies, raters are provided with clear descriptions of each of the fi ve levels. Rubin has cautioned that the instrument should be considered as a global measure of communication competence although individual items may indicate strengths and weaknesses.

Norm referencing/criterion referencing populations: Various groups of college students have been used in the development and testing of this instrument.

Cultural/gender bias: Validation data indicates that the test does not appear to be biased on the basis of gender or ethnic differences. In the technical manual, Rubin discusses many of the common forms of bias that should be addressed in rater recruiting, training, and monitoring. Subsequent research suggested that Whites performed sig-nifi cantly higher than did Blacks, Asians or Hispanics (Powell & Avila, 1986).

Validity

Predictive: Previous studies have found that the CCAI was an effective predictor of success for student teachers, and have reported that those with higher CCAI scores were more likely to persist in college. The CCAI has been positively correlated with GPA, high school speaking experiences, and college speaking experiences when the sub-jects were freshmen and sophomores.

Concurrent: Some moderate correlations have been reported between the CCAI and the Communication Compe-tence Self-Report Questionnaire although for the most part subjects seem to overestimate their competence.

Face/Content: The domain of nineteen competencies was drawn from previous taxonomies of functional com-munication abilities. Speech communication experts considered whether the specifi c items on the assessment ac-curately tapped the abilities they were intended to measure.

Reliability

Scoring: Inter-rater reliability coeffi cients have been good. The rating scale contains good descriptions of the varia-tions of behaviors being assessed and consistency was high among raters particularly if they receive training.


Practicality and usefulness: This is one of the few measures available which actually evaluates the communication abilities of the subject rather than relying on a self-report of these abilities. The drawback is that a tremendous time commitment is necessary when using this measure on a large scale.

Clarity of instructions: The directions and probes are very clear. Further research has found that adding a bit more information to each probe can improve CCAI performance.



Standardization in administration and evaluation of responses: The descriptions of each level of each compe-tency are very defi nite and provide for a standardized approach.

Range of responses or degrees of competence: The evaluation scale allows for a variety of responses rather than just one right answer.

Primary strengths: Since the probes are spoken and responses are oral, the measure does not test reading and writ-ing abilities. The situations used in the probes are familiar to students, and the classroom setting is naturalistic. The stress of the testing situation can be kept to a minimum especially if the rater is effectively trained. The stimulus for the listening test is videotaped, and the responses to this section are minimal and do not test memory.

Primary weaknesses: Presently there are no alternative forms of this measure. The public speaking portion is not given to an audience and therefore might not be a totally accurate assessment of a student’s speaking ability.

Overall adequacy: The measure would be especially effective for departments seeking to assess the communica-tion abilities for a representative sample of students.

Additional ReferencesPowell, R. G.& Avila, D.R. (1986). Ethnicity, communication competency and classroom success: a question of assessment.

Western Journal of Speech Communication, 50, 269-278.Rubin, R B. (1986) A response to “Ethnicity, communication competency and classroom success”. Western Journal of Speech

Communication. 50, 279-282.Rubin, R.B. (1985). The validity of the communication competency assessment instrument. Communication Monographs, 52,

173-185.Rubin. R.B., Graham, E.E. & Mignerey, J.T. (1990). A longitudinal study of collegestudents’ communication competence.

Communication Education, 39, 1-1.Rubin, R.B & Welch, S.A. (1995). Performance-based assessment of high school speech instruction. Communication Educa-

tion, 44, 30-40.




Standardization in administration and evaluation of responses: The descriptions of each level of each compe-tency are very defi nite and provide for a standardized approach.

Range of responses or degrees of competence: The evaluation scale allows for a variety of responses rather than just one right answer.

Primary strengths: Since the probes are spoken and responses are oral, the measure does not test reading and writ-ing abilities. The situations used in the probes are familiar to students, and the classroom setting is naturalistic. The stress of the testing situation can be kept to a minimum especially if the rater is effectively trained. The stimulus for the listening test is videotaped, and the responses to this section are minimal and do not test memory.

Primary weaknesses: Presently there are no alternative forms of this measure. The public speaking portion is not given to an audience and therefore might not be a totally accurate assessment of a student’s speaking ability.

Overall adequacy: The measure would be especially effective for departments seeking to assess the communica-tion abilities for a representative sample of students.

Additional ReferencesPowell, R. G.& Avila, D.R. (1986). Ethnicity, communication competency and classroom success: a question of assessment.

Western Journal of Speech Communication, 50, 269-278.Rubin, R B. (1986) A response to “Ethnicity, communication competency and classroom success”. Western Journal of Speech

Communication. 50, 279-282.Rubin, R.B. (1985). The validity of the communication competency assessment instrument. Communication Monographs, 52,

173-185.Rubin. R.B., Graham, E.E. & Mignerey, J.T. (1990). A longitudinal study of collegestudents’ communication competence.

Communication Education, 39, 1-1.Rubin, R.B & Welch, S.A. (1995). Performance-based assessment of high school speech instruction. Communication Educa-

tion, 44, 30-40.





Author / Developers: S.P. Morreale, M.R. Moore, D.S. Tatum & L. Webster





Description: This instrument presents a standardized form and approach for evaluating public speaking behav-iors. It is based upon eight competencies, four related to preparation and four related to delivery. The second edition includes a holistic form as well the original atomistic form. In the newest, both electronic and non-electronic sources are included in the evaluation of supporting material. The manual includes a discussion of each competency and an explanation of how each would be demonstrated at excellent, satisfactory, and unsatisfactory levels along with detailed instruction on the preparation of a training video.




Validity



Reliability








Author / Developers: S.P. Morreale, M.R. Moore, D.S. Tatum & L. Webster





Description: This instrument presents a standardized form and approach for evaluating public speaking behav-iors. It is based upon eight competencies, four related to preparation and four related to delivery. The second edition includes a holistic form as well the original atomistic form. In the newest, both electronic and non-electronic sources are included in the evaluation of supporting material. The manual includes a discussion of each competency and an explanation of how each would be demonstrated at excellent, satisfactory, and unsatisfactory levels along with detailed instruction on the preparation of a training video.




Validity



Reliability







Connected Classroom Climate Inventory

Author / Developers: Karen Kangas Dwyer, Shereen G. Bingham, Robert E. Carlson, Marshall Prisbell, Ana M. Cruz, and Dennis A. Fus

Address: Karen Kangas Dwyer, University of Nebraska-Omaha, Department of Communication, 60 & Dodge Sts ASH 107J, Omaha, NE, 68182-0112. Phone: 402-554-2253. E-mail: [email protected]

Source: Dwyer, K. K., Bingham, S. G., Carlson, R. E., Prisbell, M., Cruz, A. M., & Fus, D. A. (2004). Communica-tion and connectedness in the classroom: Development of the Connected Classroom Climate Inventory. Communi-cation Research Reports, 21(3), 264-272.

Intended Context or Behavior: Instructional communication



Description: This 18-item instrument measures student perceptions of a supportive and cooperative communication environment among students in university classrooms.

Scoring Description: Respondents rate on a fi ve-point Likert-type scale the extent to which they agree or disagree with statements. A score of fi ve indicates “Strongly Agree” and a score of one indicates “Strongly Disagree.” How-ever directions for totaling the scores are not provided. It is assumed that item scores are summed.

Norm/Criterion Reference Populations: This instrument is based on the population of college students. The ar-ticle provides information about the range of respondent scores that can be used as a basis for comparison through norms. There is no information about specifi c criteria that respondents are expected to know or be able to do.


Validity


Face/Content: Looking at the Connected Classroom Climate Inventory (CCCI) on its face, the items appear to operationalize the construct of a connected climate. The article provides a detailed description of how to defi ne the content of connectedness.

Construct: Four global validity items were tested with the CCCI to explore criterion-related validity. All four global items were signifi cantly correlated to the CCCI. Therefore, the CCCI has initial construct validity since its opera-tionalization accurately refl ects its construct.


Reliability



Internal Consistency: Cronbach’s alpha of .94 was reported.


Practicality and Usefulness: Authors believe the CCCI may be useful for assessment purposes, such as part of an overall basic communication course assessment process. On a classroom level, the CCCI could be used as feedback for new instructors to help in their professional development. On an institutional level, this feedback could be used to monitor student retention efforts or improve instructional development programs.



Connected Classroom Climate Inventory

Author / Developers: Karen Kangas Dwyer, Shereen G. Bingham, Robert E. Carlson, Marshall Prisbell, Ana M. Cruz, and Dennis A. Fus

Address: Karen Kangas Dwyer, University of Nebraska-Omaha, Department of Communication, 60 & Dodge Sts ASH 107J, Omaha, NE, 68182-0112. Phone: 402-554-2253. E-mail: [email protected]

Source: Dwyer, K. K., Bingham, S. G., Carlson, R. E., Prisbell, M., Cruz, A. M., & Fus, D. A. (2004). Communica-tion and connectedness in the classroom: Development of the Connected Classroom Climate Inventory. Communi-cation Research Reports, 21(3), 264-272.

Intended Context or Behavior: Instructional communication



Description: This 18-item instrument measures student perceptions of a supportive and cooperative communication environment among students in university classrooms.

Scoring Description: Respondents rate on a fi ve-point Likert-type scale the extent to which they agree or disagree with statements. A score of fi ve indicates “Strongly Agree” and a score of one indicates “Strongly Disagree.” How-ever directions for totaling the scores are not provided. It is assumed that item scores are summed.

Norm/Criterion Reference Populations: This instrument is based on the population of college students. The ar-ticle provides information about the range of respondent scores that can be used as a basis for comparison through norms. There is no information about specifi c criteria that respondents are expected to know or be able to do.


Validity


Face/Content: Looking at the Connected Classroom Climate Inventory (CCCI) on its face, the items appear to operationalize the construct of a connected climate. The article provides a detailed description of how to defi ne the content of connectedness.

Construct: Four global validity items were tested with the CCCI to explore criterion-related validity. All four global items were signifi cantly correlated to the CCCI. Therefore, the CCCI has initial construct validity since its opera-tionalization accurately refl ects its construct.


Reliability



Internal Consistency: Cronbach’s alpha of .94 was reported.


Practicality and Usefulness: Authors believe the CCCI may be useful for assessment purposes, such as part of an overall basic communication course assessment process. On a classroom level, the CCCI could be used as feedback for new instructors to help in their professional development. On an institutional level, this feedback could be used to monitor student retention efforts or improve instructional development programs.



Clarity of Instructions: The 18 items of the CCCI are reprinted in the article. Instructions are provided in the ar-ticle. Students were instructed to answer the questions in reference to their present public speaking class and rate each statement on a scale of one to fi ve (strongly disagree to strongly agree). The article does not include instruc-tions for tabulating scores.

Standardization in Administration and Evaluation: The instrument is still in development, which would explain the lack of standardization in its administration. There is no description of how this instrument was used for evalu-ation of teachers or students.

Range of Reponses or Degrees of Competence: The range of response scores was 18 to 90. The mean score was 70.97 and the standard deviation of 9.91. The means on all 18 items were positively skewed from neutral to strongly agree. There is no description of how to determine anchor points of high, middle, or low scores. This instrument does not provide a range of scores that constitute various degrees of competence for this scale.

Primary Strengths: This instrument is designed to be easy to complete and take a minimal amount of time to ad-minister. The CCCI instrument could be administered in any small university classroom because it is not limited to course content.

Primary Weaknesses: The CCCI has only been tested in sections of a public speaking course in which there is fre-quent student interaction and small enrollments (between ten to 23 students in each section). It has not been tested in large lecture-hall type classrooms.

Overall Adequacy: More research is needed to test the reliability and usefulness of the CCCI, its association with other variables, and its predictive validity before it is used on a larger scale.




Clarity of Instructions: The 18 items of the CCCI are reprinted in the article. Instructions are provided in the ar-ticle. Students were instructed to answer the questions in reference to their present public speaking class and rate each statement on a scale of one to fi ve (strongly disagree to strongly agree). The article does not include instruc-tions for tabulating scores.

Standardization in Administration and Evaluation: The instrument is still in development, which would explain the lack of standardization in its administration. There is no description of how this instrument was used for evalu-ation of teachers or students.

Range of Reponses or Degrees of Competence: The range of response scores was 18 to 90. The mean score was 70.97 and the standard deviation of 9.91. The means on all 18 items were positively skewed from neutral to strongly agree. There is no description of how to determine anchor points of high, middle, or low scores. This instrument does not provide a range of scores that constitute various degrees of competence for this scale.

Primary Strengths: This instrument is designed to be easy to complete and take a minimal amount of time to ad-minister. The CCCI instrument could be administered in any small university classroom because it is not limited to course content.

Primary Weaknesses: The CCCI has only been tested in sections of a public speaking course in which there is fre-quent student interaction and small enrollments (between ten to 23 students in each section). It has not been tested in large lecture-hall type classrooms.

Overall Adequacy: More research is needed to test the reliability and usefulness of the CCCI, its association with other variables, and its predictive validity before it is used on a larger scale.




Conversational Sensitivity Scale

Author / Developers: J.A. Daly, A.L. Vangelisti, and S.M. Daughton

Address: John A. Daly, Department of Speech Communication, University of Texas, Austin, TX, 78712

Source: Daly, J.A., Vangelisti, A.L. & Daughton, S.M. (1988). The nature and correlates of conversational sensitiv-ity. Human Communication Research, 14, 167-202.

Context or communication behavior for which intended: Interaction/Interpersonal abilities



Description: This 36-item self-report is designed to measure the individual’s competence in conversational situations. Conversational sensitivity includes eight major components such as perceptiveness, memory, tact, and listening.

Description of scoring: Scoring information will need to be obtained from the senior author. In one study con-ducted using this measure, subjects with a mean score of 109.21 were classifi ed as the low sensitivity group and a mean score of 139.33 as the high sensitivity group.

Norm referencing/criterion referencing populations: Various groups of undergraduate students were used in studies of this instrument.

Tests for cultural/gender bias: No differences were found between the scores of males and females.

Validity

Predictive: Scores on the CSM did relate to certain behavioral correlates.

Concurrent: Positive correlations were reported between the CSM and Riggio’s Social Skills Inventory and Martin & Rubin’s Cognitive Flexibility Scale.

Face/Content: The theoretical underpinnings of the conversational sensitivity construct were explained in the ar-ticle.

Construct: Measures of various communication related variables such as apprehension, empathy, loneliness, self-monitoring, assertiveness, self-esteem, and rhetorical sensitivity were appropriately related to conversational sen-sitivity.

Reliability

Internal consistency: Alpha coeffi cients above .80 were reported.


Practicality and usefulness: With more elaboration, this measure might be used on a pre/post basis to measure development of interpersonal abilities.

Identifi cation of a range of responses or degrees of competence: High and low sensitivity groups can be identi-fi ed.

Primary strengths: The underlying factors identifi ed in this measure relate to many of the concepts and abilities that we consider in interpersonal instruction. This might be one way to measure such instruction.

Primary weaknesses: The measure is designed more to defi ne and explain concept rather than measure the devel-opment of ability.

Overall adequacy: This offers a viable alternative as an assessment instrument and model.




Conversational Sensitivity Scale

Author / Developers: J.A. Daly, A.L. Vangelisti, and S.M. Daughton

Address: John A. Daly, Department of Speech Communication, University of Texas, Austin, TX, 78712

Source: Daly, J.A., Vangelisti, A.L. & Daughton, S.M. (1988). The nature and correlates of conversational sensitiv-ity. Human Communication Research, 14, 167-202.

Context or communication behavior for which intended: Interaction/Interpersonal abilities



Description: This 36-item self-report is designed to measure the individual’s competence in conversational situations. Conversational sensitivity includes eight major components such as perceptiveness, memory, tact, and listening.

Description of scoring: Scoring information will need to be obtained from the senior author. In one study con-ducted using this measure, subjects with a mean score of 109.21 were classifi ed as the low sensitivity group and a mean score of 139.33 as the high sensitivity group.

Norm referencing/criterion referencing populations: Various groups of undergraduate students were used in studies of this instrument.

Tests for cultural/gender bias: No differences were found between the scores of males and females.

Validity

Predictive: Scores on the CSM did relate to certain behavioral correlates.

Concurrent: Positive correlations were reported between the CSM and Riggio’s Social Skills Inventory and Martin & Rubin’s Cognitive Flexibility Scale.

Face/Content: The theoretical underpinnings of the conversational sensitivity construct were explained in the ar-ticle.

Construct: Measures of various communication related variables such as apprehension, empathy, loneliness, self-monitoring, assertiveness, self-esteem, and rhetorical sensitivity were appropriately related to conversational sen-sitivity.

Reliability



Practicality and usefulness: With more elaboration, this measure might be used on a pre/post basis to measure development of interpersonal abilities.

Identifi cation of a range of responses or degrees of competence: High and low sensitivity groups can be identi-fi ed.

Primary strengths: The underlying factors identifi ed in this measure relate to many of the concepts and abilities that we consider in interpersonal instruction. This might be one way to measure such instruction.

Primary weaknesses: The measure is designed more to defi ne and explain concept rather than measure the devel-opment of ability.

Overall adequacy: This offers a viable alternative as an assessment instrument and model.




Chesbro, J.L. (1999). The relationship between listening styles and conversational sensitivity. Communication Research Re-ports, 16, 233-238.

Chesbro, J.L. & Martin, M.M. (2003). The relationship between conversational sensitivity, cognitive fl exibility, verbal aggres-siveness and indirect interpersonal aggressiveness. Communication Research Reports, 20:2, 143.150.




Chesbro, J.L. (1999). The relationship between listening styles and conversational sensitivity. Communication Research Re-ports, 16, 233-238.

Chesbro, J.L. & Martin, M.M. (2003). The relationship between conversational sensitivity, cognitive fl exibility, verbal aggres-siveness and indirect interpersonal aggressiveness. Communication Research Reports, 20:2, 143.150.




Conversational Skills Rating Scale, 2nd Edition

Author / Developers: Brian H. Spitzberg

Address: Brian H. Spitzberg, Dept. of Speech Comm., San Diego State University, San Diego, CA, 92182

Source: National Communication Association, 1765 N Street, NW, Washington, D.C., 20036. www.natcom.org

Context or communication behavior for which intended: Interpersonal communication


Time required: One hour training and fi ve practice assessments when using rater format; 10-15 minutes for self-report format.

Description: The CSRS includes 25 items related to specifi c behaviors considered appropriate to interpersonal interactions rated on a fi ve-point Likert-type scale and fi ve semantic differential items of overall performance. Subscales for altercentrism, composure, expressiveness, and interaction management are identifi ed. Sample forms are provided to use this measure in a variety of formats. First, as a self-report, respondents recall a recent conversa-tion and rate their own performance. A trained rater or instructor can also be used to evaluate the interaction of two individuals. A conversational partner can also rate another individual Frequency and rating versions of the form are also included for each of the three formats.

Description of scoring: A training guide for raters is provided. The items are totaled for an overall score and sub-scales can also be tallied. Score interpretation is not offered.

Norm referencing/criterion referencing populations: College students

Tests for cultural/gender bias: No overall studies were reported.

Validity

Concurrent: The CSRS reported appropriate correlations with other measures of altercentrism.

Face/Content: The items selected for the scale are based upon an extensive four-stage process. These stages and the relational competence model on which the measure is based are provided in the booklet.

Construct: “There is clearly evidence that the SCRS, and its component constructs , are strongly and sensibly related to motivation to communicate, knowledge of communication, the production of ‘typical’ episodes, molar perceptions of self competence and partner competence, and contextual expectancy fulfi llment, especially when these constructs are measured at the same level of episodic frame “ (p. 10).

Reliability


Interrater reliability: Coeffi cients of .75 were reported.


Practicality and usefulness: This instrument is suitable as a beginning point for the discussion of interpersonal competence and as a diagnostic tool in developing interpersonal competence. It could also be used as a pre/post measure of course effectiveness. It has been used at Edison Community College and Samford University as a part of their institutional assessment plans.

Clarity of instructions: The directions for completing the measure are very clear. The training guide is helpful in orienting raters.

Degree of standardization in administration and evaluation of responses: The items are straightforward and clear, and the guide provides further explanation to standardized the rating process.



Conversational Skills Rating Scale, 2nd Edition

Author / Developers: Brian H. Spitzberg

Address: Brian H. Spitzberg, Dept. of Speech Comm., San Diego State University, San Diego, CA, 92182

Source: National Communication Association, 1765 N Street, NW, Washington, D.C., 20036. www.natcom.org

Context or communication behavior for which intended: Interpersonal communication


Time required: One hour training and fi ve practice assessments when using rater format; 10-15 minutes for self-report format.

Description: The CSRS includes 25 items related to specifi c behaviors considered appropriate to interpersonal interactions rated on a fi ve-point Likert-type scale and fi ve semantic differential items of overall performance. Subscales for altercentrism, composure, expressiveness, and interaction management are identifi ed. Sample forms are provided to use this measure in a variety of formats. First, as a self-report, respondents recall a recent conversa-tion and rate their own performance. A trained rater or instructor can also be used to evaluate the interaction of two individuals. A conversational partner can also rate another individual Frequency and rating versions of the form are also included for each of the three formats.

Description of scoring: A training guide for raters is provided. The items are totaled for an overall score and sub-scales can also be tallied. Score interpretation is not offered.


Tests for cultural/gender bias: No overall studies were reported.

Validity

Concurrent: The CSRS reported appropriate correlations with other measures of altercentrism.

Face/Content: The items selected for the scale are based upon an extensive four-stage process. These stages and the relational competence model on which the measure is based are provided in the booklet.

Construct: “There is clearly evidence that the SCRS, and its component constructs , are strongly and sensibly related to motivation to communicate, knowledge of communication, the production of ‘typical’ episodes, molar perceptions of self competence and partner competence, and contextual expectancy fulfi llment, especially when these constructs are measured at the same level of episodic frame “ (p. 10).

Reliability


Interrater reliability: Coeffi cients of .75 were reported.


Practicality and usefulness: This instrument is suitable as a beginning point for the discussion of interpersonal competence and as a diagnostic tool in developing interpersonal competence. It could also be used as a pre/post measure of course effectiveness. It has been used at Edison Community College and Samford University as a part of their institutional assessment plans.

Clarity of instructions: The directions for completing the measure are very clear. The training guide is helpful in orienting raters.

Degree of standardization in administration and evaluation of responses: The items are straightforward and clear, and the guide provides further explanation to standardized the rating process.



Identifi cation of a range of responses or degrees of competence: No interpretative ranges were offered.

Primary strengths: This is a very clear, easy to use measure that can be adapted to a variety of situations.

Primary weaknesses: Follow-up on its adequacy for assessment purposes has not been summarized. As the CSRS gains users, such a report would be helpful.

Overall adequacy: The CSRS would provide an effective, standardized protocol for assessing interpersonal ef-fectiveness.

Additional ReferencesEgland, K. L.& Spitzberg, B. H. (1996). Flirtation and conversational competence in cross-sex platonic and romantic relation-

ships. Communication Reports, 9:2, 105-117. Guerrero, L. K. & Jones, S. M. (2005). Differences in conversational skills as a function of attachment style: a follow-up study.

Communication Quarterly, 53:3, 305-321. Spitzberg, B.H. & Hurt, H.T. (1987). The measurement of interpersonal skills in instructional contexts. Communication Educa-

tion, 36, 28-45.Waldron, V. R., and Lavitt, M. R. (2000). Welfare-to-work: Assessing communication competencies and client outcomes in a

job training program. Southern Communication Journal. 66 :1, 1-15.




Identifi cation of a range of responses or degrees of competence: No interpretative ranges were offered.

Primary strengths: This is a very clear, easy to use measure that can be adapted to a variety of situations.

Primary weaknesses: Follow-up on its adequacy for assessment purposes has not been summarized. As the CSRS gains users, such a report would be helpful.

Overall adequacy: The CSRS would provide an effective, standardized protocol for assessing interpersonal ef-fectiveness.

Additional ReferencesEgland, K. L.& Spitzberg, B. H. (1996). Flirtation and conversational competence in cross-sex platonic and romantic relation-

ships. Communication Reports, 9:2, 105-117. Guerrero, L. K. & Jones, S. M. (2005). Differences in conversational skills as a function of attachment style: a follow-up study.

Communication Quarterly, 53:3, 305-321. Spitzberg, B.H. & Hurt, H.T. (1987). The measurement of interpersonal skills in instructional contexts. Communication Educa-

tion, 36, 28-45.Waldron, V. R., and Lavitt, M. R. (2000). Welfare-to-work: Assessing communication competencies and client outcomes in a

job training program. Southern Communication Journal. 66 :1, 1-15.





Author / Developers: Carolyn M. Anderson, Matthew M. Martin, and Dominic A. Infante.

Address: Carolyn M. Anderson, University of Akron, School of Communication, Kolbe Hall 108, Akron, OH 44325-1003. Phone: 330-972-6218. E-mail: [email protected]

Source: Anderson, C. M., Martin, M. M., & Infante, D. A. (1998). Decision-making collaboration scale: Tests of validity. Communication Research Reports, 15(3), 245-255.

Intended Context or Behavior: Interpersonal decision-making



Description: This 13-item instrument measures willingness to participate collaboratively with another person in decision-making situations.

Scoring Description: Respondents rate on a fi ve-point Likert-type scale how true each of the statements are for them concerning their communication behavior in decision-making situations involving another person. A score of fi ve indicates “almost always true” and a score of one indicates “almost never true.” However, directions for totaling the scores are not provided.



Validity

Concurrent: Willingness to participate collaboratively in decision-making was positively correlated to argumen-tativeness (r = .58), willingness to communicate (r = .43), and interpersonal communication competence (r = .52). Willingness to participate collaboratively in decision-making was negatively correlated to verbal aggressiveness (r = -.19). Therefore the Decision-Making Collaboration Scale is able to distinguish among groups of people with various communication traits and has concurrent validity.

Face/Content: Looking at the Decision-Making Collaboration Scale on its face, the items appear to operationalize the construct of willingness to collaborate in decision-making through participatory decision-making, assertiveness, and negotiation. The article provides a detailed description of how to defi ne the content of willingness to collaborate in decision-making.

Construct: A positive relationship was supported between a person’s self-report of his or her willingness to par-ticipate collaboratively in decision-making and someone else’s perception of that person’s willingness to participate collaboratively in decision-making. Based on a sample of 82 college students and their friends, the correlation between self-report and other-report ratings was moderate (r = .52). Therefore the Decision Making Collaboration Scale has initial construct validity since its operationalization accurately refl ects its construct.

Predictive: Authors speculate that willingness to participate collaboratively in decision-making should make a dif-ference in goal attainment, job completion, and problem solving. However more research is needed to test the ability of the Decision-Making Collaboration Scale to correctly predict something it should theoretically predict.

Reliability

Alternate Form: The Decision-Making Collaboration Scale can be conducted as a self-report or as an other-report.


Internal Consistency: Alpha coeffi cient of .81 was reported.




Author / Developers: Carolyn M. Anderson, Matthew M. Martin, and Dominic A. Infante.

Address: Carolyn M. Anderson, University of Akron, School of Communication, Kolbe Hall 108, Akron, OH 44325-1003. Phone: 330-972-6218. E-mail: [email protected]

Source: Anderson, C. M., Martin, M. M., & Infante, D. A. (1998). Decision-making collaboration scale: Tests of validity. Communication Research Reports, 15(3), 245-255.

Intended Context or Behavior: Interpersonal decision-making



Description: This 13-item instrument measures willingness to participate collaboratively with another person in decision-making situations.

Scoring Description: Respondents rate on a fi ve-point Likert-type scale how true each of the statements are for them concerning their communication behavior in decision-making situations involving another person. A score of fi ve indicates “almost always true” and a score of one indicates “almost never true.” However, directions for totaling the scores are not provided.



Validity

Concurrent: Willingness to participate collaboratively in decision-making was positively correlated to argumen-tativeness (r = .58), willingness to communicate (r = .43), and interpersonal communication competence (r = .52). Willingness to participate collaboratively in decision-making was negatively correlated to verbal aggressiveness (r = -.19). Therefore the Decision-Making Collaboration Scale is able to distinguish among groups of people with various communication traits and has concurrent validity.

Face/Content: Looking at the Decision-Making Collaboration Scale on its face, the items appear to operationalize the construct of willingness to collaborate in decision-making through participatory decision-making, assertiveness, and negotiation. The article provides a detailed description of how to defi ne the content of willingness to collaborate in decision-making.

Construct: A positive relationship was supported between a person’s self-report of his or her willingness to par-ticipate collaboratively in decision-making and someone else’s perception of that person’s willingness to participate collaboratively in decision-making. Based on a sample of 82 college students and their friends, the correlation between self-report and other-report ratings was moderate (r = .52). Therefore the Decision Making Collaboration Scale has initial construct validity since its operationalization accurately refl ects its construct.

Predictive: Authors speculate that willingness to participate collaboratively in decision-making should make a dif-ference in goal attainment, job completion, and problem solving. However more research is needed to test the ability of the Decision-Making Collaboration Scale to correctly predict something it should theoretically predict.

Reliability

Alternate Form: The Decision-Making Collaboration Scale can be conducted as a self-report or as an other-report.


Internal Consistency: Alpha coeffi cient of .81 was reported.




Practicality and Usefulness: The Decision-Making Collaboration Scale can measure people’s decision-making tendencies interpersonally and can be adapted for measuring small group situations as an individual trait that may infl uence tasks, relationships, and group processes. It may also be useful in explaining factors that infl uence com-munication outcomes such as communication satisfaction.

Clarity of Instructions: The instructions for completing the Decision-Making Collaboration Scale are not pro-vided. The article also does not include instructions for tabulating scores.

Standardization in Administration and Evaluation: The 13 items are listed in the article; however standardized administration and evaluation procedures are not included. The original scale containing 42 items was administered through a questionnaire booklet outside of class in scheduled research sessions. For the second study, there is no description of how this instrument was administered. There is no description of how this instrument was used for evaluation of students.


Primary Strengths: The scale can quickly measure a person’s willingness to collaborate with another person in decision-making contexts.

Primary Weaknesses: This paper and pencil test relies on self-report from respondents. It is possible that respon-dents will show bias in marking socially desirable answers.

Overall Adequacy: When used with other measures of interpersonal behaviors, it may be a viable indicator of communication traits. However, assessing communication skills in decision-making situations should be assessed through observation of actual performance.

Additional ReferencesAnderson, C. M., & Infante, D. A. (1992, October). Test of a communication model of patients and physicians as collaborators

in patients’ health care. Paper presented at the meeting of the Speech Communication Association, Chicago, IL.Anderson, C. M., Martin, M. M., & Infante, D. A. (1993, April). Willingness to collaborate as a new communication trait: Scale

development and a predictive model of related communication traits. Paper presented at the joint meeting of the Central and Southern States Communication Associations, Lexington, KY.





Practicality and Usefulness: The Decision-Making Collaboration Scale can measure people’s decision-making tendencies interpersonally and can be adapted for measuring small group situations as an individual trait that may infl uence tasks, relationships, and group processes. It may also be useful in explaining factors that infl uence com-munication outcomes such as communication satisfaction.

Clarity of Instructions: The instructions for completing the Decision-Making Collaboration Scale are not pro-vided. The article also does not include instructions for tabulating scores.

Standardization in Administration and Evaluation: The 13 items are listed in the article; however standardized administration and evaluation procedures are not included. The original scale containing 42 items was administered through a questionnaire booklet outside of class in scheduled research sessions. For the second study, there is no description of how this instrument was administered. There is no description of how this instrument was used for evaluation of students.


Primary Strengths: The scale can quickly measure a person’s willingness to collaborate with another person in decision-making contexts.

Primary Weaknesses: This paper and pencil test relies on self-report from respondents. It is possible that respon-dents will show bias in marking socially desirable answers.

Overall Adequacy: When used with other measures of interpersonal behaviors, it may be a viable indicator of communication traits. However, assessing communication skills in decision-making situations should be assessed through observation of actual performance.

Additional ReferencesAnderson, C. M., & Infante, D. A. (1992, October). Test of a communication model of patients and physicians as collaborators

in patients’ health care. Paper presented at the meeting of the Speech Communication Association, Chicago, IL.Anderson, C. M., Martin, M. M., & Infante, D. A. (1993, April). Willingness to collaborate as a new communication trait: Scale

development and a predictive model of related communication traits. Paper presented at the joint meeting of the Central and Southern States Communication Associations, Lexington, KY.




Interaction Involvement Scale

Author / Developers: D.J. Cegala

Address: Donald J. Cegala, Ohio State University, Communication Department, Columbus, OH 43210

Source: Cegala, D.J. (1981). Interaction involvement: A cognitive dimension of communicative competence. Com-munication Education, 30, 108-121.

Cegala, D.J., Savage, G.T., Brunner, C.G. & Conrad, A.B. (1982) An elaboration of the meaning of interaction in-volvement. Communication Monographs, 49, 229-49.

Context or communication behavior for which intended: Willingness to become involved in social interactions



Description of test: This self-report asks respondents to consider 18 statements related to their thoughts and behav-iors during conversations with others. Subjects respond on a seven-point Likert-type scale as to how each statement pertains to them.

Description of scoring: Ratings can be totaled for an overall score. Scores in the upper 20 percent are considered high in interaction involvement; those in the bottom 20% are considered low. The scale also yields three subscores that indicate the individual’s responsiveness, perceptiveness, and attentiveness.

Norm referencing/criterion referencing populations: Various groups of undergraduate students have been used in studies of this measure.

Tests for cultural/gender bias: Considerable attention has been given to differences between males and females. Specifi c fi ndings in relation to each of the subscales are discussed in the 1982 article cited below.

Validity

Predictive: Behaviors during interaction can be predicted on the basis of SSI scores particularly with regard to the perceptiveness factor.

Concurrent: The SSI was correlated with measures of personality types, communication apprehension and com-munication competence. Appropriate relationships among these measures were reported.

Face/Content: The theoretical background and scale development are discussed in detail.

Construct: Nonverbal correlations of interaction involvement have been reported.

Reliability

Test/retest: A test-retest reliability of .81 was reported.

Internal consistency: Alpha coeffi cients for each of the three subscales ranged from .75-.83.


Practicality and usefulness: This measure might be used on a pre-post basis in an interpersonal course to measure the effectiveness of instruction.

Clarity of instructions: Directions for completing the measure are very clear.


Identifi cation of a range of responses or degrees of competence: Interpretative ranges are not offered. Distinc-tions were made based on percentages within the referencing population.



Interaction Involvement Scale

Author / Developers: D.J. Cegala

Address: Donald J. Cegala, Ohio State University, Communication Department, Columbus, OH 43210

Source: Cegala, D.J. (1981). Interaction involvement: A cognitive dimension of communicative competence. Com-munication Education, 30, 108-121.

Cegala, D.J., Savage, G.T., Brunner, C.G. & Conrad, A.B. (1982) An elaboration of the meaning of interaction in-volvement. Communication Monographs, 49, 229-49.

Context or communication behavior for which intended: Willingness to become involved in social interactions



Description of test: This self-report asks respondents to consider 18 statements related to their thoughts and behav-iors during conversations with others. Subjects respond on a seven-point Likert-type scale as to how each statement pertains to them.

Description of scoring: Ratings can be totaled for an overall score. Scores in the upper 20 percent are considered high in interaction involvement; those in the bottom 20% are considered low. The scale also yields three subscores that indicate the individual’s responsiveness, perceptiveness, and attentiveness.


Tests for cultural/gender bias: Considerable attention has been given to differences between males and females. Specifi c fi ndings in relation to each of the subscales are discussed in the 1982 article cited below.

Validity

Predictive: Behaviors during interaction can be predicted on the basis of SSI scores particularly with regard to the perceptiveness factor.

Concurrent: The SSI was correlated with measures of personality types, communication apprehension and com-munication competence. Appropriate relationships among these measures were reported.

Face/Content: The theoretical background and scale development are discussed in detail.

Construct: Nonverbal correlations of interaction involvement have been reported.

Reliability

Test/retest: A test-retest reliability of .81 was reported.

Internal consistency: Alpha coeffi cients for each of the three subscales ranged from .75-.83.


Practicality and usefulness: This measure might be used on a pre-post basis in an interpersonal course to measure the effectiveness of instruction.

Clarity of instructions: Directions for completing the measure are very clear.


Identifi cation of a range of responses or degrees of competence: Interpretative ranges are not offered. Distinc-tions were made based on percentages within the referencing population.



Primary strengths: This measure has undergone extensive validation. It provides a good overall assessment of interpersonal performance that encompasses a wide range of behaviors.

Primary weaknesses: Scoring and interpretation are a bit ambiguous.

Overall adequacy: This measure seems to provide a valid option for assessing a broad spectrum of interpersonal abilities.

Additional ReferencesAllen, T. (1991). Effects of metaknowledge on talk duration and interaction involvement in small group decision making. Com-

munication Research Reports. 8, 1-7.Campbell, S.W., & Neer, M. R. (2001). The relationship of communication apprehension and interaction involvement to per-

ceptions of computer-mediated communication. Communication Research Reports. 18, 391-398. Cegala, D.J., Savage, G.T., Brunner, C.C., & Conrad, A.B. (1982). An elaboration of the meaning of interaction involvement:

Toward the development of a theoretical concept. Communication Monographs, 49, 229-248.Cegala, D.J., & Sillars, A.L., (1989). Further examination of nonverbal manifestations of interaction involvement. Communica-

tion Reports, 2, 39-47.Chen, G.M. (1992). Communication adaptability and interaction involvement as predictors of cross-cultural adjustment. Com-

munication Research Reports. 9, 33-41.




Primary strengths: This measure has undergone extensive validation. It provides a good overall assessment of interpersonal performance that encompasses a wide range of behaviors.

Primary weaknesses: Scoring and interpretation are a bit ambiguous.

Overall adequacy: This measure seems to provide a valid option for assessing a broad spectrum of interpersonal abilities.

Additional ReferencesAllen, T. (1991). Effects of metaknowledge on talk duration and interaction involvement in small group decision making. Com-

munication Research Reports. 8, 1-7.Campbell, S.W., & Neer, M. R. (2001). The relationship of communication apprehension and interaction involvement to per-

ceptions of computer-mediated communication. Communication Research Reports. 18, 391-398. Cegala, D.J., Savage, G.T., Brunner, C.C., & Conrad, A.B. (1982). An elaboration of the meaning of interaction involvement:

Toward the development of a theoretical concept. Communication Monographs, 49, 229-248.Cegala, D.J., & Sillars, A.L., (1989). Further examination of nonverbal manifestations of interaction involvement. Communica-

tion Reports, 2, 39-47.Chen, G.M. (1992). Communication adaptability and interaction involvement as predictors of cross-cultural adjustment. Com-

munication Research Reports. 9, 33-41.





Author / Developers: Geo-Ming Chen and William Starosta

Address: Geo-Ming Chen, Department of Communication Studies, University of Rhode Island, Kingston, RI 02881, [email protected]; William Starosta, Department of Communication, Howard University, Washington, D.C., 20059, [email protected]

Source: Chen, G.M. & Starosta, W.J. (2000) The development and validation of the intercultural communication sensitivity scale. Human Communication, 3, 1-15.

Intended context or behavior: Affective dimension of intercultural communication competency

Intended academic level: College/Adult


Description: On a fi ve-point Likert scale (5=strongly agree; 1=strongly disagree), subjects respond to 24 statements on their feelings about interacting with individuals of a different culture.

Description of scoring: Nine items are reversed before totaling the scale. Subscales on interaction engagement, respect for cultural differences and interaction confi dence are identifi ed.


Validity

Predictive: Subjects scoring high on the Intercultural Sensitivity Scale were also more effective in intercultural interactions and held more positive attitudes toward those interactions.

Concurrent: Signifi cant correlations were reported with the fi ve different scales measuring self-esteem, self-moni-toring, perspective taking, attentiveness and impression rewarding.

Face/Content: The initial 73 items were drawn from a review of literature on intercultural communication compe-tency.

Construct: A factor analysis indicated that intercultural sensitivity is composed of engagement, respect, confi dence and enjoyment.

Reliability

Internal consistency: Cronbach Alpha coeffi cients of .86 and .88 were reported.


Practicality and usefulness: This measure would be useful for assessing the impact of intercultural training and experience. It could also be used to evaluate the impact of international study.

Clarity of instructions: Directions are clear.

Standardization in administration and evaluation: Responses are standardized.

Range of responses or degrees of competence: Ranges have not been identifi ed.

Primary strengths: This is a standardized, effi cient means of identifying whether individuals are open to intercul-tural interactions.

Primary weaknesses: The measure only considers the affective dimension of intercultural competency. Other mea-sures would be needed to assess the behavioral and cognitive dimensions.

Overall adequacy: This measure would help students focus on their responses to intercultural encounters and de-velop an awareness of their emotional reactions.




Author / Developers: Geo-Ming Chen and William Starosta

Address: Geo-Ming Chen, Department of Communication Studies, University of Rhode Island, Kingston, RI 02881, [email protected]; William Starosta, Department of Communication, Howard University, Washington, D.C., 20059, [email protected]

Source: Chen, G.M. & Starosta, W.J. (2000) The development and validation of the intercultural communication sensitivity scale. Human Communication, 3, 1-15.

Intended context or behavior: Affective dimension of intercultural communication competency

Intended academic level: College/Adult


Description: On a fi ve-point Likert scale (5=strongly agree; 1=strongly disagree), subjects respond to 24 statements on their feelings about interacting with individuals of a different culture.

Description of scoring: Nine items are reversed before totaling the scale. Subscales on interaction engagement, respect for cultural differences and interaction confi dence are identifi ed.


Validity

Predictive: Subjects scoring high on the Intercultural Sensitivity Scale were also more effective in intercultural interactions and held more positive attitudes toward those interactions.

Concurrent: Signifi cant correlations were reported with the fi ve different scales measuring self-esteem, self-moni-toring, perspective taking, attentiveness and impression rewarding.

Face/Content: The initial 73 items were drawn from a review of literature on intercultural communication compe-tency.

Construct: A factor analysis indicated that intercultural sensitivity is composed of engagement, respect, confi dence and enjoyment.

Reliability

Internal consistency: Cronbach Alpha coeffi cients of .86 and .88 were reported.


Practicality and usefulness: This measure would be useful for assessing the impact of intercultural training and experience. It could also be used to evaluate the impact of international study.

Clarity of instructions: Directions are clear.

Standardization in administration and evaluation: Responses are standardized.

Range of responses or degrees of competence: Ranges have not been identifi ed.

Primary strengths: This is a standardized, effi cient means of identifying whether individuals are open to intercul-tural interactions.

Primary weaknesses: The measure only considers the affective dimension of intercultural competency. Other mea-sures would be needed to assess the behavioral and cognitive dimensions.

Overall adequacy: This measure would help students focus on their responses to intercultural encounters and de-velop an awareness of their emotional reactions.



Additional ReferencesFritz, W., Mollenberg, A. & Chen, G.M. (2002) Measuring intercultural sensitivity in different cultural contexts. Intercultural

Communication Studies, 11, 165-176. Peng, S.Y. Rangsipaht, S. & Thaipakdee, S.(2005) Measuring intercultural sensitivity: A comparative study of ethnic Chinese

and Thai nationals. Journal of Intercultural Communication Research. 23:2, 119-137.




Additional ReferencesFritz, W., Mollenberg, A. & Chen, G.M. (2002) Measuring intercultural sensitivity in different cultural contexts. Intercultural

Communication Studies, 11, 165-176. Peng, S.Y. Rangsipaht, S. & Thaipakdee, S.(2005) Measuring intercultural sensitivity: A comparative study of ethnic Chinese

and Thai nationals. Journal of Intercultural Communication Research. 23:2, 119-137.




Intercultural Willingness to Communicate Scale

Author / Developers: Jeffrey Kassing

Address: Department of Speech Communication, St. Cloud State University, St. Cloud, MN 56301

Source: Kassing, J.W. (1997). Development of the intercultural willingness to communicate scale. Communication Research Reports, 14, 399-407.

Intended context or behavior: Predisposition to interact with individuals from different cultures

Intended academic level: High school, college, adult


Description: The 12-item scale includes six statements describing various intercultural interactions that are inter-spersed with six fi ller statements from the original Willingness to Communicate Scale (McCroskey, 1992). On a scale of zero to 100 percent, respondents indicate the likelihood of interacting with individuals in these situations.

Description of Scoring: The six items are summed. A mean score of 29.67 was reported. In one study a median split was used to categorize subjects as high or low in intercultural willingness to communicate.

Norm/criterion reference population: Undergraduates

Culture and gender bias: The sample was primarily Caucasian. The impact of gender of was not reported.

Validity

Content/Face: The IWTC scale was based on the WTC scale. The six statements were drawn from current concep-tualizations of intercultural communication.

Predictive: Individuals high in IWTC had more friends from other countries.

Construct: The appropriate relationship between WTC and IWTC was established. IWTC seems to be a separate construct from WTC.

Reliability



Practicality and usefulness: This is a very simple and effi cient yet powerful measure of the affective dimension of intercultural communication.

Clarity of instructions: The directions can be easily followed.

Standardization in administration and evaluation: The scale can be administrated in a variety of settings.

Range of responses or degrees of competence: The 100-point scale is adequate. Break points of high and low IWTC would be helpful.

Strengths: Previous research has demonstrated that individuals from different cultures vary in their willingness to communicate. This measure provides a means of comparing intracultural WTC with intercultural WTC.

Weaknesses: The instrument needs more testing with a more diverse sample. Such research has been initiated.

Overall adequacy: In addition to evaluating the impact of instruction in intercultural communication, the IWTC scale would be helpful in monitoring the impact of international study experiences.




Intercultural Willingness to Communicate Scale

Author / Developers: Jeffrey Kassing

Address: Department of Speech Communication, St. Cloud State University, St. Cloud, MN 56301

Source: Kassing, J.W. (1997). Development of the intercultural willingness to communicate scale. Communication Research Reports, 14, 399-407.

Intended context or behavior: Predisposition to interact with individuals from different cultures

Intended academic level: High school, college, adult


Description: The 12-item scale includes six statements describing various intercultural interactions that are inter-spersed with six fi ller statements from the original Willingness to Communicate Scale (McCroskey, 1992). On a scale of zero to 100 percent, respondents indicate the likelihood of interacting with individuals in these situations.

Description of Scoring: The six items are summed. A mean score of 29.67 was reported. In one study a median split was used to categorize subjects as high or low in intercultural willingness to communicate.

Norm/criterion reference population: Undergraduates

Culture and gender bias: The sample was primarily Caucasian. The impact of gender of was not reported.

Validity

Content/Face: The IWTC scale was based on the WTC scale. The six statements were drawn from current concep-tualizations of intercultural communication.

Predictive: Individuals high in IWTC had more friends from other countries.

Construct: The appropriate relationship between WTC and IWTC was established. IWTC seems to be a separate construct from WTC.

Reliability



Practicality and usefulness: This is a very simple and effi cient yet powerful measure of the affective dimension of intercultural communication.

Clarity of instructions: The directions can be easily followed.

Standardization in administration and evaluation: The scale can be administrated in a variety of settings.

Range of responses or degrees of competence: The 100-point scale is adequate. Break points of high and low IWTC would be helpful.

Strengths: Previous research has demonstrated that individuals from different cultures vary in their willingness to communicate. This measure provides a means of comparing intracultural WTC with intercultural WTC.

Weaknesses: The instrument needs more testing with a more diverse sample. Such research has been initiated.

Overall adequacy: In addition to evaluating the impact of instruction in intercultural communication, the IWTC scale would be helpful in monitoring the impact of international study experiences.




Knutson, T.& Posirisuk, S. (2005) A Study of Thai Relational Development and Rhetorical Sensitivity: Jai Yen. A paper pre-sented at the annual meeting of the International Communication Association, New York, NY.

Yang L., Rancer, A.S.& Sunhee Lim, A.S.(2003). Ethnocentrism and intercultural willingness to communicate: A cross-cultural comparison between Korean and American college students. Journal of Intercultural Communication Research, 32, 117-129.

Yang L. & Rancer, A.S.(2003). Ethnocentrism, intercultural communication apprehension, intercultural willingness-to-com-municate, and intentions to participate in an intercultural dialogue program: Testing a proposed model. Communication Research Reports, 20, 189-190.




Knutson, T.& Posirisuk, S. (2005) A Study of Thai Relational Development and Rhetorical Sensitivity: Jai Yen. A paper pre-sented at the annual meeting of the International Communication Association, New York, NY.

Yang L., Rancer, A.S.& Sunhee Lim, A.S.(2003). Ethnocentrism and intercultural willingness to communicate: A cross-cultural comparison between Korean and American college students. Journal of Intercultural Communication Research, 32, 117-129.

Yang L. & Rancer, A.S.(2003). Ethnocentrism, intercultural communication apprehension, intercultural willingness-to-com-municate, and intentions to participate in an intercultural dialogue program: Testing a proposed model. Communication Research Reports, 20, 189-190.




Interpersonal Communication Competency Scale

Author / Developers: R.B. Rubin and M.M. Martin

Address: R. B. Rubin, c/o School of Communication Studies, Kent State University, Kent OH 44242; M.M. Martin, Communication Studies, P.O. Box 6293. West Virginia University, Morgantown, WV 26506-6293

Source: Rubin, R.B. & Martin, M.M. (1994). Development of a measure of interpersonal communication compe-tence. Communication Research Reports, 11, 33-44.

Intended context or behavior: Interpersonal skills



Description: On a fi ve-point Likert-type scale, students respond to 30 statements that are designed to measure 10 important interpersonal skills. These include self-disclosure, empathy, social relaxation, assertiveness, interac-tion management, altercentrism, expressiveness, supportiveness, immediacy and environmental control. A ten-item short-form of the measure is also available.

Description of Scoring: Several items are reversed for scoring. The scale is intended for use in instructional settings to enable students to see the results of interpersonal instruction. Scores should be compared on a pre-post basis.


Validity

Concurrent: The ten short-form items were moderately correlated with separate measures of the 10 interpersonal skills. The short form also showed signifi cant positive correlations to measures of cognitive and communication fl exibility.

Content: Items for the scale were drawn from existing scales or from defi nitions of the various constructs.

Construct: Extensive review of the literature resulted in the focus upon ten important interpersonal skills. Factor analysis was used in selecting the pertinent items.

Reliability

Alternate form: The 30-item and the 10-item versions correlated at .86.

Internal consistency: The alphas for the 10 three-item subscales ranged from .41 to .72. The overall alpha was .86.


Practicality and Usefulness: The measure is based on 10 interpersonal skills which frequently receive attention in interpersonal instruction. It is a very clear way to illustrate these skills.

Clarity of instructions: The scale is very clear and easy to use.

Degree of standardization: It should be relatively simple for everyone to follow.

Identifi cation of a range of responses or degrees of competence: Interpretative ranges are not offered.

Primary strengths: This is an excellent resource for interpersonal instruction. It will effectively demonstrate to students how their abilities have changed as a result of instruction.

Primary weaknesses: It would be interesting to see if this scale could be adapted to an other-report format so it could be used as an indicator of student achievement.

Overall adequacy: The scale succeeds at accomplishing its intended objective.



Interpersonal Communication Competency Scale

Author / Developers: R.B. Rubin and M.M. Martin

Address: R. B. Rubin, c/o School of Communication Studies, Kent State University, Kent OH 44242; M.M. Martin, Communication Studies, P.O. Box 6293. West Virginia University, Morgantown, WV 26506-6293

Source: Rubin, R.B. & Martin, M.M. (1994). Development of a measure of interpersonal communication compe-tence. Communication Research Reports, 11, 33-44.

Intended context or behavior: Interpersonal skills



Description: On a fi ve-point Likert-type scale, students respond to 30 statements that are designed to measure 10 important interpersonal skills. These include self-disclosure, empathy, social relaxation, assertiveness, interac-tion management, altercentrism, expressiveness, supportiveness, immediacy and environmental control. A ten-item short-form of the measure is also available.

Description of Scoring: Several items are reversed for scoring. The scale is intended for use in instructional settings to enable students to see the results of interpersonal instruction. Scores should be compared on a pre-post basis.


Validity

Concurrent: The ten short-form items were moderately correlated with separate measures of the 10 interpersonal skills. The short form also showed signifi cant positive correlations to measures of cognitive and communication fl exibility.

Content: Items for the scale were drawn from existing scales or from defi nitions of the various constructs.

Construct: Extensive review of the literature resulted in the focus upon ten important interpersonal skills. Factor analysis was used in selecting the pertinent items.

Reliability

Alternate form: The 30-item and the 10-item versions correlated at .86.

Internal consistency: The alphas for the 10 three-item subscales ranged from .41 to .72. The overall alpha was .86.


Practicality and Usefulness: The measure is based on 10 interpersonal skills which frequently receive attention in interpersonal instruction. It is a very clear way to illustrate these skills.

Clarity of instructions: The scale is very clear and easy to use.

Degree of standardization: It should be relatively simple for everyone to follow.

Identifi cation of a range of responses or degrees of competence: Interpretative ranges are not offered.

Primary strengths: This is an excellent resource for interpersonal instruction. It will effectively demonstrate to students how their abilities have changed as a result of instruction.

Primary weaknesses: It would be interesting to see if this scale could be adapted to an other-report format so it could be used as an indicator of student achievement.

Overall adequacy: The scale succeeds at accomplishing its intended objective.



Additional ReferencesAnderson, C. M.& Martin, M.M. (1998). The effects of interpersonal communication competence and sex on communication

satisfaction in china.. World Communication, 27:2, 15-28.Martin, M. M.& Anderson, C. M. (1995). Roommate Similarity: Are Roommates Who Are Similar in Their Communication

Traits More Satisfi ed? Communication Research Reports, 12:1, 46-52.Rubin, R.B., Martin, M.M., Bruning, S.S. & Powers, D.E.(1993). Test of a self-effi cacy model of interpersonal communication

competence. Communication Quarterly, 41, 210-20.Rubin, R.B. (1990). Communication competence. In G.M. Phillips & J.T. Wood (Eds.), Speech Communication: Essays To

Commemorate The 75th Anniversary Of The Speech Communication Association (pp. 94-129). Carbondale: Southern Illinois Press.




Additional ReferencesAnderson, C. M.& Martin, M.M. (1998). The effects of interpersonal communication competence and sex on communication

satisfaction in china.. World Communication, 27:2, 15-28.Martin, M. M.& Anderson, C. M. (1995). Roommate Similarity: Are Roommates Who Are Similar in Their Communication

Traits More Satisfi ed? Communication Research Reports, 12:1, 46-52.Rubin, R.B., Martin, M.M., Bruning, S.S. & Powers, D.E.(1993). Test of a self-effi cacy model of interpersonal communication

competence. Communication Quarterly, 41, 210-20.Rubin, R.B. (1990). Communication competence. In G.M. Phillips & J.T. Wood (Eds.), Speech Communication: Essays To

Commemorate The 75th Anniversary Of The Speech Communication Association (pp. 94-129). Carbondale: Southern Illinois Press.




Interpersonal Communication Satisfaction Inventory

Author / Developers: M.L. Hecht

Address: Michael L. Hecht, Department of Communication, Arizona State University, Tempe AZ 85187

Source: Hecht, M.L. (1978). The conceptualization and measurement of interpersonal communication satisfaction. Human Communication Review, 4, 253-64.

Context or communication behavior for which intended: Satisfaction with interpersonal interactions



Description: After either participating in an actual 15-minute conversation or recalling a recent conversation, re-spondents consider 19 statements pertaining to the interaction. Using a seven-point Likert-type scale, the respon-dents rate each statement as to whether they agree or disagree with it.

Description of scoring: A scoring key is included in the study. Rating for eight items are reversed and score is totaled for the 19 items. Mean scores and ranges of responses were not reported in this study.

Norm referencing/criterion referencing populations: Various groups of fi rst year students enrolled in an intro-ductory communication course at the University of Illinois participated in the different stages of this study.

Validity

Concurrent: High to moderate correlations were reported between this measure and another using nonverbal faces as means of measuring communication satisfaction.

Face/Content: Conceptualization and construction of the instrument were reported in the study. Resulting factor analyses were also reported.

Reliability

Alternate forms: Reliability coeffi cients for a 16-item version were reported. The 19 item version was judged as superior.

Internal consistency: Coeffi cients ranging from .90 to.97 were reported for the measure when studied in various contexts.


Practicality and usefulness: The measure might be appropriate for a pre/post indicator of changes in interpersonal effectiveness.


Degree of standardization in administration and evaluation of responses: Actual conversation topics were not provided. The topic may affect the behaviors measured.

Identifi cation of a range of responses or degrees of competence: Interpretative ranges were not provided.

Primary strengths: The process of identifying and verifying items for this measure was very thorough. It has been used and modifi ed to fi t the consideration of satisfaction in many different communication contexts.

Overall adequacy: The measure may be better suited to research settings than to instructional ones.

Additional ReferencesGlenn, T.H., Rhea, J., & Wheeless, L. R. (1997). Interpersonal communication satisfaction and biological sex: Nurse-physician

relationships. Communication Research Reports. 14:1, 24-32.



Interpersonal Communication Satisfaction Inventory

Author / Developers: M.L. Hecht

Address: Michael L. Hecht, Department of Communication, Arizona State University, Tempe AZ 85187

Source: Hecht, M.L. (1978). The conceptualization and measurement of interpersonal communication satisfaction. Human Communication Review, 4, 253-64.

Context or communication behavior for which intended: Satisfaction with interpersonal interactions



Description: After either participating in an actual 15-minute conversation or recalling a recent conversation, re-spondents consider 19 statements pertaining to the interaction. Using a seven-point Likert-type scale, the respon-dents rate each statement as to whether they agree or disagree with it.

Description of scoring: A scoring key is included in the study. Rating for eight items are reversed and score is totaled for the 19 items. Mean scores and ranges of responses were not reported in this study.

Norm referencing/criterion referencing populations: Various groups of fi rst year students enrolled in an intro-ductory communication course at the University of Illinois participated in the different stages of this study.

Validity

Concurrent: High to moderate correlations were reported between this measure and another using nonverbal faces as means of measuring communication satisfaction.

Face/Content: Conceptualization and construction of the instrument were reported in the study. Resulting factor analyses were also reported.

Reliability

Alternate forms: Reliability coeffi cients for a 16-item version were reported. The 19 item version was judged as superior.

Internal consistency: Coeffi cients ranging from .90 to.97 were reported for the measure when studied in various contexts.


Practicality and usefulness: The measure might be appropriate for a pre/post indicator of changes in interpersonal effectiveness.


Degree of standardization in administration and evaluation of responses: Actual conversation topics were not provided. The topic may affect the behaviors measured.

Identifi cation of a range of responses or degrees of competence: Interpretative ranges were not provided.

Primary strengths: The process of identifying and verifying items for this measure was very thorough. It has been used and modifi ed to fi t the consideration of satisfaction in many different communication contexts.

Overall adequacy: The measure may be better suited to research settings than to instructional ones.

Additional ReferencesGlenn, T.H., Rhea, J., & Wheeless, L. R. (1997). Interpersonal communication satisfaction and biological sex: Nurse-physician

relationships. Communication Research Reports. 14:1, 24-32.



Grant, J.A.& King, P.E. (1994). Compliance-gaining strategies, communication satisfaction, and willingness to comply. Com-munication Reports, 7:2, 99-108.

Hecht, M.L. (1989). An African American perspective on interethnic communication. Communication Monographs, 56, 385-410.

Hecht, M.L. (1984). An investigation of the effects of sex on self and other on perceptions of communication satisfaction. Sex Roles, 10, 733-741.

Hecht, M. L., & Sereno, K. K. (1985) Interpersonal communication satisfaction: Relationship to satisfaction with self and an-other. Communication Research Reports. 2: 1, 141-148.

Martin, J.N., Hecht, M.L. & Larkey, L.K. (1994). Conversational improvement strategies for interethnic communication: Afri-can American and European American perspective. Communication Monographs, 61:3, 236-255.

Neuliep, J. W. & Grohskopf, E. L (2000). Uncertainty reduction and communication satisfaction during initial interaction: an initial test and replication of a new axiom.Communication Reports, 13:2, 67-78.




Grant, J.A.& King, P.E. (1994). Compliance-gaining strategies, communication satisfaction, and willingness to comply. Com-munication Reports, 7:2, 99-108.

Hecht, M.L. (1989). An African American perspective on interethnic communication. Communication Monographs, 56, 385-410.

Hecht, M.L. (1984). An investigation of the effects of sex on self and other on perceptions of communication satisfaction. Sex Roles, 10, 733-741.

Hecht, M. L., & Sereno, K. K. (1985) Interpersonal communication satisfaction: Relationship to satisfaction with self and an-other. Communication Research Reports. 2: 1, 141-148.

Martin, J.N., Hecht, M.L. & Larkey, L.K. (1994). Conversational improvement strategies for interethnic communication: Afri-can American and European American perspective. Communication Monographs, 61:3, 236-255.

Neuliep, J. W. & Grohskopf, E. L (2000). Uncertainty reduction and communication satisfaction during initial interaction: an initial test and replication of a new axiom.Communication Reports, 13:2, 67-78.




Kentucky Listening Comprehension Test

Author / Developers: R. N. Bostrom and E.S. Waldhart

Address: [email protected]

Context or communication behavior for which intended: Listening abilities


Time required: Approximately 60 to 90 minutes

Description : This instrument includes an audiotape, and a four-part question/response sheet. The 42 items in Part I ask respondents to immediately identify a particular number of letter from a series heard on the audiotape. Similarly the 12 items in Part II ask respondents to identify a letter or number from a series but with a pause included. Part III includes four short interactions and asks respondents to complete 10 multiple choice items about these interactions. In Part IV respondents listen to a fairly extensive lecture and then respond to 14 multiple choice questions.

Description of scoring: Correct items are totaled to yield a global score. In addition, fi ve subscales are available that indicate the specifi c listening strengths and weaknesses of the respondents. The subscales measure short term listening, short term listening with rehearsal, interpretative listening, informative listening, and the ability to over-come distraction when listening. The materials provide means and percentile rankings for the global score and the subscales.

Norm referencing/criterion referencing populations: Forms of the scale have been administered to over 20,000 college students and adults. Subgroups of this population were used for various statistical analyses.

Validity

Concurrent: The authors reported moderate correlations between the short term with rehearsal and the lecture section scores and student performance on a speech course examination. Other researchers have reported that this measure is correlated with the Watson-Baker Listening Test and the listening items on the Communication Compe-tency Assessment Instrument.

Face/Content: Communication department faculty from the University of Kentucky helped to validate the interac-tion section of the test. Although there was not unanimous agreement, these faculty determined the appropriateness of the response options.

Construct: Comparisons of the KCLT with other tests of cognitive ability suggest that it is measuring a distinct ability different from others that are closely related.

Reliability

Alternate forms: The reliability coeffi cient between Form A and Form B was .72. The interpretative subscale (Part III), however, for these alternative forms yielded a coeffi cient of .36.

Test/retest: Test-retest coeffi cients on an earlier version of this scale ranged from .78 to .87 for the various sub-scales.

Internal consistency: The Cronbach alpha for the global scores on Forms A and B were .78 to .82. The subscales ranged from .46 to .76.


Practicality and usefulness: This measure is a very straightforward and economical way to assess the listening profi ciency of a large number of people.

Clarity of instructions: The written form of the audiotape is clear and easy to follow. The directions for answering questions were clear.



Kentucky Listening Comprehension Test

Author / Developers: R. N. Bostrom and E.S. Waldhart

Address: [email protected]

Context or communication behavior for which intended: Listening abilities


Time required: Approximately 60 to 90 minutes

Description : This instrument includes an audiotape, and a four-part question/response sheet. The 42 items in Part I ask respondents to immediately identify a particular number of letter from a series heard on the audiotape. Similarly the 12 items in Part II ask respondents to identify a letter or number from a series but with a pause included. Part III includes four short interactions and asks respondents to complete 10 multiple choice items about these interactions. In Part IV respondents listen to a fairly extensive lecture and then respond to 14 multiple choice questions.

Description of scoring: Correct items are totaled to yield a global score. In addition, fi ve subscales are available that indicate the specifi c listening strengths and weaknesses of the respondents. The subscales measure short term listening, short term listening with rehearsal, interpretative listening, informative listening, and the ability to over-come distraction when listening. The materials provide means and percentile rankings for the global score and the subscales.

Norm referencing/criterion referencing populations: Forms of the scale have been administered to over 20,000 college students and adults. Subgroups of this population were used for various statistical analyses.

Validity

Concurrent: The authors reported moderate correlations between the short term with rehearsal and the lecture section scores and student performance on a speech course examination. Other researchers have reported that this measure is correlated with the Watson-Baker Listening Test and the listening items on the Communication Compe-tency Assessment Instrument.

Face/Content: Communication department faculty from the University of Kentucky helped to validate the interac-tion section of the test. Although there was not unanimous agreement, these faculty determined the appropriateness of the response options.

Construct: Comparisons of the KCLT with other tests of cognitive ability suggest that it is measuring a distinct ability different from others that are closely related.

Reliability

Alternate forms: The reliability coeffi cient between Form A and Form B was .72. The interpretative subscale (Part III), however, for these alternative forms yielded a coeffi cient of .36.

Test/retest: Test-retest coeffi cients on an earlier version of this scale ranged from .78 to .87 for the various sub-scales.

Internal consistency: The Cronbach alpha for the global scores on Forms A and B were .78 to .82. The subscales ranged from .46 to .76.


Practicality and usefulness: This measure is a very straightforward and economical way to assess the listening profi ciency of a large number of people.

Clarity of instructions: The written form of the audiotape is clear and easy to follow. The directions for answering questions were clear.



Degree of standardization in administration and evaluation of responses: The test administrator hands out the test booklets and turns off the tape. It was not clear to whether or not the questions and response choices were read aloud or just responded to in written form. Other researchers have reported a problem with respondents reading ahead.

Identifi cation of a range of responses or degrees of competence: There is only one “right” answer to the question. This seemed especially problematic in one section where more than one response seemed appropriate.

Primary strengths: The stimulus requires that subjects base their answers on listening to the audio tape. This al-lows for control over the testing situation which a live presentation may not. The lecture and conversation situations are ones that are familiar to most students.

Primary weaknesses: The letter and number sequence sections are not naturalistic stimuli. Test scores may be complicated by reading ability. The lecture seemed long and uninteresting.

Overall adequacy: This is a viable option for departments who must conduct large scale assessment of listening profi ciency.

Additional ReferencesBostrom, R.N. (2006) The process of listening in Handbook of Communication Skills, 3rd Edition, Owen Hargie (Ed.). Lon-

don: Routledge.Bostrom, R.N. & Waldhart, E.S. (1988). Memory models and the measurement of listening. Communication Education, 37,

1-13.Clark, A.J. (1989). Communication confi dence and listening competence: An investigation of the relationships of willingness

to communication, communication apprehension, and receiver apprehension to comprehension of content and emotional meaning in spoken messages. Communication Education, 38, 237-36.

Rubin, R.B. & Roberts, C.V. (1987). A comparative examination and analysis f three listening tests. Communication Education, 36, 142-153.

Villaume, W.A. & Brown, M.H. (1999) The development and validation of the vocalic sensitivity test. International Journal of Listening, 13, 24-45.

Villaume, W.A. & Weaver, J.B. (1996) A factorial approach to establishing reliable listening measures from the WBLT and KCLT. International Journal of Listening, 10, 1-20.




Degree of standardization in administration and evaluation of responses: The test administrator hands out the test booklets and turns off the tape. It was not clear to whether or not the questions and response choices were read aloud or just responded to in written form. Other researchers have reported a problem with respondents reading ahead.

Identifi cation of a range of responses or degrees of competence: There is only one “right” answer to the question. This seemed especially problematic in one section where more than one response seemed appropriate.

Primary strengths: The stimulus requires that subjects base their answers on listening to the audio tape. This al-lows for control over the testing situation which a live presentation may not. The lecture and conversation situations are ones that are familiar to most students.

Primary weaknesses: The letter and number sequence sections are not naturalistic stimuli. Test scores may be complicated by reading ability. The lecture seemed long and uninteresting.

Overall adequacy: This is a viable option for departments who must conduct large scale assessment of listening profi ciency.

Additional ReferencesBostrom, R.N. (2006) The process of listening in Handbook of Communication Skills, 3rd Edition, Owen Hargie (Ed.). Lon-

don: Routledge.Bostrom, R.N. & Waldhart, E.S. (1988). Memory models and the measurement of listening. Communication Education, 37,

1-13.Clark, A.J. (1989). Communication confi dence and listening competence: An investigation of the relationships of willingness

to communication, communication apprehension, and receiver apprehension to comprehension of content and emotional meaning in spoken messages. Communication Education, 38, 237-36.

Rubin, R.B. & Roberts, C.V. (1987). A comparative examination and analysis f three listening tests. Communication Education, 36, 142-153.

Villaume, W.A. & Brown, M.H. (1999) The development and validation of the vocalic sensitivity test. International Journal of Listening, 13, 24-45.

Villaume, W.A. & Weaver, J.B. (1996) A factorial approach to establishing reliable listening measures from the WBLT and KCLT. International Journal of Listening, 10, 1-20.






Address: HRDQ, 2002 Renaissance Boulevard #100, King of Prussia, PA 19406-2756. Phone: 610.279.2002 or 800.633.4533. Fax: 610.279.0524. http://www.hrdq.com/products/learntolisten.htm.








Validity




Reliability










Additional Resources: Not available.






Address: HRDQ, 2002 Renaissance Boulevard #100, King of Prussia, PA 19406-2756. Phone: 610.279.2002 or 800.633.4533. Fax: 610.279.0524. http://www.hrdq.com/products/learntolisten.htm.








Validity




Reliability










Additional Resources: Not available.





Author / Developers: C. Glenn Pearce, Iris W. Johnson and Randolph T. Barker

Address: Department of Management, Virginia Commonwealth University, [email protected].

Source: Pearce, C.G., Johnson, I.W. & Barker, R.T. (2003). Assessment of the listening styles inventory: Progress in establishing reliability and validity, Journal of Business and Technical Communication, 17:1, 84-113.

Intended context or behavior: Listening

Intended academic level: College/Adults


Description of test: On a fi ve-point Likert-scale, subjects respond to 10 statements self-assessing their listening behaviors. Based upon their responses, their listening style is classifi ed as active, involved, passive or detached.

Scoring description: Responses are summed, and ranges and description are provided for each of the four listening styles.

Norm referencing/criterion referencing populations: Adults in managerial positions

Cultural/gender bias: Males and females of varied ages and ethnicity were included in the sample.

Validity

Predictive: Listening instruction was found to improve listening style.

Concurrent: Agreement was found between managers’ assessments of employee listening and employee responses on the LSI. Similarly, agreement was found between professors’ assessments of student listening and student re-sponses on the LSI.

Face/Content: Based upon a review of the listening literature, the LSI items were also evaluated by seven listening professionals.

Construct: Factor analysis, interviews and observations of listening styles supported the construct validity of this measure.

Reliability

Test/retest: A correlation coeffi cient of .89 was reported.

Internal consistency: Cronbach alphas of .70 and .75 were reported.


Practicality and usefulness: This is a very effi cient instrument to make students aware of their listening behaviors. It can be used on a pre-post basis for assessing the impact of listening instruction.

Clarity of instructions: Directions and items are very clear.

Standardization in administration and evaluation: The instrument is highly standardized.

Range of responses or degrees of competence: Identifi cation of a range of responses or degrees of competence: The ranges for the four classifi cations of listening are provided and descriptions of each classifi cation are detailed.

Primary strengths: Rather than labeling listening as good or bad, this measure indicates that it is a stylistic differ-ence that can be addressed. It makes students aware of their listening choices.

Primary weaknesses: The instrument is designed more for the business setting than for instructional/assessment purposes.

Overall adequacy: The LSI can be used for self-assessment and for analysis of listening improvement.





Author / Developers: C. Glenn Pearce, Iris W. Johnson and Randolph T. Barker

Address: Department of Management, Virginia Commonwealth University, [email protected].

Source: Pearce, C.G., Johnson, I.W. & Barker, R.T. (2003). Assessment of the listening styles inventory: Progress in establishing reliability and validity, Journal of Business and Technical Communication, 17:1, 84-113.


Intended academic level: College/Adults


Description of test: On a fi ve-point Likert-scale, subjects respond to 10 statements self-assessing their listening behaviors. Based upon their responses, their listening style is classifi ed as active, involved, passive or detached.

Scoring description: Responses are summed, and ranges and description are provided for each of the four listening styles.

Norm referencing/criterion referencing populations: Adults in managerial positions

Cultural/gender bias: Males and females of varied ages and ethnicity were included in the sample.

Validity

Predictive: Listening instruction was found to improve listening style.

Concurrent: Agreement was found between managers’ assessments of employee listening and employee responses on the LSI. Similarly, agreement was found between professors’ assessments of student listening and student re-sponses on the LSI.

Face/Content: Based upon a review of the listening literature, the LSI items were also evaluated by seven listening professionals.

Construct: Factor analysis, interviews and observations of listening styles supported the construct validity of this measure.

Reliability

Test/retest: A correlation coeffi cient of .89 was reported.

Internal consistency: Cronbach alphas of .70 and .75 were reported.


Practicality and usefulness: This is a very effi cient instrument to make students aware of their listening behaviors. It can be used on a pre-post basis for assessing the impact of listening instruction.

Clarity of instructions: Directions and items are very clear.

Standardization in administration and evaluation: The instrument is highly standardized.

Range of responses or degrees of competence: Identifi cation of a range of responses or degrees of competence: The ranges for the four classifi cations of listening are provided and descriptions of each classifi cation are detailed.

Primary strengths: Rather than labeling listening as good or bad, this measure indicates that it is a stylistic differ-ence that can be addressed. It makes students aware of their listening choices.

Primary weaknesses: The instrument is designed more for the business setting than for instructional/assessment purposes.

Overall adequacy: The LSI can be used for self-assessment and for analysis of listening improvement.




Personal Report of Communication Apprehension (PRCA-24)

Author / Developers: James C. McCroskey

Address: Communication Studies, University of Alabama-Birmingham, 1055 Bldg., Room 122A, Birmingham, AL 35294. [email protected]

Source: McCroskey, J.C, Beatty, M.J., Kearney, P & Plax, T.G. (1985). The content validity of the PRCA-24 as a measure of communication apprehension across communication contexts. Communication Quarterly, 33, 165-173. In Assessing Motivation to Communicate. Washington, D.C.: National Communication Association, 1994.

Intended context or behavior: Communication apprehension

Intended Academic Level: College/Adult


Description of test: Likert-type items index communication apprehension in each of four different contexts: public speaking , speaking in small groups, speaking in meetings, and speaking in dyads. Six items, including three posi-tively worded and three negatively worded, address each context. Context scores can be used separately or summed for trait communication apprehension.

Description of administration and scoring: The instrument has instructions included at the beginning, so it can be administered in large settings if necessary. Scores for each context and a score for overall CA are developed by combining scores on the various items.

Norm referencing/criterion-referencing populations: Mean score is 65.60 with a standard deviation of 15.30 based on 25,000 students from 52 colleges and universities.

Culture/Gender Bias: The factor structure did not hold for a sample of Puerto Rican students, so caution should be used when using this questionnaire with intercultural students.

Validity

Content (face, intrinsic, representativeness): Studies show that 94-96 percent of the total PRCA-24 score is ac-counted for by any three of the four contexts which indicates that the scale is measuring generalized Trait-like com-munication apprehension.

Predictive: The PRCA-24 accounts for 50 percent of the variance in the RAS assertiveness scale.

Reliability

Internal consistency: Alpha reliability normally is .93 to .95.


Practicality and usefulness: The scale is easy to administer and analyze.


Degree of standardization in administration and evaluation of responses: Norms are stable and have been de-veloped on a large sample.

Identifi cation of a range of responses or degree of competence:

Primary strengths: A great deal of data has been collected to establish norms.

Overall adequacy: This is useful as an assessment instrument. Several colleges and universities use it as part of their assessment of achievement in oral communication instruction.



Personal Report of Communication Apprehension (PRCA-24)



Source: McCroskey, J.C, Beatty, M.J., Kearney, P & Plax, T.G. (1985). The content validity of the PRCA-24 as a measure of communication apprehension across communication contexts. Communication Quarterly, 33, 165-173. In Assessing Motivation to Communicate. Washington, D.C.: National Communication Association, 1994.

Intended context or behavior: Communication apprehension

Intended Academic Level: College/Adult


Description of test: Likert-type items index communication apprehension in each of four different contexts: public speaking , speaking in small groups, speaking in meetings, and speaking in dyads. Six items, including three posi-tively worded and three negatively worded, address each context. Context scores can be used separately or summed for trait communication apprehension.

Description of administration and scoring: The instrument has instructions included at the beginning, so it can be administered in large settings if necessary. Scores for each context and a score for overall CA are developed by combining scores on the various items.

Norm referencing/criterion-referencing populations: Mean score is 65.60 with a standard deviation of 15.30 based on 25,000 students from 52 colleges and universities.

Culture/Gender Bias: The factor structure did not hold for a sample of Puerto Rican students, so caution should be used when using this questionnaire with intercultural students.

Validity

Content (face, intrinsic, representativeness): Studies show that 94-96 percent of the total PRCA-24 score is ac-counted for by any three of the four contexts which indicates that the scale is measuring generalized Trait-like com-munication apprehension.

Predictive: The PRCA-24 accounts for 50 percent of the variance in the RAS assertiveness scale.

Reliability

Internal consistency: Alpha reliability normally is .93 to .95.


Practicality and usefulness: The scale is easy to administer and analyze.


Degree of standardization in administration and evaluation of responses: Norms are stable and have been de-veloped on a large sample.

Identifi cation of a range of responses or degree of competence:

Primary strengths: A great deal of data has been collected to establish norms.

Overall adequacy: This is useful as an assessment instrument. Several colleges and universities use it as part of their assessment of achievement in oral communication instruction.



Additional References Burk, J. (2001). Communication apprehension among Master’s of Business Administration students. Communication Educa-

tion, 50:1, 51-58Fordam, D.R., & Gabbin, A.L. (1996). Skills versus apprehension: Empirical evidence on oral communication. Communication

Quarterly, 59:3, 88-97.Hsu, C. (2004). Source of differences in communication apprehension between Chinese in Taiwan and Americans. Communi-

cation Quarterly, 52:4, 370-89.Levine, T.R. & McCroskey, J.C. (1990). Measuring trait communication apprehension: A test of rival measurement models of

the PRCA-24. Communication Monographs, 57, 62-71.Wilson, G.B. (1999). Language and culture: Tapping cognitive views. World Communication, 28:3, 49-67.




Additional References Burk, J. (2001). Communication apprehension among Master’s of Business Administration students. Communication Educa-

tion, 50:1, 51-58Fordam, D.R., & Gabbin, A.L. (1996). Skills versus apprehension: Empirical evidence on oral communication. Communication

Quarterly, 59:3, 88-97.Hsu, C. (2004). Source of differences in communication apprehension between Chinese in Taiwan and Americans. Communi-

cation Quarterly, 52:4, 370-89.Levine, T.R. & McCroskey, J.C. (1990). Measuring trait communication apprehension: A test of rival measurement models of

the PRCA-24. Communication Monographs, 57, 62-71.Wilson, G.B. (1999). Language and culture: Tapping cognitive views. World Communication, 28:3, 49-67.




Personal Report of Interethnic Communication Apprehension (PRECA)

Author / Developers: James W. Neuliep and James C. McCroskey

Address: James W. Neuliep, St. Norbert College, Department of Communication, 100 Grant Street, De Pere, WI 54115-2099. Phone: 920-403-3135. E-mail: [email protected]

Source: Neuliep, J. W. & McCroskey, J. C. (1997). The development of intercultural and interethnic communication apprehension scales. Communication Research Reports, 14(2), 145-156.

Intended Context or Behavior: Intercultural / Interethnic Communication



Description: Both instruments (PRICA and PRECA) are self-report measures including 14 items each and are modeled after the PRCA-24 (Personal Report of Communication Apprehension). PRICA has statements about the fear or anxiety associated with real or anticipated communication with people from different cultural groups. PRECA has statements about the fear or anxiety associated with real or anticipated communication with people from different ethnic and/or racial groups.

Scoring Description: Respondents rate on a fi ve-point Likert-type scale how true the statements are for them con-cerning their fear or anxiety associated with intercultural or interethnic communication. A score of fi ve indicates “Strongly Agree” and a score of one indicates “Strongly Disagree.” There are seven items that describe positive communication behaviors and feeling and seven items that describe negative behaviors or feelings. The total score of negative items is subtracted from the total score of positive items. Higher scores refl ect lower levels of communi-cation apprehension. Scores that are one standard deviation above and below the sample mean are used to determine categories of respondents with low and high communication apprehension.

Norm/Criterion Reference Populations: These instruments are based on a population of college students. Two out of four published studies reported the range of scores from the reference population. These ranges of scores provide a basis for comparison through norms. There is no information about specifi c criteria that respondents are expected to know or be able to do.

Culture and Gender Bias: Respondents have been predominantly Caucasian or Euro-American. Men have scored signifi cantly higher on both measures than women in initial tests. Later studies did not examine the effect of gender on test scores.

Validity

Concurrent: The PRICA and PRECA were both signifi cantly correlated with the PRCA-24. Therefore these instru-ments have the ability to distinguish between groups of people with various levels of communication apprehension and have concurrent validity.

Face/Content: By looking at the face validity of the PRICA and PRECA instruments, the items appropriately mea-sure the construct of intercultural communication apprehension. Intercultural communication apprehension relates to a person’s fear or anxiety with communicating with people from different cultural, ethnic, and/or racial groups. Measuring intercultural communication apprehension should focus on the circumstances in which a person would interact with different types of people. Both PRICA and PRECA have items that deal with those circumstances. The items were modeled after the PRCA-24 that is recognized as a valid instrument for measuring communication ap-prehension. The article provides a detailed description of how to defi ne the content of intercultural and interethnic communication apprehension.

Construct: PRICA and PRECA are highly correlated to each other since they measure similar communication be-haviors. Respondents react to different cultural groups much like they react to different ethnic and/or racial groups. PRICA and PRECA are not identical to the PRCA-24 since they focus on communication with different cultural, ethnic, and/or racial groups. Therefore both instruments have construct validity.



Personal Report of Interethnic Communication Apprehension (PRECA)

Author / Developers: James W. Neuliep and James C. McCroskey

Address: James W. Neuliep, St. Norbert College, Department of Communication, 100 Grant Street, De Pere, WI 54115-2099. Phone: 920-403-3135. E-mail: [email protected]

Source: Neuliep, J. W. & McCroskey, J. C. (1997). The development of intercultural and interethnic communication apprehension scales. Communication Research Reports, 14(2), 145-156.

Intended Context or Behavior: Intercultural / Interethnic Communication



Description: Both instruments (PRICA and PRECA) are self-report measures including 14 items each and are modeled after the PRCA-24 (Personal Report of Communication Apprehension). PRICA has statements about the fear or anxiety associated with real or anticipated communication with people from different cultural groups. PRECA has statements about the fear or anxiety associated with real or anticipated communication with people from different ethnic and/or racial groups.

Scoring Description: Respondents rate on a fi ve-point Likert-type scale how true the statements are for them con-cerning their fear or anxiety associated with intercultural or interethnic communication. A score of fi ve indicates “Strongly Agree” and a score of one indicates “Strongly Disagree.” There are seven items that describe positive communication behaviors and feeling and seven items that describe negative behaviors or feelings. The total score of negative items is subtracted from the total score of positive items. Higher scores refl ect lower levels of communi-cation apprehension. Scores that are one standard deviation above and below the sample mean are used to determine categories of respondents with low and high communication apprehension.

Norm/Criterion Reference Populations: These instruments are based on a population of college students. Two out of four published studies reported the range of scores from the reference population. These ranges of scores provide a basis for comparison through norms. There is no information about specifi c criteria that respondents are expected to know or be able to do.

Culture and Gender Bias: Respondents have been predominantly Caucasian or Euro-American. Men have scored signifi cantly higher on both measures than women in initial tests. Later studies did not examine the effect of gender on test scores.

Validity

Concurrent: The PRICA and PRECA were both signifi cantly correlated with the PRCA-24. Therefore these instru-ments have the ability to distinguish between groups of people with various levels of communication apprehension and have concurrent validity.

Face/Content: By looking at the face validity of the PRICA and PRECA instruments, the items appropriately mea-sure the construct of intercultural communication apprehension. Intercultural communication apprehension relates to a person’s fear or anxiety with communicating with people from different cultural, ethnic, and/or racial groups. Measuring intercultural communication apprehension should focus on the circumstances in which a person would interact with different types of people. Both PRICA and PRECA have items that deal with those circumstances. The items were modeled after the PRCA-24 that is recognized as a valid instrument for measuring communication ap-prehension. The article provides a detailed description of how to defi ne the content of intercultural and interethnic communication apprehension.

Construct: PRICA and PRECA are highly correlated to each other since they measure similar communication be-haviors. Respondents react to different cultural groups much like they react to different ethnic and/or racial groups. PRICA and PRECA are not identical to the PRCA-24 since they focus on communication with different cultural, ethnic, and/or racial groups. Therefore both instruments have construct validity.



Predictive: Both PRICA and PRECA are predictive of the frequency of contact with people from a different country; however PRECA is additionally predictive of the frequency of contact with people from a different race. In another study, PRECA is signifi cantly associated with communication satisfaction and anxiety of people who engaged in an interaction with someone of a different ethnicity. Therefore both instruments have predictive validity since there is evidence of these instruments correctly predicting something that they should theoretically be able to predict.

Reliability



Internal Consistency: Cronbach’s alpha for PRICA ranged from .90 to .941. Cronbach’s alpha for PRECA ranged from .915 to .94.


Practicality and Usefulness: PRICA and PRECA can be administered to teachers, teaching assistants, and students on college campuses to determine intercultural or interethnic communication apprehension in multicultural class-rooms. These measures could also be administered to managers and employees in multinational organizations to identify intercultural or interethnic communication apprehension in diverse work settings. Finally, these measures could also be administered for government use.

Clarity of Instructions: Clarity ranges among published studies. The instructions for administering and scoring the PRICA and PRECA are provided in three studies (Bippus & Dorjee, 2002; Hong, 2003; Toale & McCroskey, 2001). However, the developers of these measures did not provide instructions for administering and scoring the PRICA or PRECA (Neuliep & McCroskey, 1997).

Standardization in Administration and Evaluation: Neuliep and McCroskey (1997) published the questionnaire items for both the PRICA and PRECA. However, the descriptions of how to score these measures differ among pub-lished studies. Some researchers have summed all 14 items to determine the score for communication apprehension. Others have split the score into a positive-item sum and negative-item sum. The negative score is subtracted from the positive score to get the fi nal score. There is standardization in determining low and high levels of communication apprehension by using the standard deviation above and below the sample mean. There is no description of how these instruments were used for evaluation of students.

Range of Reponses or Degrees of Competence: In two out of four published studies, there is no information about the range of respondent scores. However there was a study in which 158 Korean international students and 158 do-mestic students attending two Midwestern American universities completed the PRICA instrument and the range of responses was reported (Hong, 2003). American students scored between -16 and 28 while Korean students scored between -21 and 24. In another published study (Bippus & Dorjee, 2002), 150 college students attending a large Western university completed the PRECA instrument. Scores on the PRECA ranged from 16 to 54. High, middle, and low scores of communication apprehension are based on standard deviations from the sample mean. These anchor scores constitute various degrees of competence but only based on the sample. The ranges of responses or degrees of competence have not been standardized on a large scale.

Primary Strengths: These self-report questionnaires are easily administered as paper and pencil tests. Each mea-sure has only 14 items that can be answered quickly. Since the PRICA and PRECA are modeled after the reliable and valid PRCA-24, there is confi dence in their use.

Primary Weaknesses: PRICA and PRECA have been tested mostly on Caucasians. Not enough information is known how other ethnic and/or racial groups experience communication apprehension. There is also the possibility of a social desirability bias in which respondents are reluctant to report their apprehension about communicating with different cultural, ethnic, and/or racial groups.



Predictive: Both PRICA and PRECA are predictive of the frequency of contact with people from a different country; however PRECA is additionally predictive of the frequency of contact with people from a different race. In another study, PRECA is signifi cantly associated with communication satisfaction and anxiety of people who engaged in an interaction with someone of a different ethnicity. Therefore both instruments have predictive validity since there is evidence of these instruments correctly predicting something that they should theoretically be able to predict.

Reliability



Internal Consistency: Cronbach’s alpha for PRICA ranged from .90 to .941. Cronbach’s alpha for PRECA ranged from .915 to .94.


Practicality and Usefulness: PRICA and PRECA can be administered to teachers, teaching assistants, and students on college campuses to determine intercultural or interethnic communication apprehension in multicultural class-rooms. These measures could also be administered to managers and employees in multinational organizations to identify intercultural or interethnic communication apprehension in diverse work settings. Finally, these measures could also be administered for government use.

Clarity of Instructions: Clarity ranges among published studies. The instructions for administering and scoring the PRICA and PRECA are provided in three studies (Bippus & Dorjee, 2002; Hong, 2003; Toale & McCroskey, 2001). However, the developers of these measures did not provide instructions for administering and scoring the PRICA or PRECA (Neuliep & McCroskey, 1997).

Standardization in Administration and Evaluation: Neuliep and McCroskey (1997) published the questionnaire items for both the PRICA and PRECA. However, the descriptions of how to score these measures differ among pub-lished studies. Some researchers have summed all 14 items to determine the score for communication apprehension. Others have split the score into a positive-item sum and negative-item sum. The negative score is subtracted from the positive score to get the fi nal score. There is standardization in determining low and high levels of communication apprehension by using the standard deviation above and below the sample mean. There is no description of how these instruments were used for evaluation of students.

Range of Reponses or Degrees of Competence: In two out of four published studies, there is no information about the range of respondent scores. However there was a study in which 158 Korean international students and 158 do-mestic students attending two Midwestern American universities completed the PRICA instrument and the range of responses was reported (Hong, 2003). American students scored between -16 and 28 while Korean students scored between -21 and 24. In another published study (Bippus & Dorjee, 2002), 150 college students attending a large Western university completed the PRECA instrument. Scores on the PRECA ranged from 16 to 54. High, middle, and low scores of communication apprehension are based on standard deviations from the sample mean. These anchor scores constitute various degrees of competence but only based on the sample. The ranges of responses or degrees of competence have not been standardized on a large scale.

Primary Strengths: These self-report questionnaires are easily administered as paper and pencil tests. Each mea-sure has only 14 items that can be answered quickly. Since the PRICA and PRECA are modeled after the reliable and valid PRCA-24, there is confi dence in their use.

Primary Weaknesses: PRICA and PRECA have been tested mostly on Caucasians. Not enough information is known how other ethnic and/or racial groups experience communication apprehension. There is also the possibility of a social desirability bias in which respondents are reluctant to report their apprehension about communicating with different cultural, ethnic, and/or racial groups.



Overall Adequacy: These measures provide a valid option for measuring intercultural or interethnic communica-tion apprehension among Caucasians, but more research is needed to assess communication apprehension among other cultural, ethnic, and racial groups.

Additional ReferencesBippus, A. M., & Dorjee, T. (2002). The validity of the PRECA as an index of interethnic communication apprehension. Com-

munication Research Reports, 19(2), 130-137.Hong, J. (2003, May). A cross-cultural comparison of the relationship between ICA, ICMS, and assertiveness/cooperativeness

tendencies. Paper presented at the meeting of the International Communication Association, San Diego, CA.Toale, M. C., & McCroskey, J. C. (2001). Ethnocentrism and trait communication apprehension as predictors of interethnic

communication apprehension and use of relational maintenance strategies in interethnic communication. Communication Quarterly, 49(1),70-83.

Reviewer: Sherry Wien



Overall Adequacy: These measures provide a valid option for measuring intercultural or interethnic communica-tion apprehension among Caucasians, but more research is needed to assess communication apprehension among other cultural, ethnic, and racial groups.

Additional ReferencesBippus, A. M., & Dorjee, T. (2002). The validity of the PRECA as an index of interethnic communication apprehension. Com-

munication Research Reports, 19(2), 130-137.Hong, J. (2003, May). A cross-cultural comparison of the relationship between ICA, ICMS, and assertiveness/cooperativeness

tendencies. Paper presented at the meeting of the International Communication Association, San Diego, CA.Toale, M. C., & McCroskey, J. C. (2001). Ethnocentrism and trait communication apprehension as predictors of interethnic

communication apprehension and use of relational maintenance strategies in interethnic communication. Communication Quarterly, 49(1),70-83.

Reviewer: Sherry Wien



Public Speaking Competency Instrument

Author / Developers: S. Thomson & M.L. Rucker

Address: S. Thomson, Department of Communications, Xavier University of Louisiana, New Orleans, LA 70125; M.L. Rucker, Department of Communication, Wright State University, Dayton, OH 45435

Source: Thomson, S. & Rucker, M.L. (2002). The development of a specialized public speaking competency scale: Test of reliability. Communication Research Reports, 67, 449-459.

Intended context or behavior: Public speaking

Intended academic level: College but is adaptable to other levels

Description: This is a 20-item instrument designed to provide precise yet quick judgments of a public speech. “Items 1-3 asked the evaluator to rate elements of the introduction. Items 4-8 asked the evaluator to rate elements of the body of the speech. Items 9-12 asked the evaluator to rate elements of the conclusion of the speech. Items 12-19 asked to the evaluator to rate elements of delivery. Item 20 asked the evaluator to provide a global assessment of the speaker’s speech communication competence based on the speech as a whole.” It “has the potential to be used as a standardized measure for incoming student assessment, as an assessment tool within the public speaking classroom, and for the evaluation of public speaking competency outcome assessment.”

Scoring description: Raters score each item on a 5-1 Likert-type scale for a scoring range of 100-20.

Norm/criterion reference populations: A videotape of a college sophomore speech was used by a group of 11 faculty members to evaluate using the instrument.

Culture and gender bias: None reported, but it is noted that the instrument “can also be used to test students with English as a second language.”

Validity

Face/content: Items on the Public Speaking Competency Instrument were matched to the Competent Speaker Speech Evaluation Form (Morreale et al, 1993) the competencies from the SCA College Entry Level Competency Task Force, (SCA, 1995), and to a prominent public speaking text.

Reliability



Practicality and usefulness: This evaluation form can be adapted to different types of speeches and can be quickly completed so that ratings could be done for a large group of speakers in a relatively short time.

Clarity of instructions: The scoring instructions are clear and the items precise so with limited training it could be easily used by evaluators with limited experience in the communication fi eld.

Range of responses or degrees of competence: The fi ve-point scale appears suffi cient.

Primary strengths: The items are very precise and effectively capture the necessary organizational and delivery elements of most public speeches.

Primary weaknesses: More testing is needed to explore biases and validity, and provide other measures of reliabil-ity. The content of a speech does not appear to be adequately assessed. For example, it would be diffi cult to evaluate the adequacy of the arguments made in a persuasive speech using this instrument.

Overall adequacy: Given its stated purpose of being an instrument to evaluate a large number of speakers in a rela-tively short period of time, the Public Speaking Competency Instrument appears promising. It could be used as a diagnostic measure to determine areas where more instruction and practice is needed as well as an outcome measure of public speaking competency.




Public Speaking Competency Instrument

Author / Developers: S. Thomson & M.L. Rucker

Address: S. Thomson, Department of Communications, Xavier University of Louisiana, New Orleans, LA 70125; M.L. Rucker, Department of Communication, Wright State University, Dayton, OH 45435

Source: Thomson, S. & Rucker, M.L. (2002). The development of a specialized public speaking competency scale: Test of reliability. Communication Research Reports, 67, 449-459.

Intended context or behavior: Public speaking

Intended academic level: College but is adaptable to other levels

Description: This is a 20-item instrument designed to provide precise yet quick judgments of a public speech. “Items 1-3 asked the evaluator to rate elements of the introduction. Items 4-8 asked the evaluator to rate elements of the body of the speech. Items 9-12 asked the evaluator to rate elements of the conclusion of the speech. Items 12-19 asked to the evaluator to rate elements of delivery. Item 20 asked the evaluator to provide a global assessment of the speaker’s speech communication competence based on the speech as a whole.” It “has the potential to be used as a standardized measure for incoming student assessment, as an assessment tool within the public speaking classroom, and for the evaluation of public speaking competency outcome assessment.”

Scoring description: Raters score each item on a 5-1 Likert-type scale for a scoring range of 100-20.

Norm/criterion reference populations: A videotape of a college sophomore speech was used by a group of 11 faculty members to evaluate using the instrument.

Culture and gender bias: None reported, but it is noted that the instrument “can also be used to test students with English as a second language.”

Validity

Face/content: Items on the Public Speaking Competency Instrument were matched to the Competent Speaker Speech Evaluation Form (Morreale et al, 1993) the competencies from the SCA College Entry Level Competency Task Force, (SCA, 1995), and to a prominent public speaking text.

Reliability



Practicality and usefulness: This evaluation form can be adapted to different types of speeches and can be quickly completed so that ratings could be done for a large group of speakers in a relatively short time.

Clarity of instructions: The scoring instructions are clear and the items precise so with limited training it could be easily used by evaluators with limited experience in the communication fi eld.

Range of responses or degrees of competence: The fi ve-point scale appears suffi cient.

Primary strengths: The items are very precise and effectively capture the necessary organizational and delivery elements of most public speeches.

Primary weaknesses: More testing is needed to explore biases and validity, and provide other measures of reliabil-ity. The content of a speech does not appear to be adequately assessed. For example, it would be diffi cult to evaluate the adequacy of the arguments made in a persuasive speech using this instrument.

Overall adequacy: Given its stated purpose of being an instrument to evaluate a large number of speakers in a rela-tively short period of time, the Public Speaking Competency Instrument appears promising. It could be used as a diagnostic measure to determine areas where more instruction and practice is needed as well as an outcome measure of public speaking competency.




Receiver Apprehension Test

Author / Developers: Lawrence Wheeless

Address: Department of Communication Studies, University of North Texas, P.O. Box 305268, Denton, TX 76203-5268, e-mail: [email protected]

Source: Wheeless, L.R. (1975). An investigation of receiver apprehension and social context dimensions of commu-nication apprehension. The Speech Teacher, 24, 261-268.


Academic level for which intended: College/adult


Description of test: 20 Likert-type items index receiver apprehension on fi ve-point scales.

Description of administration and scoring: The instrument has instructions included at the beginning, so it can be administered in large settings if necessary. A score is developed by combining scores on all the items.

Norm referencing/criterion referencing populations: The original analysis resulted in a mean of 46.93, with a stan-dard deviation of 12.67 based on a sample of 324 college students.

Tests for cultural bias (ethnicity): Both males and females were included in the scale development sample.

Validity

Content (face, intrinsic, representativeness): Items were developed to have face validity.

Predictive: The RAT scores have correlated at low levels with STAI-State scores in reaction to specifi c reading exer-cises. The low correlation is appropriate because it indicates that RAT scores are more trait than state.

Construct:: RAT scores are relatively independent of communication apprehension as measured by the PRCA. The Pearson correlation for the two scales was .20.

Reliability

Split-half: The split half reliability was .91.

Internal consistency: The alpha reliability has been found to be .80.


Practicality and usefulness: The scale is easily used.




Primary weaknesses: The RAT has been used primarily in research of communication variables rather than as an assessment technique.


Additional References Ayres, J., Wilcox, A. K., & Ayres, D. M. (1995). Receiver apprehension: An explanatory model and accompanying research.

Communication Education, 44, 223-235.Beatty, M. J. (1985). The effects of anticipating listening (state) anxiety on the stability of receiver apprehension scores. Central

States Speech Journal, 36, 72-76.



Receiver Apprehension Test

Author / Developers: Lawrence Wheeless

Address: Department of Communication Studies, University of North Texas, P.O. Box 305268, Denton, TX 76203-5268, e-mail: [email protected]

Source: Wheeless, L.R. (1975). An investigation of receiver apprehension and social context dimensions of commu-nication apprehension. The Speech Teacher, 24, 261-268.




Description of test: 20 Likert-type items index receiver apprehension on fi ve-point scales.

Description of administration and scoring: The instrument has instructions included at the beginning, so it can be administered in large settings if necessary. A score is developed by combining scores on all the items.

Norm referencing/criterion referencing populations: The original analysis resulted in a mean of 46.93, with a stan-dard deviation of 12.67 based on a sample of 324 college students.

Tests for cultural bias (ethnicity): Both males and females were included in the scale development sample.

Validity

Content (face, intrinsic, representativeness): Items were developed to have face validity.

Predictive: The RAT scores have correlated at low levels with STAI-State scores in reaction to specifi c reading exer-cises. The low correlation is appropriate because it indicates that RAT scores are more trait than state.

Construct:: RAT scores are relatively independent of communication apprehension as measured by the PRCA. The Pearson correlation for the two scales was .20.

Reliability

Split-half: The split half reliability was .91.

Internal consistency: The alpha reliability has been found to be .80.


Practicality and usefulness: The scale is easily used.




Primary weaknesses: The RAT has been used primarily in research of communication variables rather than as an assessment technique.


Additional References Ayres, J., Wilcox, A. K., & Ayres, D. M. (1995). Receiver apprehension: An explanatory model and accompanying research.

Communication Education, 44, 223-235.Beatty, M. J. (1985). The effects of anticipating listening (state) anxiety on the stability of receiver apprehension scores. Central

States Speech Journal, 36, 72-76.



Beatty, M.J., Behnke, R. R. & Henderson, L.S. (1980). An empirical validation of the receiver apprehension test as a measure of trait listening anxiety. The Western Journal of Speech Communication, 44, 132-136.

Chesebro, J.L, and McCroskey, J.C. (2001). The relationship of teacher clarity and immediacy with student state receiver ap-prehension, affect, and cognitive learning. Communication Education, 50, 1, 59-68.

Fitch-Hauser, M., Barker, D. A., Hughes, A. (1990). Receiver apprehension and listening comprehension: A linear or curvilin-ear relationship? The Southern Communication Journal, 56, 62-71.

Wheeless, L. R., Preiss, R. W., & Gayle, B. M. (1997). Receiver apprehension, informational receptivity, and cognitive process-ing. In J. A. Daly, J. C.

McCroskey, J. Ayres, T. Hopf, & D. M. Ayres (Eds.), Avoiding Communication: Shyness, reticence, and apprehension (2nd ed.) (pp. 151-187). Cresskill, NJ: Hampton Press.

Wilcox, A. K. (1997). Cognitive components of communication apprehension: What are they thinking? In J. A. Daly, J. C. Mc-Croskey, J. Ayres, T. Hopf, & D. M. Ayres (Eds.), Avoiding Communication: Shyness, reticence, and apprehension (2nd ed.) (pp. 367-377). Cresskill, NJ: Hampton Press.




Beatty, M.J., Behnke, R. R. & Henderson, L.S. (1980). An empirical validation of the receiver apprehension test as a measure of trait listening anxiety. The Western Journal of Speech Communication, 44, 132-136.

Chesebro, J.L, and McCroskey, J.C. (2001). The relationship of teacher clarity and immediacy with student state receiver ap-prehension, affect, and cognitive learning. Communication Education, 50, 1, 59-68.

Fitch-Hauser, M., Barker, D. A., Hughes, A. (1990). Receiver apprehension and listening comprehension: A linear or curvilin-ear relationship? The Southern Communication Journal, 56, 62-71.

Wheeless, L. R., Preiss, R. W., & Gayle, B. M. (1997). Receiver apprehension, informational receptivity, and cognitive process-ing. In J. A. Daly, J. C.

McCroskey, J. Ayres, T. Hopf, & D. M. Ayres (Eds.), Avoiding Communication: Shyness, reticence, and apprehension (2nd ed.) (pp. 151-187). Cresskill, NJ: Hampton Press.

Wilcox, A. K. (1997). Cognitive components of communication apprehension: What are they thinking? In J. A. Daly, J. C. Mc-Croskey, J. Ayres, T. Hopf, & D. M. Ayres (Eds.), Avoiding Communication: Shyness, reticence, and apprehension (2nd ed.) (pp. 367-377). Cresskill, NJ: Hampton Press.




Relationship Assessment Scale

Author / Developers: Susan S. Hendrick

Address: Department of Psychology, Texas Tech University, Lubbock, TX 79409 [email protected].

Source: Hendrick, S.S. (1988). A generic measure of relationship satisfaction. Journal of Marriage and Family, 50, 93-98.

Intended context or behavior: Satisfaction with close relationships


Time required: Five minutes

Description: Subjects respond to seven questions on a fi ve-point Likert scale. Subjects consider whether their needs and expectations are met and how satisfi ed they are with the relationship.

Scoring description: Responses range from one (low satisfaction) to fi ve (high satisfaction). Two items are re-versed for scoring. Mean scores for each item are reported.


Culture and gender bias: None reported.

Validity

Concurrent: A correlation of .80 was reported between the RAS and the 32-item Dyadic Adjustment Scale (Spa-nier and Thompson, 1982). Signifi cant but modest correlations in the predicted direction were reported with the Love Attitudes Scale (Hendrick & Hendrick, 1986), the Sexual Attitudes Scale (Hendrick, Hendrick, Slapion-Foote, & Foote, 1985) and the Self-Disclosure Index and Opener Scale (Miller, Berg & Archer, 1983).

Face/content: Scale items were based on the Marital Assessment Questionnaire (Hendrick, 1981).

Predictive: When the scale was administered to dating couples, it correctly predicted 91 percent of the couples who stayed together and 86 percent of the couples who ended their relationship.

Reliability

Internal consistency: An alpha of .86 and a standardized alpha of .87 were reported.


Practicality and usefulness: This is a very easy and quick instrument to use.


Standardization in administration and evaluation: Administration is standardized.

Range of responses or degrees of competence: Mean scores are reported.

Primary strengths: This scale would be useful in research and for class discussions of relational satisfaction.

Additional ReferencesBowman. A.D. (2004) Marital satisfaction and relational attachment in a sample of newly married couples. Psychological Re-

port, 95: 3, 989-991Cramer, D. (2004) Satisfaction with a romantic relationship, depression, support and confl ict. Psychology & Psychotherapy:

Theory, Research & Practice, 77: 4, 449-461.Vaugh, M.J. & Baier, M.E. (1999) Reliability and validity of the relationship assessment scale. American Journal of Family

Therapy, 27:2, 137-148.




Relationship Assessment Scale

Author / Developers: Susan S. Hendrick

Address: Department of Psychology, Texas Tech University, Lubbock, TX 79409 [email protected].

Source: Hendrick, S.S. (1988). A generic measure of relationship satisfaction. Journal of Marriage and Family, 50, 93-98.

Intended context or behavior: Satisfaction with close relationships


Time required: Five minutes

Description: Subjects respond to seven questions on a fi ve-point Likert scale. Subjects consider whether their needs and expectations are met and how satisfi ed they are with the relationship.

Scoring description: Responses range from one (low satisfaction) to fi ve (high satisfaction). Two items are re-versed for scoring. Mean scores for each item are reported.



Validity

Concurrent: A correlation of .80 was reported between the RAS and the 32-item Dyadic Adjustment Scale (Spa-nier and Thompson, 1982). Signifi cant but modest correlations in the predicted direction were reported with the Love Attitudes Scale (Hendrick & Hendrick, 1986), the Sexual Attitudes Scale (Hendrick, Hendrick, Slapion-Foote, & Foote, 1985) and the Self-Disclosure Index and Opener Scale (Miller, Berg & Archer, 1983).

Face/content: Scale items were based on the Marital Assessment Questionnaire (Hendrick, 1981).

Predictive: When the scale was administered to dating couples, it correctly predicted 91 percent of the couples who stayed together and 86 percent of the couples who ended their relationship.

Reliability

Internal consistency: An alpha of .86 and a standardized alpha of .87 were reported.


Practicality and usefulness: This is a very easy and quick instrument to use.


Standardization in administration and evaluation: Administration is standardized.

Range of responses or degrees of competence: Mean scores are reported.

Primary strengths: This scale would be useful in research and for class discussions of relational satisfaction.

Additional ReferencesBowman. A.D. (2004) Marital satisfaction and relational attachment in a sample of newly married couples. Psychological Re-

port, 95: 3, 989-991Cramer, D. (2004) Satisfaction with a romantic relationship, depression, support and confl ict. Psychology & Psychotherapy:

Theory, Research & Practice, 77: 4, 449-461.Vaugh, M.J. & Baier, M.E. (1999) Reliability and validity of the relationship assessment scale. American Journal of Family

Therapy, 27:2, 137-148.




Relational Communication Scale

Author / Developers: J.H. Burgoon and J.L. Hale

Address: Judee H. Burgoon, Communication Department, University of Arizona, Tucson, AZ 85721

Source: Burgoon, J.H. & Hale, J.L. (1987). Validation and measurement of the fundamental themes of relational communication. Communication Monographs, 54, 19-41.

Intended context or behavior : Relational messages in interpersonal interactions



Description: This measure contains 30 statements which relation to the relational

themes of immediacy/affect, similarity/depth, receptivity/trust, composure, formality, dominance, equality, and task orientation. Following an actual or recalled conversation, participants and/or observers rate interaction participants on an eight-point Likert-type scale as to the applicability of each statement in describing the interaction.

Description of scoring: Overall scores and interpretive ranges are not provided in this article. Mean scores for each item are included.

Norm referencing/criterion referencing populations: Various groups of undergraduate students were used in developing and testing this measure.

Culture/Gender bias: Male and female participants demonstrated signifi cant differences.

Validity

Predictive: The measure showed appropriate correlations with nonverbal immediate behaviors, eye contact and gaze, vocal qualities, high reward communicators, credibility and personality.

Face/Content: The article includes background information on the development of the construct along with reports of various factor analyses which were performed in developing this instrument.

Reliability

Internal consistency: Reliability coeffi cients for the various relational themes ranged from .81 to .42.


Practicality and usefulness: This measure could be used on a pre/post basis to demonstrate possible changes in interaction patterns.

Clarity of instructions: Since the article is primarily research focused, it does not include defi nite instructions for using this measure.

Degree of standardization in administration and evaluation of responses: Again because of the research focus of this article, standardized administration and evaluation procedures are not included.

Range of responses or degrees of competence: Interpretive ranges are not provided.

Primary strengths: This measure would provide an especially good beginning for discussing the themes or factors which infl uence our interactions.

Primary weaknesses: In order to be used in instructional settings, further elaboration is necessary.

Overall adequacy: This measure provides a very comprehensive perspective on interpersonal communication.

Additional ReferencesBurgoon, J. K. & Le Poire, B. A. (1999). Nonverbal cues and interpersonal judgments: participant and observer perceptions of

intimacy. Communication Monographs, 66:2, 105-124.



Relational Communication Scale

Author / Developers: J.H. Burgoon and J.L. Hale

Address: Judee H. Burgoon, Communication Department, University of Arizona, Tucson, AZ 85721

Source: Burgoon, J.H. & Hale, J.L. (1987). Validation and measurement of the fundamental themes of relational communication. Communication Monographs, 54, 19-41.

Intended context or behavior : Relational messages in interpersonal interactions



Description: This measure contains 30 statements which relation to the relational

themes of immediacy/affect, similarity/depth, receptivity/trust, composure, formality, dominance, equality, and task orientation. Following an actual or recalled conversation, participants and/or observers rate interaction participants on an eight-point Likert-type scale as to the applicability of each statement in describing the interaction.

Description of scoring: Overall scores and interpretive ranges are not provided in this article. Mean scores for each item are included.

Norm referencing/criterion referencing populations: Various groups of undergraduate students were used in developing and testing this measure.

Culture/Gender bias: Male and female participants demonstrated signifi cant differences.

Validity

Predictive: The measure showed appropriate correlations with nonverbal immediate behaviors, eye contact and gaze, vocal qualities, high reward communicators, credibility and personality.

Face/Content: The article includes background information on the development of the construct along with reports of various factor analyses which were performed in developing this instrument.

Reliability

Internal consistency: Reliability coeffi cients for the various relational themes ranged from .81 to .42.


Practicality and usefulness: This measure could be used on a pre/post basis to demonstrate possible changes in interaction patterns.

Clarity of instructions: Since the article is primarily research focused, it does not include defi nite instructions for using this measure.

Degree of standardization in administration and evaluation of responses: Again because of the research focus of this article, standardized administration and evaluation procedures are not included.

Range of responses or degrees of competence: Interpretive ranges are not provided.

Primary strengths: This measure would provide an especially good beginning for discussing the themes or factors which infl uence our interactions.

Primary weaknesses: In order to be used in instructional settings, further elaboration is necessary.

Overall adequacy: This measure provides a very comprehensive perspective on interpersonal communication.

Additional ReferencesBurgoon, J. K. & Le Poire, B. A. (1999). Nonverbal cues and interpersonal judgments: participant and observer perceptions of

intimacy. Communication Monographs, 66:2, 105-124.



Campbell, K. & Wright, K.B. (2002). On-line support groups: an investigation of relationships among source credibility, dimensions of relational communication, and perceptions of emotional support. Communication Research Reports, 19: 2, 183-193.

Pawlowski, D.R.; Rocca, K. A. & Myers, S. A. (2000). Relational messages in confl ict situations among siblings. Communica-tion Research Reports, 17:3, 271-277.

Umphrey, L.R. & Sherblom, J.C. (2001). The role of relational interdependence, relationship thinking, and relational commu-nication in three stages of romantic relationships. Communication Research Reports, 18:4, 324-333.




Campbell, K. & Wright, K.B. (2002). On-line support groups: an investigation of relationships among source credibility, dimensions of relational communication, and perceptions of emotional support. Communication Research Reports, 19: 2, 183-193.

Pawlowski, D.R.; Rocca, K. A. & Myers, S. A. (2000). Relational messages in confl ict situations among siblings. Communica-tion Research Reports, 17:3, 271-277.

Umphrey, L.R. & Sherblom, J.C. (2001). The role of relational interdependence, relationship thinking, and relational commu-nication in three stages of romantic relationships. Communication Research Reports, 18:4, 324-333.





Author / Developers: Carolyn Anderson, Matthew Martin, Bruce Riddle

Address: Carolyn Anderson, University of Akron, School of Communication, Akron, OH 44325-1003; Matthew Martin, Communication Studies Department, West Virginia University, Morgantown, WV 26506; Bruce Riddle, School of Communication Studies, Kent State University, Kent OH 44242.

Source: Anderson, C.M., Martin, M.M. & Riddle, B.L. (2001) Small group relational satisfaction scale: Development, reliability and validity. Communication Studies, 52:3, 220-234.

Intended context or behavior: Relational dimension of group communication



Description: Subjects are asked to consider their experience in a particular task group. They respond to 12 statements that measures their relational satisfaction with the group.

Scoring description: Responses to the statements range from 1 (strongly disagree) to 5 (strongly agree). Mean scores of 43.91 and 44.50 were reported in the two studies presented in the article.

Norm/criterion reference populations: Adult full-time employees in on-going work groups; college students after completing a group project;


Validity

Concurrent/construct: Measures of cohesion and consensus were positively correlated with the RSS while a measure of loneliness was negatively correlated with relational satisfaction.

Face/content: Scale items were based on descriptions of satisfying group experiences and a review of the literature on relationships in groups.

Predictive: Attitudes toward the group, assertiveness, responsiveness and feedback were found to predict relational satisfaction.

Reliability



Practicality and usefulness: This instrument could be administered at the completion of a group task and would help students develop an awareness of those elements that impact satisfaction.

Clarity of instructions: The instructions are very clear.

Standardization in administration and evaluation: The simplicity of the instrument contributes to its ease of use.

Range of responses or degrees of competence: The fi ve- point scale seems an adequate to measure sat-isfaction.

Primary strengths: Task and relational behaviors are the frequent focus of instruction in group com-munication. This instrument is helpful in developing student awareness of the relational dimension and could be used on a pre-post basis to determine if relational behaviors improve as a result of instruction and practice.




Author / Developers: Carolyn Anderson, Matthew Martin, Bruce Riddle

Address: Carolyn Anderson, University of Akron, School of Communication, Akron, OH 44325-1003; Matthew Martin, Communication Studies Department, West Virginia University, Morgantown, WV 26506; Bruce Riddle, School of Communication Studies, Kent State University, Kent OH 44242.

Source: Anderson, C.M., Martin, M.M. & Riddle, B.L. (2001) Small group relational satisfaction scale: Development, reliability and validity. Communication Studies, 52:3, 220-234.

Intended context or behavior: Relational dimension of group communication



Description: Subjects are asked to consider their experience in a particular task group. They respond to 12 statements that measures their relational satisfaction with the group.

Scoring description: Responses to the statements range from 1 (strongly disagree) to 5 (strongly agree). Mean scores of 43.91 and 44.50 were reported in the two studies presented in the article.

Norm/criterion reference populations: Adult full-time employees in on-going work groups; college students after completing a group project;


Validity

Concurrent/construct: Measures of cohesion and consensus were positively correlated with the RSS while a measure of loneliness was negatively correlated with relational satisfaction.

Face/content: Scale items were based on descriptions of satisfying group experiences and a review of the literature on relationships in groups.

Predictive: Attitudes toward the group, assertiveness, responsiveness and feedback were found to predict relational satisfaction.

Reliability



Practicality and usefulness: This instrument could be administered at the completion of a group task and would help students develop an awareness of those elements that impact satisfaction.


Standardization in administration and evaluation: The simplicity of the instrument contributes to its ease of use.

Range of responses or degrees of competence: The fi ve- point scale seems an adequate to measure sat-isfaction.

Primary strengths: Task and relational behaviors are the frequent focus of instruction in group com-munication. This instrument is helpful in developing student awareness of the relational dimension and could be used on a pre-post basis to determine if relational behaviors improve as a result of instruction and practice.



Primary weaknesses: Since this is a self-report measure, it does not measure actual behaviors in group settings.

Overall adequacy: This is a very useful scale. It provides a measure of the effectiveness of the group’s interaction which can complement the assessment of the group’s fi nal product.




Primary weaknesses: Since this is a self-report measure, it does not measure actual behaviors in group settings.

Overall adequacy: This is a very useful scale. It provides a measure of the effectiveness of the group’s interaction which can complement the assessment of the group’s fi nal product.




Reticence Scale

Author / Developers: James A. Keaten, Lynne Kelly, and Cynthia Finch

Address: James A. Keaten, Department of Communication, University of Northern Colorado, Greeley, CO 80639.

Source: Keaten, J.A., Kelly, L. & Finch, C. (1997) Development of an instrument to measure reticence. Communi-cation Quarterly, 45:1, 37-54.

Intended context or behavior: Communication reticence-anxiety associated with lacking abilities to communicate effectively



Description: On a six-point Likert scale (6=strongly agree; 1=strongly disagree), subjects respond to 24 statements on their perceptions of their communication abilities.

Description of scoring: Total scores are based on six subscales. “The total scores ranged from 6 to 126, with a lower number indicating lower levels of reticence.” A mean of 70.53 was reported. Scoring for each of the subscales is described. The subscales are anxiety, knowledge, timing, organization, delivery, and memory. Means for each subscale are also included. Scores of 54 or less indicate low reticence. Scores of 78 or more indicate high reticence (Kelly, Keaten, Finch, 2004).


Validity

Concurrent: Very high or high correlations were reported between the scale and responses to open ended questions related to the six subscales.

Face/Content: Items were based upon experience in the Penn State Reticence Program.

Construct: Appropriate and signifi cant correlations with the Personal Report of Communication Apprehension (PRCA-24), the Willingness to Communicate scale, and the Conversational Skills Rating Scale were reported. Reti-cence was established as a distinct yet related construct.

Reliability

Internal consistency: An overall reliability score of .95 was reported. Subscale reliabilities were also high.


Practicality and usefulness: This measure is useful for monitoring whether or not skills training

has had an impact. Students should feel less reticence as their abilities to communicate improve.

Clarity of instructions: Instructions and items are very clear.

Standardization in administration and evaluation: Administration and responses are

standardized.

Range of responses or degrees of competence: No ranges are reported.

Primary strengths: The six dimensions allow for more focus on specifi c targets for instruction and developing individualized plans for improving communication abilities.

Overall adequacy: This measure is very directly related to instruction and can serve as an effective indicator of the impact of instruction.



Reticence Scale

Author / Developers: James A. Keaten, Lynne Kelly, and Cynthia Finch

Address: James A. Keaten, Department of Communication, University of Northern Colorado, Greeley, CO 80639.

Source: Keaten, J.A., Kelly, L. & Finch, C. (1997) Development of an instrument to measure reticence. Communi-cation Quarterly, 45:1, 37-54.

Intended context or behavior: Communication reticence-anxiety associated with lacking abilities to communicate effectively



Description: On a six-point Likert scale (6=strongly agree; 1=strongly disagree), subjects respond to 24 statements on their perceptions of their communication abilities.

Description of scoring: Total scores are based on six subscales. “The total scores ranged from 6 to 126, with a lower number indicating lower levels of reticence.” A mean of 70.53 was reported. Scoring for each of the subscales is described. The subscales are anxiety, knowledge, timing, organization, delivery, and memory. Means for each subscale are also included. Scores of 54 or less indicate low reticence. Scores of 78 or more indicate high reticence (Kelly, Keaten, Finch, 2004).


Validity

Concurrent: Very high or high correlations were reported between the scale and responses to open ended questions related to the six subscales.

Face/Content: Items were based upon experience in the Penn State Reticence Program.

Construct: Appropriate and signifi cant correlations with the Personal Report of Communication Apprehension (PRCA-24), the Willingness to Communicate scale, and the Conversational Skills Rating Scale were reported. Reti-cence was established as a distinct yet related construct.

Reliability

Internal consistency: An overall reliability score of .95 was reported. Subscale reliabilities were also high.


Practicality and usefulness: This measure is useful for monitoring whether or not skills training

has had an impact. Students should feel less reticence as their abilities to communicate improve.

Clarity of instructions: Instructions and items are very clear.

Standardization in administration and evaluation: Administration and responses are

standardized.

Range of responses or degrees of competence: No ranges are reported.

Primary strengths: The six dimensions allow for more focus on specifi c targets for instruction and developing individualized plans for improving communication abilities.

Overall adequacy: This measure is very directly related to instruction and can serve as an effective indicator of the impact of instruction.



Additional ReferencesKelly, L., Keaten, J. A. & Finch, C. (2004). Reticent and non-reticent college students’ preferred communication channels for

interacting with faculty. Communication Research Reports, 21:2, 197-209.




Additional ReferencesKelly, L., Keaten, J. A. & Finch, C. (2004). Reticent and non-reticent college students’ preferred communication channels for

interacting with faculty. Communication Research Reports, 21:2, 197-209.




Revised Self-Disclosure Scale

Author / Developers: L.R. Wheeless

Address: L.R. Wheeless, Department of Communication Studies, University of North Texas, Box 5266, Denton, TX 76203-0266.

Source: Wheeless, L.R. (1978). A follow-up study of the relationships among trust, disclosure, and interpersonal solidarity. Human Communication Research, 4, 143-57.

Intended context or behavior: Self-disclosure



Description: With a target person in mind, students respond on a seven-point Likert-type scale to 31 statements designed to measure fi ve dimensions of self-disclosure. The fi ve dimensions were intent, amount, content, depth and accuracy. To measure general disclosiveness the target individual instruction is replaced by more general instruc-tions.

Description of scoring: Scores on several items are reversed. No overall interpretative framework for scores is provided.

Validity

Predictive: Self-disclosure and trustworthiness are predictors of interpersonal solidarity.

Content: Items are drawn from a conceptualization of self-disclosure that included eight factors found in the lit-erature.

Construct: Various analyses were performed with earlier versions of the scale in developing this fi ve dimension ver-sion. Self-disclosure correlated in an appropriate manner with measures of solidarity, apprehension and intimacy.

Reliability

Scoring: The scale has produced co-effi cient alphas ranging from .81 to .91.

Internal consistency: Reliability for the various dimensions range from .84 to .91.


Practicality and Usefulness: This is a very easy, quick means of demonstrating the various dimensions of self-disclosure.

Clarity of instructions: Directions are clear and simple.

Degree of standardization: One version uses specifi c targets for self-disclosure. Standardization is more likely with the more general version.

Identifi cation of a range of responses or degrees of competence: Interpretation for responses is not provided.

Primary strengths: This instrument has undergone thorough development and would be effective in illustrating to students how the content and amount of disclosure can change in various situations.

Primary weaknesses: It would be helpful to have some general indicators of how scores compare. Concurrent va-lidity could have been established with similar measures.

Overall adequacy: It fulfi lls its intended objective.




Revised Self-Disclosure Scale

Author / Developers: L.R. Wheeless

Address: L.R. Wheeless, Department of Communication Studies, University of North Texas, Box 5266, Denton, TX 76203-0266.

Source: Wheeless, L.R. (1978). A follow-up study of the relationships among trust, disclosure, and interpersonal solidarity. Human Communication Research, 4, 143-57.

Intended context or behavior: Self-disclosure



Description: With a target person in mind, students respond on a seven-point Likert-type scale to 31 statements designed to measure fi ve dimensions of self-disclosure. The fi ve dimensions were intent, amount, content, depth and accuracy. To measure general disclosiveness the target individual instruction is replaced by more general instruc-tions.

Description of scoring: Scores on several items are reversed. No overall interpretative framework for scores is provided.

Validity

Predictive: Self-disclosure and trustworthiness are predictors of interpersonal solidarity.

Content: Items are drawn from a conceptualization of self-disclosure that included eight factors found in the lit-erature.

Construct: Various analyses were performed with earlier versions of the scale in developing this fi ve dimension ver-sion. Self-disclosure correlated in an appropriate manner with measures of solidarity, apprehension and intimacy.

Reliability

Scoring: The scale has produced co-effi cient alphas ranging from .81 to .91.

Internal consistency: Reliability for the various dimensions range from .84 to .91.


Practicality and Usefulness: This is a very easy, quick means of demonstrating the various dimensions of self-disclosure.

Clarity of instructions: Directions are clear and simple.

Degree of standardization: One version uses specifi c targets for self-disclosure. Standardization is more likely with the more general version.

Identifi cation of a range of responses or degrees of competence: Interpretation for responses is not provided.

Primary strengths: This instrument has undergone thorough development and would be effective in illustrating to students how the content and amount of disclosure can change in various situations.

Primary weaknesses: It would be helpful to have some general indicators of how scores compare. Concurrent va-lidity could have been established with similar measures.

Overall adequacy: It fulfi lls its intended objective.




Graham, E.E. (1994). Revised self-disclosure scale. In R.B.Rubin, P.Palmgreen, H.E. Sypher (Eds.), Communication Research Measures: A Sourcebook. pp. 322-26. New York: Guilford Press.

Myers, Scott A.; Johnson, Aaron D. (2004).Perceived solidarity, self-disclosure, and trust in organizational peer relationships. Communication Research Reports, 21:1, 75-83.

Wheeless, L.R. (1976). Self-disclosure and interpersonal solidarity: Measurement, validation and relationships. Human Com-munication Research, 3, 47-61.

Wheeless, L.R., & Grotz, J. (1976). Conceptualization and measurement of reported self-disclosure. Human Communication Research, 2, 338-46.

Wheeless, L.R., Grotz, J. (1977). The measurement of trust and its relationship to self-disclosure. Human Communication Research, 3, 250-57.

Wheeless, L.R., Nesser, K. & McCroskey, J.C. (1986). The relationships of self-disclosure and disclosiveness to highand low communication apprehension. Communication Research Reports, 3, 129-34.




Graham, E.E. (1994). Revised self-disclosure scale. In R.B.Rubin, P.Palmgreen, H.E. Sypher (Eds.), Communication Research Measures: A Sourcebook. pp. 322-26. New York: Guilford Press.

Myers, Scott A.; Johnson, Aaron D. (2004).Perceived solidarity, self-disclosure, and trust in organizational peer relationships. Communication Research Reports, 21:1, 75-83.

Wheeless, L.R. (1976). Self-disclosure and interpersonal solidarity: Measurement, validation and relationships. Human Com-munication Research, 3, 47-61.

Wheeless, L.R., & Grotz, J. (1976). Conceptualization and measurement of reported self-disclosure. Human Communication Research, 2, 338-46.

Wheeless, L.R., Grotz, J. (1977). The measurement of trust and its relationship to self-disclosure. Human Communication Research, 3, 250-57.

Wheeless, L.R., Nesser, K. & McCroskey, J.C. (1986). The relationships of self-disclosure and disclosiveness to highand low communication apprehension. Communication Research Reports, 3, 129-34.




Self-Perceived Communication Competence Scale

Author / Developers: J.C. McCroskey and L. L. McCroskey

Address: J.C. McCroskey, Communication Studies, University of Alabama-Birmingham, 1055 Bldg., Room 122A, Birmingham, AL 35294. [email protected]

Source: McCroskey, J.C. & McCroskey, L.L. (1988). Self-report as an approach to measuring communication com-petence. Communication Education, 5, 108-13.

Intended context or behavior: Overall perception of competence as a communicator


Time required: 10-15 minutes to complete and score

Description: The 12 items on this test ask respondents to rate their communication competence on a scale of 0-100 in four contexts (public speaking, large meeting, small group and interpersonal) and with regard to three types of receivers (stranger, acquaintance, and friend).

Description of scoring: Respondents add designated items for a score on each context and each type of receiver. A global score is then computed for overall competence. Subscores on context and receiver can also be generated. The article reports mean scores for the global score and subscores that could be used as a basis for comparison.

Norm referencing/criterion referencing populations: Undergraduate students (n=344) were used in developing this instrument.

Validity

Face/Content: Items were based upon the McCroskey defi nition of communication competence, which is “ad-equate ability to pass along or give information; the ability to make known by talking or writing.”

Construct: The article discusses the role of self-report instruments as valid measures of respondent perceptions. The scale has also been used with at-risk students to identify types of communication interventions that would be of assistance in improving academic achievement.

Reliability

Internal consistency: Reliability estimates for subscores ranged from .44 to.87. Global score reliability was esti-mated at .92.


Practicality and usefulness: It provides an indication of the respondent’s perceptions of his/her communication competence. It does not measure actual competence. It could be used at the beginning and ending of a course or major to demonstrate changes in one’s perception of communication abilities.

Clarity of instructions: Instructions for completing and scoring the scale are very straightforward.

Standardization in administration and evaluation of responses: It should be relatively easy to consistently ad-minister and score this measure.

Range of responses or degrees of competence: Labels are not attached to scores. While a mean score is given and could be used as basis for comparison, no judgments are made on the perception of competence.

Primary strengths: This measure is very effi cient. It provides a quick, easy indication of growth.

Primary weaknesses: Not much testing has been done with regard to the reliability, validity, and bias of this mea-sure.



Self-Perceived Communication Competence Scale

Author / Developers: J.C. McCroskey and L. L. McCroskey

Address: J.C. McCroskey, Communication Studies, University of Alabama-Birmingham, 1055 Bldg., Room 122A, Birmingham, AL 35294. [email protected]

Source: McCroskey, J.C. & McCroskey, L.L. (1988). Self-report as an approach to measuring communication com-petence. Communication Education, 5, 108-13.

Intended context or behavior: Overall perception of competence as a communicator


Time required: 10-15 minutes to complete and score

Description: The 12 items on this test ask respondents to rate their communication competence on a scale of 0-100 in four contexts (public speaking, large meeting, small group and interpersonal) and with regard to three types of receivers (stranger, acquaintance, and friend).

Description of scoring: Respondents add designated items for a score on each context and each type of receiver. A global score is then computed for overall competence. Subscores on context and receiver can also be generated. The article reports mean scores for the global score and subscores that could be used as a basis for comparison.

Norm referencing/criterion referencing populations: Undergraduate students (n=344) were used in developing this instrument.

Validity

Face/Content: Items were based upon the McCroskey defi nition of communication competence, which is “ad-equate ability to pass along or give information; the ability to make known by talking or writing.”

Construct: The article discusses the role of self-report instruments as valid measures of respondent perceptions. The scale has also been used with at-risk students to identify types of communication interventions that would be of assistance in improving academic achievement.

Reliability

Internal consistency: Reliability estimates for subscores ranged from .44 to.87. Global score reliability was esti-mated at .92.


Practicality and usefulness: It provides an indication of the respondent’s perceptions of his/her communication competence. It does not measure actual competence. It could be used at the beginning and ending of a course or major to demonstrate changes in one’s perception of communication abilities.

Clarity of instructions: Instructions for completing and scoring the scale are very straightforward.

Standardization in administration and evaluation of responses: It should be relatively easy to consistently ad-minister and score this measure.

Range of responses or degrees of competence: Labels are not attached to scores. While a mean score is given and could be used as basis for comparison, no judgments are made on the perception of competence.

Primary strengths: This measure is very effi cient. It provides a quick, easy indication of growth.

Primary weaknesses: Not much testing has been done with regard to the reliability, validity, and bias of this mea-sure.



Overall adequacy: It provides another way of measuring the development of communication abilities. It would be particularly good as a “discussion starter” and may be of assistance in effectively planning interventions for aca-demically at-risk students.

It has been used at the University of Dayton as part of their assessment of their basic communication course.

Additional ReferencesChesebro, J. W., McCroskey, J.C., Atwater. D.F., Behrenfuss, R.M., Cawelti. G., Gaudino, J.L. & Hodges, H.(1992). Com-

munication apprehension and self-perceived communication competence of at-risk students. Communication Education, 41, 345-360.

Pederson, J. & Tkachuk, H. (2005) A Cultural Comparison Of Ethnocentrism, Willingness To Communicate And Self- Per-ceived Communication Competence Between Indian And American College Students. A paper presented at the Annual Meet-ing of the National Communication Association, Boston, November 2005.

Rosenfeld, L.B., Grant, C.H. & McCroskey, J.C. (1995). Communication apprehension and self-perceived communication competence of academically gifted students. Communication Education, 41, 79-86.




Overall adequacy: It provides another way of measuring the development of communication abilities. It would be particularly good as a “discussion starter” and may be of assistance in effectively planning interventions for aca-demically at-risk students.

It has been used at the University of Dayton as part of their assessment of their basic communication course.

Additional ReferencesChesebro, J. W., McCroskey, J.C., Atwater. D.F., Behrenfuss, R.M., Cawelti. G., Gaudino, J.L. & Hodges, H.(1992). Com-

munication apprehension and self-perceived communication competence of at-risk students. Communication Education, 41, 345-360.

Pederson, J. & Tkachuk, H. (2005) A Cultural Comparison Of Ethnocentrism, Willingness To Communicate And Self- Per-ceived Communication Competence Between Indian And American College Students. A paper presented at the Annual Meet-ing of the National Communication Association, Boston, November 2005.

Rosenfeld, L.B., Grant, C.H. & McCroskey, J.C. (1995). Communication apprehension and self-perceived communication competence of academically gifted students. Communication Education, 41, 79-86.




Socio-Communicative Style and Orientation

Author / Developers: Virginia Richmond & James McCroskey


Source: Richmond, V.P. (2002) Socio-communicative style and orientation. In Chesebro, J.L. & McCroskey, J.C. (eds.) Communication for Teachers. Boston: Allyn & Bacon, 104-115.

Intended context or behavior: Communication style



Test description: The orientation scale is completed by an individual with whom the subject interacts. The Style scale is a self-assessment. Both scales lists 10 assertive personality characteristics and 10 responsive personality scale characteristics.

Description: Responses to each characteristic range from one (strongly disagree that it applies) to fi ve (strongly agree that it applies). High assertives/high responsives are categorized as expressives (competent). High assertives/low responsives are drivers (aggressive). Low assertives/high responsives are classifi ed as amiables (submissive). Low assertives/low responsives are termed analyticals (noncompetent).


Culture and gender bias: Ishii & Thomason (1989) reported gender and cultural differences.

Validity

Concurrent: A measure of teacher immediacy was positively correlated with assertiveness and responsiveness (Thomas et al 1994).

Face/content: Items were drawn from previous research on communication style.

Predictive: Students trust teachers who are categorized as expressive/competent (Wooten & McCroskey, 1996).

Construct: “Competent communicators (expressive) were less tense, smiled more, and moved more than submis-sive and aggressive communicators” (Richmond, 2002).

Reliability

Internal consistency: Split half reliabilities of .88 for assertiveness and .93 for responsive were reported. (Rich-mond & McCroskey, 1990).


Practicality and usefulness: The instrument provides a good introduction of communication style and could be used to evaluate whether an individual is fl exible in adapting his/her style to the situation.


Standardization in administration and evaluation: Mean scores and break points for the categories are not pro-vided.

Range of responses or degrees of competence: The fi ve-point scale and four classifi cations are adequate. Descrip-tions of each classifi cation are fully explained.



Socio-Communicative Style and Orientation

Author / Developers: Virginia Richmond & James McCroskey


Source: Richmond, V.P. (2002) Socio-communicative style and orientation. In Chesebro, J.L. & McCroskey, J.C. (eds.) Communication for Teachers. Boston: Allyn & Bacon, 104-115.

Intended context or behavior: Communication style



Test description: The orientation scale is completed by an individual with whom the subject interacts. The Style scale is a self-assessment. Both scales lists 10 assertive personality characteristics and 10 responsive personality scale characteristics.

Description: Responses to each characteristic range from one (strongly disagree that it applies) to fi ve (strongly agree that it applies). High assertives/high responsives are categorized as expressives (competent). High assertives/low responsives are drivers (aggressive). Low assertives/high responsives are classifi ed as amiables (submissive). Low assertives/low responsives are termed analyticals (noncompetent).


Culture and gender bias: Ishii & Thomason (1989) reported gender and cultural differences.

Validity

Concurrent: A measure of teacher immediacy was positively correlated with assertiveness and responsiveness (Thomas et al 1994).

Face/content: Items were drawn from previous research on communication style.

Predictive: Students trust teachers who are categorized as expressive/competent (Wooten & McCroskey, 1996).

Construct: “Competent communicators (expressive) were less tense, smiled more, and moved more than submis-sive and aggressive communicators” (Richmond, 2002).

Reliability

Internal consistency: Split half reliabilities of .88 for assertiveness and .93 for responsive were reported. (Rich-mond & McCroskey, 1990).


Practicality and usefulness: The instrument provides a good introduction of communication style and could be used to evaluate whether an individual is fl exible in adapting his/her style to the situation.


Standardization in administration and evaluation: Mean scores and break points for the categories are not pro-vided.

Range of responses or degrees of competence: The fi ve-point scale and four classifi cations are adequate. Descrip-tions of each classifi cation are fully explained.



Primary strengths: Since much of the research has occurred in the instructional setting, these instruments would be particularly effective when working with pre-service teachers.

Primary weaknesses: It would be helpful to pair these instruments with a measure of fl exibility to ascertain the relationship between the three components of communication competency.

Overall adequacy: Both measures could be used a pre-post evaluation of instruction to develop interpersonal com-munication abilities.

Additional ReferencesMcCroskey, L.L., Richmond, V.P. & McCroskey, J.C. (2002). The scholarship of teaching and learning: Contributions from the

discipline of Communication. Communication Education, 51, 383-392.Myers, S.A. (1998). Instructor socio-communicative style, argumentativeness and verbal Aggressiveness in the college class-

room. Communication Research Reports, 15, 141-150.Myers, S.A., Martin, M.M., & Mottet, T. P. (2002). Students’ motives for communicating with their instructors: Considering

instructor socio-communicative style, student socio-communicative orientation, and student gender. Communication Educa-tion. 51, 121-133.

Wanzer, M. B., & McCroskey, J. C. (1998). Teacher socio-communicative style as a correlate of student affect toward teacher and course material. Communication Education. 47, 43-52.

Wooten, A.G., & McCroskey, J. C. (1996). Student trust of teacher as a function of socio-communicative style of teacher and socio-communicative orientation of student. Communication Research Reports. 13, 94-100.




Primary strengths: Since much of the research has occurred in the instructional setting, these instruments would be particularly effective when working with pre-service teachers.

Primary weaknesses: It would be helpful to pair these instruments with a measure of fl exibility to ascertain the relationship between the three components of communication competency.

Overall adequacy: Both measures could be used a pre-post evaluation of instruction to develop interpersonal com-munication abilities.

Additional ReferencesMcCroskey, L.L., Richmond, V.P. & McCroskey, J.C. (2002). The scholarship of teaching and learning: Contributions from the

discipline of Communication. Communication Education, 51, 383-392.Myers, S.A. (1998). Instructor socio-communicative style, argumentativeness and verbal Aggressiveness in the college class-

room. Communication Research Reports, 15, 141-150.Myers, S.A., Martin, M.M., & Mottet, T. P. (2002). Students’ motives for communicating with their instructors: Considering

instructor socio-communicative style, student socio-communicative orientation, and student gender. Communication Educa-tion. 51, 121-133.

Wanzer, M. B., & McCroskey, J. C. (1998). Teacher socio-communicative style as a correlate of student affect toward teacher and course material. Communication Education. 47, 43-52.

Wooten, A.G., & McCroskey, J. C. (1996). Student trust of teacher as a function of socio-communicative style of teacher and socio-communicative orientation of student. Communication Research Reports. 13, 94-100.




Social Skills Inventory

Author / Developers: R.E. Riggio

Address: California State University, Department of Psychology, Fullerton, CA 92634

Source: Riggio, R.E. (1986). Assessment of basic social skills. Journal of Personality and Social Psychology, 51, 649-60.

Intended context or behavior: Interpersonal/social communication abilities



Description: This self-report inventory includes 105 statements designed to assess an

individual’s social skills on seven different subscales: emotional expressiveness, emotional sensitivity, emotional control, social expressiveness, social sensitivity, social control, and social manipulation. Respondents indicate on a nine point Likert-type scale (-4 to +4), the extent to which each statement applies to them.

Description of scoring: After reversing the scoring on 37 items, each subscale (15 items each) is totaled. A global score can be computed by totaling the seven subscales. The article includes mean scores for each subscale and for the global score.

Norm referencing/criterion referencing populations: Various groups of undergraduate students were used in the testing and development of this measure.

Cultural/gender bias: Males and females scored differently on several of the subscales.

Validity

Predictive: Respondents with higher scores were evaluated more highly by observers.

Concurrent: Correlations were reported between the SSI and a self-report measure of social behaviors.

Face/Content: Conceptualization of the seven scale structure, development of instrument and resulting factor anal-yses are discussed in the article.

Construct: Measures of personality, self-monitoring, social desirability, and self-confi dence correlated with the SSI in the predicted manner.

Reliability

Test/retest: After a two week interval, test-retest correlation ranged from .81 to .96 for the seven subscales.

Internal consistency: Reliability coeffi cients ranging from .75 to .88 were reported for the various subscales.


Practicality and usefulness: This may provide a very thorough pre/post measure suitable for interpersonal com-munication programs.

Clarity of instructions: Directions for completing and scoring the measure are clear.

Standardization in administration and evaluation of responses: The measure is highly standardized.

Identifi cation of a range of responses or degrees of competence: Interpretative ranges are not offered by the author. Mean scores are provided.

Primary strengths: This measure has undergone extensive validation. It is based upon a very complete conceptu-alization of social abilities and is, therefore, very comprehensive in its coverage.



Social Skills Inventory

Author / Developers: R.E. Riggio

Address: California State University, Department of Psychology, Fullerton, CA 92634

Source: Riggio, R.E. (1986). Assessment of basic social skills. Journal of Personality and Social Psychology, 51, 649-60.

Intended context or behavior: Interpersonal/social communication abilities



Description: This self-report inventory includes 105 statements designed to assess an

individual’s social skills on seven different subscales: emotional expressiveness, emotional sensitivity, emotional control, social expressiveness, social sensitivity, social control, and social manipulation. Respondents indicate on a nine point Likert-type scale (-4 to +4), the extent to which each statement applies to them.

Description of scoring: After reversing the scoring on 37 items, each subscale (15 items each) is totaled. A global score can be computed by totaling the seven subscales. The article includes mean scores for each subscale and for the global score.

Norm referencing/criterion referencing populations: Various groups of undergraduate students were used in the testing and development of this measure.

Cultural/gender bias: Males and females scored differently on several of the subscales.

Validity

Predictive: Respondents with higher scores were evaluated more highly by observers.

Concurrent: Correlations were reported between the SSI and a self-report measure of social behaviors.

Face/Content: Conceptualization of the seven scale structure, development of instrument and resulting factor anal-yses are discussed in the article.

Construct: Measures of personality, self-monitoring, social desirability, and self-confi dence correlated with the SSI in the predicted manner.

Reliability

Test/retest: After a two week interval, test-retest correlation ranged from .81 to .96 for the seven subscales.

Internal consistency: Reliability coeffi cients ranging from .75 to .88 were reported for the various subscales.


Practicality and usefulness: This may provide a very thorough pre/post measure suitable for interpersonal com-munication programs.

Clarity of instructions: Directions for completing and scoring the measure are clear.

Standardization in administration and evaluation of responses: The measure is highly standardized.

Identifi cation of a range of responses or degrees of competence: Interpretative ranges are not offered by the author. Mean scores are provided.

Primary strengths: This measure has undergone extensive validation. It is based upon a very complete conceptu-alization of social abilities and is, therefore, very comprehensive in its coverage.



Primary weaknesses: The SSI may be better suited for research than instruction since it may not be sensitive to change in social behavior. It could be used on pre/post basis for long term assessment of majors.

Overall adequacy: It could be used in conjunction with other measures as an indicator of interpersonal effective-ness. When coupled with two other measures, the SSI has predicted college persistence (Kahn et. al., 2002).

Additional ReferencesEngin, D.M., Hamarta, E., Ari, R. (2005). An investigation of social skills and loneliness levels of university students with

respect to their attachment styles in a sample of Turkish students. Social Behavior & Personality, 33, 19-32.Friedman, H.S., Riggio, R.E. & Casella, D.F. (1988). Nonverbal skill, personal charisma, and initial attraction. Personality and

Social Psychology Bulletin, 14, 203-11.Hirokawa, K., Yagi, A., Miyata, Y. (2004). An experimental examination of the effects of sex and masculinity/femininity on

psychological, physiological, and behavioral responses during communication situations. Sex Roles, 51, 91-100.Kahn, J.H., Nauta, M.M., Gailbreath, R.D., Tipps, J. & Chartrand, J.M. (2002). The utility of career and personality assessment

in predicting academic progress. Journal of Career Assessment, 10, 3-23.Riggio R.E. & Friedman, H.S. (1986). Impression formation: the role of expressive behavior. Journal of Personality and Social

Psychology, 50, 421-7.Riggio, R.E. & Sotoodeh, Y. (1989) Social skills and birth order. Psychological Reports, 64, 211-17.Riggio, R.E., & Throckmorton, B. (1988) The relative effects of verbal and nonverbal behavior, appearance, and social skills

on evaluations made in hiring interviews. Journal of Applied Social Psychology, 18, 331-48.Tucker, J.S. & Riggio, R.E. (1988) The role of social skills in encoding posed and spontaneous facial expressions. Journal of

Nonverbal Behavior, 12, 87-97.




Primary weaknesses: The SSI may be better suited for research than instruction since it may not be sensitive to change in social behavior. It could be used on pre/post basis for long term assessment of majors.

Overall adequacy: It could be used in conjunction with other measures as an indicator of interpersonal effective-ness. When coupled with two other measures, the SSI has predicted college persistence (Kahn et. al., 2002).

Additional ReferencesEngin, D.M., Hamarta, E., Ari, R. (2005). An investigation of social skills and loneliness levels of university students with

respect to their attachment styles in a sample of Turkish students. Social Behavior & Personality, 33, 19-32.Friedman, H.S., Riggio, R.E. & Casella, D.F. (1988). Nonverbal skill, personal charisma, and initial attraction. Personality and

Social Psychology Bulletin, 14, 203-11.Hirokawa, K., Yagi, A., Miyata, Y. (2004). An experimental examination of the effects of sex and masculinity/femininity on

psychological, physiological, and behavioral responses during communication situations. Sex Roles, 51, 91-100.Kahn, J.H., Nauta, M.M., Gailbreath, R.D., Tipps, J. & Chartrand, J.M. (2002). The utility of career and personality assessment

in predicting academic progress. Journal of Career Assessment, 10, 3-23.Riggio R.E. & Friedman, H.S. (1986). Impression formation: the role of expressive behavior. Journal of Personality and Social

Psychology, 50, 421-7.Riggio, R.E. & Sotoodeh, Y. (1989) Social skills and birth order. Psychological Reports, 64, 211-17.Riggio, R.E., & Throckmorton, B. (1988) The relative effects of verbal and nonverbal behavior, appearance, and social skills

on evaluations made in hiring interviews. Journal of Applied Social Psychology, 18, 331-48.Tucker, J.S. & Riggio, R.E. (1988) The role of social skills in encoding posed and spontaneous facial expressions. Journal of

Nonverbal Behavior, 12, 87-97.




Speech Evaluation Instrument

Author / Developers: C.J. Zahn and R. Hopper

Address: C.J. Zahn, Dept. of Communication, Bachelor Hall 160, Miami University, Oxford, OH 45056

Source: C.J. Zahn & Hopper, R. (1985). Measuring language attitudes: The speech evaluation instrument. Journal of Language and Social Psychology, 4, 113-123.

Context or communication behavior for which intended: Reactions to various dialects and vocal quali-ties



Description: After listening (or reading a transcript) to speakers of various dialectics,

respondents rate each on 30 qualities (a 22-item version is also available) structured on a seven point bi-polar semantic differential. The items have been factored into three subscales: superiority, attractiveness, and dynamism.

Description of scoring: The authors suggest considering each subscale separately and comparing the speakers across subscales. Since scores will depend upon the stimulus, no interpretative ranges are of-fered.


Validity

Predictive: The authors report that various studies have been conducted demonstrating the predictive validity of each subscale.

Concurrent: The authors report that appropriate correlations were found with related measures.

Face/Content: During development factor analyses were used to validate the items in the subscales.

Reliability

Internal consistency: Reliability co-effi cients for the three subscales ranged from .80 to .90.


Practicality and usefulness: This instrument might be adapted for intercultural/multicultural courses as an indica-tor of changes in student sensitivity to diversity.

Clarity of instructions: Directions for completing the instrument are very clear.

Degree of standardization in administration and evaluation of responses: Test administrators develop their own tapes as the stimulus for this measure so standardization occurs only with a particular context.

Identifi cation of a range of responses or degrees of competence: Interpretation of scores is not provided.

Primary strengths: This measure has been successful used to study reactions to various groups of people. It would be an interesting starting point for discussion of cultural or gender difference.

Primary weaknesses: The instrument seems better suited to research than to classroom use.

Overall adequacy: The semantic differential is very thorough and the three subscales would provide for interesting discussion.



Speech Evaluation Instrument

Author / Developers: C.J. Zahn and R. Hopper

Address: C.J. Zahn, Dept. of Communication, Bachelor Hall 160, Miami University, Oxford, OH 45056

Source: C.J. Zahn & Hopper, R. (1985). Measuring language attitudes: The speech evaluation instrument. Journal of Language and Social Psychology, 4, 113-123.

Context or communication behavior for which intended: Reactions to various dialects and vocal quali-ties



Description: After listening (or reading a transcript) to speakers of various dialectics,

respondents rate each on 30 qualities (a 22-item version is also available) structured on a seven point bi-polar semantic differential. The items have been factored into three subscales: superiority, attractiveness, and dynamism.

Description of scoring: The authors suggest considering each subscale separately and comparing the speakers across subscales. Since scores will depend upon the stimulus, no interpretative ranges are of-fered.


Validity

Predictive: The authors report that various studies have been conducted demonstrating the predictive validity of each subscale.

Concurrent: The authors report that appropriate correlations were found with related measures.

Face/Content: During development factor analyses were used to validate the items in the subscales.

Reliability

Internal consistency: Reliability co-effi cients for the three subscales ranged from .80 to .90.


Practicality and usefulness: This instrument might be adapted for intercultural/multicultural courses as an indica-tor of changes in student sensitivity to diversity.

Clarity of instructions: Directions for completing the instrument are very clear.

Degree of standardization in administration and evaluation of responses: Test administrators develop their own tapes as the stimulus for this measure so standardization occurs only with a particular context.

Identifi cation of a range of responses or degrees of competence: Interpretation of scores is not provided.

Primary strengths: This measure has been successful used to study reactions to various groups of people. It would be an interesting starting point for discussion of cultural or gender difference.

Primary weaknesses: The instrument seems better suited to research than to classroom use.

Overall adequacy: The semantic differential is very thorough and the three subscales would provide for interesting discussion.



Additional References Gundersen, D.F. (1989, April). An application of the SEI short form to public speaking events. Paper presented at the annual

meeting of the Southern States Communication Association, Louisville. (ERIC Document No. 306617)Gundersen, D. F., & Perrill, N. P. (1989). Extending the ‘speech evaluation instrument’ to public speaking settings. Journal of

Language and Social Psychology. 8 , 59-61. Zahn, Christopher J. (1989). Some data on the predictive validity of the Speech Evaluation Instrument. Communication Re-

search Reports. 6 , 53-58. Zahn, C.J. (1989). The bases for differing evaluations of male and female speech. Evidence from ratings of transcribed com-

munication. Communication Monographs, 56:1, 59-74.




Additional References Gundersen, D.F. (1989, April). An application of the SEI short form to public speaking events. Paper presented at the annual

meeting of the Southern States Communication Association, Louisville. (ERIC Document No. 306617)Gundersen, D. F., & Perrill, N. P. (1989). Extending the ‘speech evaluation instrument’ to public speaking settings. Journal of

Language and Social Psychology. 8 , 59-61. Zahn, Christopher J. (1989). Some data on the predictive validity of the Speech Evaluation Instrument. Communication Re-

search Reports. 6 , 53-58. Zahn, C.J. (1989). The bases for differing evaluations of male and female speech. Evidence from ratings of transcribed com-

munication. Communication Monographs, 56:1, 59-74.





Author / Developers: Joe Ayres and Debbie Ayres Sonandré

Address: Joe Ayres, Emeritus Professor, Washington State University, Edward R. Murrow School of Communica-tion, 101 Communication Addition, P.O. Box 642520, Pullman, WA 99164-0000. Phone: 509-335-1556. E-mail: [email protected].

Debbie Ayres Sonandré, Tacoma Community College, Speech Department, 6501 South 19th Street, Tacoma, WA 98466. Phone: 253-566-5241. E-mail: [email protected]

Source: Ayres, J., & Sonandré, D. A. (2002). The Stroop Test for public speaking apprehension: Validity and reli-ability. Communication Research Reports, 19(2), 167-174.

Intended Context or Behavior: Public-speaking apprehension


Time Required: Information not available.

Description: This instrument is an indirect way to measure fear of public speaking. The Stroop Test involves ask-ing respondents to name colors rather than words appearing on cards. The nature of the words used can facilitate or inhibit the time it takes to name the color. For this study, three Stroop cards were used. Each card displayed fi ve words that appeared 20 times. The words were printed in red, blue, green, yellow, and white on a black background. The fi rst card contained neutral stimulus words (table, shoe, sock, glove, cloth). The second card contained stimulus words for public speaking apprehension (audience, presentation, public, stage, and speech). The third card con-tained fi ve control words matched with speech apprehension words for number of letters and syllables (elephant, subdivisions, nature, clouds, and roads). Respondents named colors from these three cards and their response times were recorded using a stopwatch. At the end of presenting three cards, respondents were shown the public speaking card again. It is predicted that people with high communication apprehension will require more response time in such circumstances than people with low communication apprehension.

Scoring Description: Scoring is based on the length of time required to name colors rather than words appearing on cards. The Stroop Test is associated with delayed responses among “high apprehensives” in earlier tests. However directions for scoring are not provided.

Norm/Criterion Reference Populations: This instrument is based on a population of college students. There is no information about the range of response times that can be used as a basis for comparison through norms. There is no information about specifi c criteria that respondents are expected to know or be able to do.


Validity

Concurrent: The Stroop Test for Public Speaking Apprehension has a strong positive correlation (r = .84) with the trait measure public speaking sub-scale of the PRCA (Personal Report of Communication Apprehension). The Stroop Test for Public Speaking also has a strong positive correlation (r = .86) with the state measure of the STAI (State-Trait Anxiety Inventory). Therefore the Stroop Test for Public Speaking Apprehension is able to distinguish among groups of people with various communication traits and states and has concurrent validity.

Face/Content: It is not possible to directly look at the Stroop Test for Public Speaking Apprehension on its face. This is an indirect test of apprehension so it is diffi cult to evaluate if the measure accurately translates the construct of public speaking apprehension and determine face validity In terms of content validity, there is limited description of the research and theory contributing to the Stroop Test for Public Speaking Apprehension. Most details are not provided in published accounts and therefore there is no detailed description of the content domain.




Author / Developers: Joe Ayres and Debbie Ayres Sonandré

Address: Joe Ayres, Emeritus Professor, Washington State University, Edward R. Murrow School of Communica-tion, 101 Communication Addition, P.O. Box 642520, Pullman, WA 99164-0000. Phone: 509-335-1556. E-mail: [email protected].

Debbie Ayres Sonandré, Tacoma Community College, Speech Department, 6501 South 19th Street, Tacoma, WA 98466. Phone: 253-566-5241. E-mail: [email protected]

Source: Ayres, J., & Sonandré, D. A. (2002). The Stroop Test for public speaking apprehension: Validity and reli-ability. Communication Research Reports, 19(2), 167-174.

Intended Context or Behavior: Public-speaking apprehension


Time Required: Information not available.

Description: This instrument is an indirect way to measure fear of public speaking. The Stroop Test involves ask-ing respondents to name colors rather than words appearing on cards. The nature of the words used can facilitate or inhibit the time it takes to name the color. For this study, three Stroop cards were used. Each card displayed fi ve words that appeared 20 times. The words were printed in red, blue, green, yellow, and white on a black background. The fi rst card contained neutral stimulus words (table, shoe, sock, glove, cloth). The second card contained stimulus words for public speaking apprehension (audience, presentation, public, stage, and speech). The third card con-tained fi ve control words matched with speech apprehension words for number of letters and syllables (elephant, subdivisions, nature, clouds, and roads). Respondents named colors from these three cards and their response times were recorded using a stopwatch. At the end of presenting three cards, respondents were shown the public speaking card again. It is predicted that people with high communication apprehension will require more response time in such circumstances than people with low communication apprehension.

Scoring Description: Scoring is based on the length of time required to name colors rather than words appearing on cards. The Stroop Test is associated with delayed responses among “high apprehensives” in earlier tests. However directions for scoring are not provided.

Norm/Criterion Reference Populations: This instrument is based on a population of college students. There is no information about the range of response times that can be used as a basis for comparison through norms. There is no information about specifi c criteria that respondents are expected to know or be able to do.


Validity

Concurrent: The Stroop Test for Public Speaking Apprehension has a strong positive correlation (r = .84) with the trait measure public speaking sub-scale of the PRCA (Personal Report of Communication Apprehension). The Stroop Test for Public Speaking also has a strong positive correlation (r = .86) with the state measure of the STAI (State-Trait Anxiety Inventory). Therefore the Stroop Test for Public Speaking Apprehension is able to distinguish among groups of people with various communication traits and states and has concurrent validity.

Face/Content: It is not possible to directly look at the Stroop Test for Public Speaking Apprehension on its face. This is an indirect test of apprehension so it is diffi cult to evaluate if the measure accurately translates the construct of public speaking apprehension and determine face validity In terms of content validity, there is limited description of the research and theory contributing to the Stroop Test for Public Speaking Apprehension. Most details are not provided in published accounts and therefore there is no detailed description of the content domain.



Construct: The Stroop Test correlated positively as expected with the PRCA and STAI (r = .84 and .86 respec-tively). The Stroop Test had low correlations with measures of imagery control (r = .-04) and dogmatism (r = .11). “Since the Stroop Test for Public Speaking Apprehension shares substantial variance with other measures of public speaking apprehension and little variance with measures of imagery control and dogmatism, it appears to demon-strate a reasonable level of construct validity” (Ayres & Sonandré, 2002, p. 172).


Reliability


Test/Retest: Respondents’ reaction times to the fi rst and second administrations of the Stroop Test for Public Speak-ing Apprehension were correlated. A test-retest correlation of .94 was reported.

Internal Consistency: Information not available.


Practicality and Usefulness: The Stroop Test for Public Speaking Apprehension is useful as an indirect measure of communication apprehension. Self-report measures, such as the PRCA, might allow respondents to distort their answers. There are circumstances in which respondents might want to appear more or less apprehensive than they are. They might want to qualify for a study to receive extra credit or gain admission to a public speaking course that is perceived as “easier” than a traditional course. Directors of such public speaking “special sections” might fi nd the Stroop Test useful to test students in a non-obvious way in order to limit enrollment to those most in need of assistance.

Clarity of Instructions: The instructions for conducting the Stroop Test are clear. However, scoring instructions for determining high and low apprehensive individuals from timed responses are not provided.

Standardization in Administration and Evaluation: Administration of the general Stroop Test appears standard-ized. However more details on administering the Stroop Test for Public Speaking Apprehension are needed. Stan-dardized administration and evaluation procedures are not included. There is no description of how this instrument was used for evaluation of students.

Range of Reponses or Degrees of Competence: This test does not provide a range of responses that constitute various degrees of competence. There is no description of how to determine anchor points of high, middle, or low scores.

Primary Strengths: This tool is a creative, indirect way to measure public speaking apprehension. Although the time required to administer the Stroop Test is not given, it appears to be quick and easy to administer since respon-dents answer within seconds of exposure to each card.

Primary Weaknesses: Students who are colorblind cannot participate in the Stroop Test. Only fi ve words associ-ated with public speaking were used in the current study.

Overall Adequacy: If there is a concern with respondent awareness interfering with the assessment of public speak-ing apprehension, then the Stroop Test is a relatively indirect, reliable, and valid measure. To use the Stroop Test on a larger scale, more details need to be provided on how to administer the test, how to score students, and how these scores will be used in the evaluation of students.

Additional References Mandeville, M. Y., Ries, B. J., Turk, C. L., McChargue, D. E., & McNeil, D. W. (1994, November). Using a unique cognitive

approach – a variation of the Stroop Test – in the identifi cation and the measurement of public speaking anxiety. Paper pre-sented at the meeting of the Speech Communication Association, New Orleans, LA.

McNeil, D. W., Ries, B. J., Boone, M. L., Turk, C. L., Carter, L., Taylor, L. J., & Lewin, M. R. (1995). Comparison of social phobia subtypes using Stroop tests. Journal of Anxiety Disorders, 9(1), 47-57.




Construct: The Stroop Test correlated positively as expected with the PRCA and STAI (r = .84 and .86 respec-tively). The Stroop Test had low correlations with measures of imagery control (r = .-04) and dogmatism (r = .11). “Since the Stroop Test for Public Speaking Apprehension shares substantial variance with other measures of public speaking apprehension and little variance with measures of imagery control and dogmatism, it appears to demon-strate a reasonable level of construct validity” (Ayres & Sonandré, 2002, p. 172).


Reliability


Test/Retest: Respondents’ reaction times to the fi rst and second administrations of the Stroop Test for Public Speak-ing Apprehension were correlated. A test-retest correlation of .94 was reported.

Internal Consistency: Information not available.


Practicality and Usefulness: The Stroop Test for Public Speaking Apprehension is useful as an indirect measure of communication apprehension. Self-report measures, such as the PRCA, might allow respondents to distort their answers. There are circumstances in which respondents might want to appear more or less apprehensive than they are. They might want to qualify for a study to receive extra credit or gain admission to a public speaking course that is perceived as “easier” than a traditional course. Directors of such public speaking “special sections” might fi nd the Stroop Test useful to test students in a non-obvious way in order to limit enrollment to those most in need of assistance.

Clarity of Instructions: The instructions for conducting the Stroop Test are clear. However, scoring instructions for determining high and low apprehensive individuals from timed responses are not provided.

Standardization in Administration and Evaluation: Administration of the general Stroop Test appears standard-ized. However more details on administering the Stroop Test for Public Speaking Apprehension are needed. Stan-dardized administration and evaluation procedures are not included. There is no description of how this instrument was used for evaluation of students.

Range of Reponses or Degrees of Competence: This test does not provide a range of responses that constitute various degrees of competence. There is no description of how to determine anchor points of high, middle, or low scores.

Primary Strengths: This tool is a creative, indirect way to measure public speaking apprehension. Although the time required to administer the Stroop Test is not given, it appears to be quick and easy to administer since respon-dents answer within seconds of exposure to each card.

Primary Weaknesses: Students who are colorblind cannot participate in the Stroop Test. Only fi ve words associ-ated with public speaking were used in the current study.

Overall Adequacy: If there is a concern with respondent awareness interfering with the assessment of public speak-ing apprehension, then the Stroop Test is a relatively indirect, reliable, and valid measure. To use the Stroop Test on a larger scale, more details need to be provided on how to administer the test, how to score students, and how these scores will be used in the evaluation of students.

Additional References Mandeville, M. Y., Ries, B. J., Turk, C. L., McChargue, D. E., & McNeil, D. W. (1994, November). Using a unique cognitive

approach – a variation of the Stroop Test – in the identifi cation and the measurement of public speaking anxiety. Paper pre-sented at the meeting of the Speech Communication Association, New Orleans, LA.

McNeil, D. W., Ries, B. J., Boone, M. L., Turk, C. L., Carter, L., Taylor, L. J., & Lewin, M. R. (1995). Comparison of social phobia subtypes using Stroop tests. Journal of Anxiety Disorders, 9(1), 47-57.




Teacher Clarity Short Inventory (TCSI)

Author / Developers: Joseph L. Chesebro and James C. McCroskey

Address: Joseph L. Chesebro, State University of New York at Brockport, Department of Communication, 350 New Campus Drive, Brockport, NY 14420. E-mail: [email protected]

Source: Chesebro, J. L., & McCroskey, J. C. (1998). The development of the Teacher Clarity Short Inventory (TCSI) to measure clear teaching in the classroom. Communication Research Reports, 15(3), 262-266.

Intended Context or Behavior: Instructional Communication

Intended Academic Level: College (could be used in secondary education classrooms)


Text Description: This 10-item instrument measures a teacher’s clarity of communication regarding course con-tent and processes as evaluated by students. The scale measures the degree to which teachers are able to effectively stimulate the desired meanings of course content and classroom processes through verbal and nonverbal messages.

Scoring Description: Respondents rate on a fi ve-point Likert-type scale how true the statements are for students concerning their teacher’s clarity. A score of fi ve indicates “Strongly Agree” and a score of one indicates “Strongly Disagree.” Six items are positive statements and four items are negative statements. Scoring instructions for respon-dents are not provided in the article. Directions for totaling the scores are also not provided, other than the four negative items should be reverse coded. It is assumed that the scores are summed.

Norm/Criterion Reference Populations: This instrument is based on the population of college students in the United States and China. There is no information about the range of respondent scores that can be used as a basis for comparison through norms. While the items list instructional objectives or standards, there is no information about specifi c criteria that respondents should use to evaluate what teachers are expected to know or be able to do. In other words, there is not a percentage of items that indicate a particular pass/fail performance level for teachers.


Validity


Face/Content: Looking at the TCSI on its face, the items appear to operationalize the construct of teacher clarity in terms of course content presentation and communicating classroom processes. The article provides a detailed description of how to defi ne the content of teacher clarity and how the construct was operationalized in earlier measures.

Construct: Since teacher clarity has been operationalized and research from various perspectives in other studies, more research is needed to show that the TCSI operationalization accurately refl ects the construct of teacher clarity.

Predictive: Teacher clarity is positively related to increased student state motivation to learn, positive effect, and perceived cognitive learning (Chesebro & McCroskey, 2001; Zhang & Zhang, 2005). Teacher clarity is also posi-tively related to attributional confi dence in which students report confi dence in predicting teacher behavior (Avtgis, 2001). However while strong correlations have been documented, causation has not yet been established. The TCSI has initial predictive validity.

Reliability

Alternate Form: The TCSI was “translated into Chinese and back translated into English by different bilingual scholars to ensure conceptual equivalence. The questionnaire was administered in Chinese to students during class”



Teacher Clarity Short Inventory (TCSI)

Author / Developers: Joseph L. Chesebro and James C. McCroskey

Address: Joseph L. Chesebro, State University of New York at Brockport, Department of Communication, 350 New Campus Drive, Brockport, NY 14420. E-mail: [email protected]

Source: Chesebro, J. L., & McCroskey, J. C. (1998). The development of the Teacher Clarity Short Inventory (TCSI) to measure clear teaching in the classroom. Communication Research Reports, 15(3), 262-266.

Intended Context or Behavior: Instructional Communication

Intended Academic Level: College (could be used in secondary education classrooms)


Text Description: This 10-item instrument measures a teacher’s clarity of communication regarding course con-tent and processes as evaluated by students. The scale measures the degree to which teachers are able to effectively stimulate the desired meanings of course content and classroom processes through verbal and nonverbal messages.

Scoring Description: Respondents rate on a fi ve-point Likert-type scale how true the statements are for students concerning their teacher’s clarity. A score of fi ve indicates “Strongly Agree” and a score of one indicates “Strongly Disagree.” Six items are positive statements and four items are negative statements. Scoring instructions for respon-dents are not provided in the article. Directions for totaling the scores are also not provided, other than the four negative items should be reverse coded. It is assumed that the scores are summed.

Norm/Criterion Reference Populations: This instrument is based on the population of college students in the United States and China. There is no information about the range of respondent scores that can be used as a basis for comparison through norms. While the items list instructional objectives or standards, there is no information about specifi c criteria that respondents should use to evaluate what teachers are expected to know or be able to do. In other words, there is not a percentage of items that indicate a particular pass/fail performance level for teachers.


Validity


Face/Content: Looking at the TCSI on its face, the items appear to operationalize the construct of teacher clarity in terms of course content presentation and communicating classroom processes. The article provides a detailed description of how to defi ne the content of teacher clarity and how the construct was operationalized in earlier measures.

Construct: Since teacher clarity has been operationalized and research from various perspectives in other studies, more research is needed to show that the TCSI operationalization accurately refl ects the construct of teacher clarity.

Predictive: Teacher clarity is positively related to increased student state motivation to learn, positive effect, and perceived cognitive learning (Chesebro & McCroskey, 2001; Zhang & Zhang, 2005). Teacher clarity is also posi-tively related to attributional confi dence in which students report confi dence in predicting teacher behavior (Avtgis, 2001). However while strong correlations have been documented, causation has not yet been established. The TCSI has initial predictive validity.

Reliability

Alternate Form: The TCSI was “translated into Chinese and back translated into English by different bilingual scholars to ensure conceptual equivalence. The questionnaire was administered in Chinese to students during class”



(Zhang & Zhang, 2005, p. 260). The Chinese version of the TCSI did have a lower alpha reliability (.84) than previ-ous studies that used the English version (.92).


Internal Consistency: Cronbach’s alpha is reported as .92 in various studies (except for the study using a Chinese version that scored an alpha of .84).


Practicality and Usefulness: The TCSI has been administered in college classrooms and would be useful in assess-ing teacher effectiveness. However at this point, it has been used mainly for instructional research purposes rather than practical application of teaching evaluation.

Clarity of Instructions: The 10 items of the TCSI are reprinted in the article. However no instructions for scoring are provided. In later studies, students are asked to respond to statements in reference to the teacher of the class they immediately attended before completing the scale.

Standardization in Administration and Evaluation: Details on how the TCSI was administered to students was not provided among four research articles. Therefore it is diffi cult to determine standardization of administration. There is no description of how this instrument was used for evaluation of teachers.


Primary Strengths: This measure is designed to be easy to complete and take a minimal amount of time to admin-ister.

Primary Weaknesses: The TCSI has yet to be used for teacher evaluations. It could contribute to the assessment of instructional communication. At this point, it is still primarily a tool for measuring the construct of teacher clarity.

Overall Adequacy: This measure needs to be used in assessing teacher effectiveness. What is known about the TCSI is based on students who are not evaluating the same teachers. More experimental research is needed to test the validity and reliability of this instrument.

Additional ReferencesAvtgis, T. A. (2001). Affective learning, teacher clarity, and student motivation as a function of attributional confi dence. Com-

munication Research Reports, 18(4), 345-353.Chesebro, J. L., & McCroskey, J. C. (2001). The relationship of teacher clarity and immediacy with student state receiver ap-

prehension, affect, and cognitive learning.Communication Education, 50(1), 59-68.Zhang, Q., & Zhang, J. (2005). Teacher clarity: Effects on classroom communication apprehension, student motivation, and

learning in Chinese college classrooms. Journal of Intercultural Communication Research, 34(4), 255-266.




(Zhang & Zhang, 2005, p. 260). The Chinese version of the TCSI did have a lower alpha reliability (.84) than previ-ous studies that used the English version (.92).


Internal Consistency: Cronbach’s alpha is reported as .92 in various studies (except for the study using a Chinese version that scored an alpha of .84).


Practicality and Usefulness: The TCSI has been administered in college classrooms and would be useful in assess-ing teacher effectiveness. However at this point, it has been used mainly for instructional research purposes rather than practical application of teaching evaluation.

Clarity of Instructions: The 10 items of the TCSI are reprinted in the article. However no instructions for scoring are provided. In later studies, students are asked to respond to statements in reference to the teacher of the class they immediately attended before completing the scale.

Standardization in Administration and Evaluation: Details on how the TCSI was administered to students was not provided among four research articles. Therefore it is diffi cult to determine standardization of administration. There is no description of how this instrument was used for evaluation of teachers.


Primary Strengths: This measure is designed to be easy to complete and take a minimal amount of time to admin-ister.

Primary Weaknesses: The TCSI has yet to be used for teacher evaluations. It could contribute to the assessment of instructional communication. At this point, it is still primarily a tool for measuring the construct of teacher clarity.

Overall Adequacy: This measure needs to be used in assessing teacher effectiveness. What is known about the TCSI is based on students who are not evaluating the same teachers. More experimental research is needed to test the validity and reliability of this instrument.

Additional ReferencesAvtgis, T. A. (2001). Affective learning, teacher clarity, and student motivation as a function of attributional confi dence. Com-

munication Research Reports, 18(4), 345-353.Chesebro, J. L., & McCroskey, J. C. (2001). The relationship of teacher clarity and immediacy with student state receiver ap-

prehension, affect, and cognitive learning.Communication Education, 50(1), 59-68.Zhang, Q., & Zhang, J. (2005). Teacher clarity: Effects on classroom communication apprehension, student motivation, and

learning in Chinese college classrooms. Journal of Intercultural Communication Research, 34(4), 255-266.




Verbal Aggressiveness Scale

Author / Developers: D.A. Infante and C.J. Wigley III


Source: Infante, D.A. & Wigley, C.J. (1986). Verbal aggressiveness: An interpersonal model and measure. Com-munication Monographs, 53, 61-69.

Context or communication behavior for which intended: Measures aggressiveness during interpersonal interac-tion.



Description:Verbal aggressiveness was defi ned as a personality trait that predisposes

an individual to attack the self-concept of another. The 20 statements on this self-report ask students to rate their behaviors as they try to infl uence others. Ten are worded positively and ten are worked negatively. Students respond on a fi ve-point Likert-type scale.

Description of scoring: The negative item scores are reversed and the twenty items are totaled. A mean of 49.10 was reported for the referencing population. The mean for males was 51.97. For females it was 46.38. Scores of 59-100 are considered high verbal aggressiveness; 39-50 scores are moderate; 20-38 are low.

Norm referencing/criterion referencing populations: Undergraduate students (n=427) were used for referencing this instrument.

Tests for cultural/gender bias: A signifi cant differences was discovered between the scores of men and women.

Validity

Predictive: Verbal aggressiveness correlated with a likeliness to engage in aggressive behaviors.

Concurrent: This measure correlated in the predicted direction with several other related measures of hostility, desirability, adequacy, cognitive complexity and apprehension.

Face/Content: Factor analyses and other development procedures are explained in the article.

Construct: Aggressiveness was found to be distinct from argumentativeness. Recent research has questioned whether aggressiveness is unidimensional or bidimensional.

Reliability

Test/retest: A high correlation (r=.82, p<.001) was reported after a four week test-retest study.

Scoring: An alpha coeffi cient of .81 was reported.


Practicality and usefulness: This instrument might be effective as a way of measuring the success of study in ar-gumentation and persuasion particularly when coupled with the Argumentativeness Scale.

Clarity of instructions: The instructions were clear.

Degree of standardization in administration and evaluation of responses: Directions of administrating and scor-ing the instrument are very clear.

Identifi cation of a range of responses or degrees of competence: Ranges are available.



Verbal Aggressiveness Scale

Author / Developers: D.A. Infante and C.J. Wigley III


Source: Infante, D.A. & Wigley, C.J. (1986). Verbal aggressiveness: An interpersonal model and measure. Com-munication Monographs, 53, 61-69.

Context or communication behavior for which intended: Measures aggressiveness during interpersonal interac-tion.



Description:Verbal aggressiveness was defi ned as a personality trait that predisposes

an individual to attack the self-concept of another. The 20 statements on this self-report ask students to rate their behaviors as they try to infl uence others. Ten are worded positively and ten are worked negatively. Students respond on a fi ve-point Likert-type scale.

Description of scoring: The negative item scores are reversed and the twenty items are totaled. A mean of 49.10 was reported for the referencing population. The mean for males was 51.97. For females it was 46.38. Scores of 59-100 are considered high verbal aggressiveness; 39-50 scores are moderate; 20-38 are low.

Norm referencing/criterion referencing populations: Undergraduate students (n=427) were used for referencing this instrument.

Tests for cultural/gender bias: A signifi cant differences was discovered between the scores of men and women.

Validity

Predictive: Verbal aggressiveness correlated with a likeliness to engage in aggressive behaviors.

Concurrent: This measure correlated in the predicted direction with several other related measures of hostility, desirability, adequacy, cognitive complexity and apprehension.

Face/Content: Factor analyses and other development procedures are explained in the article.

Construct: Aggressiveness was found to be distinct from argumentativeness. Recent research has questioned whether aggressiveness is unidimensional or bidimensional.

Reliability

Test/retest: A high correlation (r=.82, p<.001) was reported after a four week test-retest study.

Scoring: An alpha coeffi cient of .81 was reported.


Practicality and usefulness: This instrument might be effective as a way of measuring the success of study in ar-gumentation and persuasion particularly when coupled with the Argumentativeness Scale.

Clarity of instructions: The instructions were clear.

Degree of standardization in administration and evaluation of responses: Directions of administrating and scor-ing the instrument are very clear.

Identifi cation of a range of responses or degrees of competence: Ranges are available.



Primary strengths: This instrument would be an effective way of introducing aggressiveness in discussions of confl ict or argumentation.

Overall adequacy: While the focus for this instrument is limited, it could be used effectively in conjunction with other assessments of interpersonal communication.

Additional ReferencesDeWine, S., Nicotera, A.M. & Parry, D. (1991) argumentativeness and aggressiveness: The fl ip side of gentle persuasion. Man-


198.Infante, D.A. (1988). Arguing Constructively. Prospect Heights, IL: Waveland Press.Levine, T.R. et al (2004) The dimensionality of the verbal aggressiveness scale. Communication Monographs, 71, 245-268.Neer, M.R. (1994). Argumentative fl exibility as a factor infl uencing message response style to argumentative and aggressive







Primary strengths: This instrument would be an effective way of introducing aggressiveness in discussions of confl ict or argumentation.

Overall adequacy: While the focus for this instrument is limited, it could be used effectively in conjunction with other assessments of interpersonal communication.

Additional ReferencesDeWine, S., Nicotera, A.M. & Parry, D. (1991) argumentativeness and aggressiveness: The fl ip side of gentle persuasion. Man-


198.Infante, D.A. (1988). Arguing Constructively. Prospect Heights, IL: Waveland Press.Levine, T.R. et al (2004) The dimensionality of the verbal aggressiveness scale. Communication Monographs, 71, 245-268.Neer, M.R. (1994). Argumentative fl exibility as a factor infl uencing message response style to argumentative and aggressive







Watson-Barker Revised Listening Test Forms C & D

Author / Developers: Kittie W. Watson, Larry L. Barker, Charles V. Roberts, & John D. Roberts

Source: Innolect, Inc. 31012 Executive Point, Tega Cay, SC 29708 www.innolectinc.com

Intended behavior: Listening


Time required: 45 minutes to administer test; recommend 30 minutes for debriefi ng

Description: “The revised video WBLT is divided into fi ve parts: (1) evaluating message content, (2) understand-ing meaning in conversations, (3) understanding and remembering information in lectures, (4) evaluating emotional meanings in messages and (5) following instructions and directions. PARTS I, II, and IV are designed to measure the listening skills used in brief listening situations. PARTS III and V are designed to measure the listening skills used in longer listening situations. “A video accompanies this measure. All listening situations and the accompanying questions are presented orally and visually on the screen. An self-scoring answer sheet is available for responses.

Description of administration and scoring: Directions are provided for respondents to determine and interpret their own scores.

Norm referencing/criterion referencing populations: Over 65,000 undergraduates, managers, supervisors, ad-ministrators and professionals have completed the measure.

Cultural and gender bias: Speakers featured in the video tape come from a variety of ethnic backgrounds. Both men and women are included.

Reliability

Alternate forms: Signifi cant positive relationships are found between Versions C and D.

Validity

Content: Listening experts reviewed items.

Concurrent: Scores on this test are correlated in expected ways with the Receiver Apprehension test. “Applegate and Campbell (1985) conducted a correlation analysis of overall and sub-test scores of the WBLT and KCLT [Kentucky Comprehensive Listening Test]. Signifi cant correlations were found between various sections of each test; however, the shared variance was surprisingly low.


Practicality and usefulness: The instrument is easy to administer. It focuses on an important communication skill.


Degree of standardization in administration and evaluation of responses: Directions for administering and evaluating are clear enough to lead to standardized usage.

Identifi cation of a range of responses or degree of competence: Normative responses are available for compari-son..

Primary strengths: Focuses on an under studied aspect of communication.

Overall adequacy: This instrument is being successfully used in a variety of educational contexts. Samford Univer-sity and the University of Wisconsin-Stevens Point regularly use the WBLT in their assessment programs.



Watson-Barker Revised Listening Test Forms C & D

Author / Developers: Kittie W. Watson, Larry L. Barker, Charles V. Roberts, & John D. Roberts

Source: Innolect, Inc. 31012 Executive Point, Tega Cay, SC 29708 www.innolectinc.com

Intended behavior: Listening


Time required: 45 minutes to administer test; recommend 30 minutes for debriefi ng

Description: “The revised video WBLT is divided into fi ve parts: (1) evaluating message content, (2) understand-ing meaning in conversations, (3) understanding and remembering information in lectures, (4) evaluating emotional meanings in messages and (5) following instructions and directions. PARTS I, II, and IV are designed to measure the listening skills used in brief listening situations. PARTS III and V are designed to measure the listening skills used in longer listening situations. “A video accompanies this measure. All listening situations and the accompanying questions are presented orally and visually on the screen. An self-scoring answer sheet is available for responses.

Description of administration and scoring: Directions are provided for respondents to determine and interpret their own scores.

Norm referencing/criterion referencing populations: Over 65,000 undergraduates, managers, supervisors, ad-ministrators and professionals have completed the measure.

Cultural and gender bias: Speakers featured in the video tape come from a variety of ethnic backgrounds. Both men and women are included.

Reliability

Alternate forms: Signifi cant positive relationships are found between Versions C and D.

Validity

Content: Listening experts reviewed items.

Concurrent: Scores on this test are correlated in expected ways with the Receiver Apprehension test. “Applegate and Campbell (1985) conducted a correlation analysis of overall and sub-test scores of the WBLT and KCLT [Kentucky Comprehensive Listening Test]. Signifi cant correlations were found between various sections of each test; however, the shared variance was surprisingly low.


Practicality and usefulness: The instrument is easy to administer. It focuses on an important communication skill.


Degree of standardization in administration and evaluation of responses: Directions for administering and evaluating are clear enough to lead to standardized usage.

Identifi cation of a range of responses or degree of competence: Normative responses are available for compari-son..

Primary strengths: Focuses on an under studied aspect of communication.

Overall adequacy: This instrument is being successfully used in a variety of educational contexts. Samford Univer-sity and the University of Wisconsin-Stevens Point regularly use the WBLT in their assessment programs.



Additional ReferencesApplegate, J.S. & Campbell, J.K. (1985). A correlational analysis of overall and sub-test scores between the Watson/Barker and

the Kentucky comprehensive listening tests. Paper presented at the International Listening Association Convention, Orlando, FL.

Bommelje, R., Houston, J.M..& Smither, R. (2003) Personality characteristics of effective listeners: a fi ve factor perspective. International Journal of Listening 17, 32-46.

Means, T.L. & Klein, G.S. (1994) A short classroom unit, but a signifi cant improvement in listening ability. Bulletin of the As-sociation for Business Communication, 57:4, 13-18.

Roberts, C.V. (1988). The validation of listening tests: Cutting the Gordian knot. Journal of the International Listening Associa-tion, 2, 1-19.

Roberts, C.V. (1986). A validation of the Watson-Barker Listening Test. Communication Research Reports, 3, 115-19.Thomas, L.T. (1994) Disentangling listening and verbal recall. Human Communication Research, 21: 1, 103-129.

Reviewer: Ellen A. Hay and Judith Dallinger



Additional ReferencesApplegate, J.S. & Campbell, J.K. (1985). A correlational analysis of overall and sub-test scores between the Watson/Barker and

the Kentucky comprehensive listening tests. Paper presented at the International Listening Association Convention, Orlando, FL.

Bommelje, R., Houston, J.M..& Smither, R. (2003) Personality characteristics of effective listeners: a fi ve factor perspective. International Journal of Listening 17, 32-46.

Means, T.L. & Klein, G.S. (1994) A short classroom unit, but a signifi cant improvement in listening ability. Bulletin of the As-sociation for Business Communication, 57:4, 13-18.

Roberts, C.V. (1988). The validation of listening tests: Cutting the Gordian knot. Journal of the International Listening Associa-tion, 2, 1-19.

Roberts, C.V. (1986). A validation of the Watson-Barker Listening Test. Communication Research Reports, 3, 115-19.Thomas, L.T. (1994) Disentangling listening and verbal recall. Human Communication Research, 21: 1, 103-129.

Reviewer: Ellen A. Hay and Judith Dallinger



Willingness to Communicate Scale



Source: McCroskey, J.C. (1992). Reliability and validity of the willingness to communicate scale. Communication Quarterly, 40, 16-25. Also available in Assessing Motivation to Communicate, 2nd Edn. Washington, D.C.: National Communication Association

Intended context or behavior: Apprehension-the predisposition to approach or avoid communication situations.

Time requirements: 10 minutes

Description: Subjects respond to 20 items describing various communication situations, indicating the probability of communicating in each situation (100=always;0=never. “Eight of the items are fi llers and 12 are scored as part of the scale. When scored as indicated, it yields a total score, three subscales based on types of receivers (strangers, acquaintances, friends), and four subscores on types of communication contexts (public, meeting, group, dyad)” (p. 17).

Description of scoring: Directions are provided for scoring each of the seven subscales and summing the subscales for a total score. A mean score of 65.2 was reported. One standard deviation above or below the mean has used to identify high and low respondents.

Norm/criterion reference population: College students, pharmacy students and chiropractic assistants.

Culture/gender basis: The scale is highly cultural dependent. Mean overall scores and subscale scores were found for subjects with different cultural backgrounds.

Validity

Content: Assumptions underlying the instrument were examined and discussed in relation to the factor analysis.

Construct: Appropriate correlations were found with related instruments. Willingness emerged as a distinct con-struct from other forms of apprehension and reticence.

Predictive: The scale was found to predict whether or not individuals would participate in various communication situations. Those labeled high in willingness to communicate were signifi cantly more likely to attend scheduled communication events than those labeled low in willingness to communicate.

Reliability

Internal: Total score coeffi cient range from .86-.95. Subscale coeffi cients are lower.

Test-retest: A coeffi cient of .86 was reported.

Evaluative reactions:

Practicality and usefulness: This is a very effi cient way of measuring student response to different communication situations. If communication instruction is designed to encourage and prepare students to participate in interper-sonal, group and public contexts, this measure will monitor the impact of that instruction.

Clarity of instructions: The directions for completing and scoring the instrument are simple to follow.

Standardization in administration and evaluation: Administration is easily standardized.

Range of responses/competency: High/low designations are possible.



Willingness to Communicate Scale



Source: McCroskey, J.C. (1992). Reliability and validity of the willingness to communicate scale. Communication Quarterly, 40, 16-25. Also available in Assessing Motivation to Communicate, 2nd Edn. Washington, D.C.: National Communication Association

Intended context or behavior: Apprehension-the predisposition to approach or avoid communication situations.

Time requirements: 10 minutes

Description: Subjects respond to 20 items describing various communication situations, indicating the probability of communicating in each situation (100=always;0=never. “Eight of the items are fi llers and 12 are scored as part of the scale. When scored as indicated, it yields a total score, three subscales based on types of receivers (strangers, acquaintances, friends), and four subscores on types of communication contexts (public, meeting, group, dyad)” (p. 17).

Description of scoring: Directions are provided for scoring each of the seven subscales and summing the subscales for a total score. A mean score of 65.2 was reported. One standard deviation above or below the mean has used to identify high and low respondents.

Norm/criterion reference population: College students, pharmacy students and chiropractic assistants.

Culture/gender basis: The scale is highly cultural dependent. Mean overall scores and subscale scores were found for subjects with different cultural backgrounds.

Validity

Content: Assumptions underlying the instrument were examined and discussed in relation to the factor analysis.

Construct: Appropriate correlations were found with related instruments. Willingness emerged as a distinct con-struct from other forms of apprehension and reticence.

Predictive: The scale was found to predict whether or not individuals would participate in various communication situations. Those labeled high in willingness to communicate were signifi cantly more likely to attend scheduled communication events than those labeled low in willingness to communicate.

Reliability

Internal: Total score coeffi cient range from .86-.95. Subscale coeffi cients are lower.

Test-retest: A coeffi cient of .86 was reported.

Evaluative reactions:

Practicality and usefulness: This is a very effi cient way of measuring student response to different communication situations. If communication instruction is designed to encourage and prepare students to participate in interper-sonal, group and public contexts, this measure will monitor the impact of that instruction.

Clarity of instructions: The directions for completing and scoring the instrument are simple to follow.

Standardization in administration and evaluation: Administration is easily standardized.

Range of responses/competency: High/low designations are possible.



Primary strengths: This is a thoroughly tested instrument that considers an important affective dimension of com-munication.

Overall adequacy: It has been used on pre/post basis at institutions such as Winthrop University and the University of Colorado at Colorado Springs as a part of their assessment programs.

Additional ReferencesBurroughs, N.F., Marie, V. & McCroskey, J.C. (2003) Relationships of self-perceived communication competence and com-

munication apprehension with willingness to communicate: a comparison with fi rst and second languages in Micronesia. Communication Research Reports, 20, 230-239.

Donovan, L. A.; MacIntyre, P. D. (2004). Age and sex differences in willingness to communicate, communication apprehen-sion, and self-perceived competence. Communication Research Reports, 21, 420-427.

MacIntyre, P.D.(1994). Variables underlying willingness to communicate: A causal analysis. Communication Research Re-ports, 11, 135-142.

MacIntyre, P. D. & Babin, P.A.(1999). Willingness to communicate: Antecedents & consequences. Communication Quar-terly,47, 215-229.

Mottet, T. P., Martin, M. M. & Myers, S. A. (2004). Relationships among perceived instructor verbal approach and avoidance relational strategies and students’ motives for communicating with their instructors. Communication Education, 53 116-122.




Primary strengths: This is a thoroughly tested instrument that considers an important affective dimension of com-munication.

Overall adequacy: It has been used on pre/post basis at institutions such as Winthrop University and the University of Colorado at Colorado Springs as a part of their assessment programs.

Additional ReferencesBurroughs, N.F., Marie, V. & McCroskey, J.C. (2003) Relationships of self-perceived communication competence and com-

munication apprehension with willingness to communicate: a comparison with fi rst and second languages in Micronesia. Communication Research Reports, 20, 230-239.

Donovan, L. A.; MacIntyre, P. D. (2004). Age and sex differences in willingness to communicate, communication apprehen-sion, and self-perceived competence. Communication Research Reports, 21, 420-427.

MacIntyre, P.D.(1994). Variables underlying willingness to communicate: A causal analysis. Communication Research Re-ports, 11, 135-142.

MacIntyre, P. D. & Babin, P.A.(1999). Willingness to communicate: Antecedents & consequences. Communication Quar-terly,47, 215-229.

Mottet, T. P., Martin, M. M. & Myers, S. A. (2004). Relationships among perceived instructor verbal approach and avoidance relational strategies and students’ motives for communicating with their instructors. Communication Education, 53 116-122.





Author / Developers: C.V. Roberts & L. Vinson

Address: C. V. Roberts, East Tennessee State University; L. Vinson, McNeese State University

Source: Roberts, C.V. & Vinson, L. (1998) Relationship among willingness to listen, receiver apprehension, com-munication apprehension, communication competence, and dogmatism. International Journal of Listening. 12, 40-56.

Intended context or behavior: The developers question the external validity of popular listening measures, noting that they measure optimal listening rather than habitual listening. This scale considers an individual’s predisposition to listen in a variety of different contexts.

Intended academic level: WTL appears suitable for high school and college students as well as adults.

Time required: Approximately 15 minutes

Description: “The scale had 36 items that systematically varied the level of acquaintance of the sender (stranger, acquaintance, friend), the physical location of the communication (school, work, other interpersonal), the level of the communication context (dyadic, small group, public speaking), and whether the communication was face-to-face or mechanically mediated (e.g. television, telephone)” (p. 50).

Scoring description: Respondents indicate the percent of time they would choose to listen in the given situation (0=never, to 100=always). Means and ranges are not given.


Culture and gender bias: No specifi c testing for bias.

Validity

Face/content: The WTL is constructed in a similar manner to the McCroskey and Richmond’s (1985) Willingness to Communicate Scale. In this most recent version, the importance of the content of the message has been removed from consideration. This variable overpowered other considerations in the initial version of the WTL.

Construct: In previous research, Roberts and Vinson (1989) have established that Willingness to Listen is a sepa-rate construct from Willingness to Communicate. The predisposition toward sending messages is different from that of receiving messages.

Predictive: This article reports correlations between the WTL scale and measures of receiver apprehension, self-perceived communication competence and communication apprehension. The authors concluded, “Willingness to listen is positively correlated with communication skills and negatively related to receiver apprehension, sender-based communication apprehension, and dogmatism” (p.53).

Reliability

Internal consistency: An Alpha level of .88 was reported for the optimal items of the WTL.


Practicality and usefulness: The authors report that they will next study whether this particular trait is malleable. If so, this instrument would prove valuable for assessment of listening instruction and improvement.

Clarity of instructions: The directions are straightforward and easy to follow.

Standardization in administration and evaluation: Since means and ranges are not provided, comparative scores would need to be established for a given population.




Author / Developers: C.V. Roberts & L. Vinson

Address: C. V. Roberts, East Tennessee State University; L. Vinson, McNeese State University

Source: Roberts, C.V. & Vinson, L. (1998) Relationship among willingness to listen, receiver apprehension, com-munication apprehension, communication competence, and dogmatism. International Journal of Listening. 12, 40-56.

Intended context or behavior: The developers question the external validity of popular listening measures, noting that they measure optimal listening rather than habitual listening. This scale considers an individual’s predisposition to listen in a variety of different contexts.

Intended academic level: WTL appears suitable for high school and college students as well as adults.

Time required: Approximately 15 minutes

Description: “The scale had 36 items that systematically varied the level of acquaintance of the sender (stranger, acquaintance, friend), the physical location of the communication (school, work, other interpersonal), the level of the communication context (dyadic, small group, public speaking), and whether the communication was face-to-face or mechanically mediated (e.g. television, telephone)” (p. 50).

Scoring description: Respondents indicate the percent of time they would choose to listen in the given situation (0=never, to 100=always). Means and ranges are not given.


Culture and gender bias: No specifi c testing for bias.

Validity

Face/content: The WTL is constructed in a similar manner to the McCroskey and Richmond’s (1985) Willingness to Communicate Scale. In this most recent version, the importance of the content of the message has been removed from consideration. This variable overpowered other considerations in the initial version of the WTL.

Construct: In previous research, Roberts and Vinson (1989) have established that Willingness to Listen is a sepa-rate construct from Willingness to Communicate. The predisposition toward sending messages is different from that of receiving messages.

Predictive: This article reports correlations between the WTL scale and measures of receiver apprehension, self-perceived communication competence and communication apprehension. The authors concluded, “Willingness to listen is positively correlated with communication skills and negatively related to receiver apprehension, sender-based communication apprehension, and dogmatism” (p.53).

Reliability

Internal consistency: An Alpha level of .88 was reported for the optimal items of the WTL.


Practicality and usefulness: The authors report that they will next study whether this particular trait is malleable. If so, this instrument would prove valuable for assessment of listening instruction and improvement.

Clarity of instructions: The directions are straightforward and easy to follow.

Standardization in administration and evaluation: Since means and ranges are not provided, comparative scores would need to be established for a given population.



Range of responses or degrees of competence: The 100-point scale is adequate to indicate competence.

Primary strengths: While other listening measures assume optimal attention, this instrument taps a unique con-struct. Is the individual predisposed to listening? This instrument might prove helpful in focusing student attention on listening behaviors. It can be quickly administered in a large group or individual setting.

Primary weaknesses: More testing on the instrument is needed.

Overall adequacy: This scale would provide a good introduction to listening instruction and may help to monitor whether listening improves as a result of instruction.




Range of responses or degrees of competence: The 100-point scale is adequate to indicate competence.

Primary strengths: While other listening measures assume optimal attention, this instrument taps a unique con-struct. Is the individual predisposed to listening? This instrument might prove helpful in focusing student attention on listening behaviors. It can be quickly administered in a large group or individual setting.

Primary weaknesses: More testing on the instrument is needed.

Overall adequacy: This scale would provide a good introduction to listening instruction and may help to monitor whether listening improves as a result of instruction.




WorkKeys for Education-Listening

Author / Developers: ACT

Address: ACT, P.O. Box 162, Iowa City, IA 52243


Intended academic level: High school and college students; adults


Cost: $10

Description: This is one of eight WorkKeys assessments that can be used as a screening measure to determine if an individual possesses the skills for a particular job or as a diagnostic measure to ascertain training needs. “The test is administered via an audiotape that contains all directions and messages. Examinees are asked to listen to six audio-taped messages and then compose written messages based on what they have heard. The examinee is placed in the role of an employee who receives information from customers, co-workers, or suppliers, and must then write down the information to communicate it to someone else. This context is provided to help examinees understand that they should include all of the information in their response. The taped messages refl ect various workplace settings, but no prior knowledge of the occupations is necessary. Each message is given twice, and examinees are encouraged to take notes. After the message is given the second time, examinees have a specifi ed amount of time to write out the message before the next message is given.” (www.act.org/workkeys)

Scoring description: “Scoring is based on the accuracy and completeness of the information in the examinee’s written messages. It is not based on mechanics or writing style. Two or more raters read each message and assign it a level score from 0 to 5. Scores are then calculated to indicate the particular level of skill for each examinee. Thus, the examinee’s performance is measured against work-related criteria and not against the performance of other ex-aminees.” (www.act.org/workkeys)

Culture and gender bias: The messages are delivered by both male and female speakers of differing ages and with various accents. The experts who develop items and scoring classifi cations were “a representative sample of the job incumbents in terms of gender, race/ethnicity, age and disability status.” WorkKeys was designed to meet the nondiscriminatory standards outlined in The Uniform Guidelines on Employee Selection Procedures adopted by the United States Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, and the Department of Justice.

Validity

Face/content: Test items and scoring classifi cations are fi rst developed by job profi lers who identify the tasks and skills necessary for various jobs. These analyses are then reviewed with individuals identifi ed as experts in such positions to set appropriate cut-off points for the classifi cations.

Reliability

Internal: A generalizability co-effi cient of .66 was reported. A dependability co-effi cient of .62 was reported.


Practicality and usefulness: WorkKeys is widely used in workforce development programs in many states and lo-cal communities. High schools and community colleges in particular use these assessments to evaluate the listening abilities of graduates. WorkKeys can also be used for self-assessment.

Clarity of instructions: Instructions in the sample items were very clear.

Standardization in administration and evaluation: Responses are scored by trained raters.



WorkKeys for Education-Listening






Cost: $10

Description: This is one of eight WorkKeys assessments that can be used as a screening measure to determine if an individual possesses the skills for a particular job or as a diagnostic measure to ascertain training needs. “The test is administered via an audiotape that contains all directions and messages. Examinees are asked to listen to six audio-taped messages and then compose written messages based on what they have heard. The examinee is placed in the role of an employee who receives information from customers, co-workers, or suppliers, and must then write down the information to communicate it to someone else. This context is provided to help examinees understand that they should include all of the information in their response. The taped messages refl ect various workplace settings, but no prior knowledge of the occupations is necessary. Each message is given twice, and examinees are encouraged to take notes. After the message is given the second time, examinees have a specifi ed amount of time to write out the message before the next message is given.” (www.act.org/workkeys)

Scoring description: “Scoring is based on the accuracy and completeness of the information in the examinee’s written messages. It is not based on mechanics or writing style. Two or more raters read each message and assign it a level score from 0 to 5. Scores are then calculated to indicate the particular level of skill for each examinee. Thus, the examinee’s performance is measured against work-related criteria and not against the performance of other ex-aminees.” (www.act.org/workkeys)

Culture and gender bias: The messages are delivered by both male and female speakers of differing ages and with various accents. The experts who develop items and scoring classifi cations were “a representative sample of the job incumbents in terms of gender, race/ethnicity, age and disability status.” WorkKeys was designed to meet the nondiscriminatory standards outlined in The Uniform Guidelines on Employee Selection Procedures adopted by the United States Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, and the Department of Justice.

Validity


Reliability

Internal: A generalizability co-effi cient of .66 was reported. A dependability co-effi cient of .62 was reported.


Practicality and usefulness: WorkKeys is widely used in workforce development programs in many states and lo-cal communities. High schools and community colleges in particular use these assessments to evaluate the listening abilities of graduates. WorkKeys can also be used for self-assessment.


Standardization in administration and evaluation: Responses are scored by trained raters.



Range of responses or degrees of competence: Messages to which subjects respond become increasingly more complex. Subjects must differentiate main ideas from supporting ones and identify relationships between ideas.

Primary strengths: This is a nationally recognized assessment that clearly identifi es various levels of listening abil-ity. It is quick to administer and since responses are evaluated by ACT, does not require time to evaluate.

Primary weaknesses: It is fairly costly. Developers note that while writing ability is not graded, it may be diffi cult for some individuals to express themselves in written form.

Overall adequacy: This is a viable option for assessing listening, particularly in work related settings.

Additional ReferencesACT White Paper: WorkKeys Validation. (2002) Iowa City, IA: ACT.A Comparison of Career-Related Assessment Tools/Models. Final [Report] (1999).WestEd, San Francisco, CA., 1999 ERIC

Document No ED445313Keller, Bess. (2005) Michigan chooses ACT as high school exam. Education Week, 25:6, 20.Stientjes, M.K. (1998). A study of listening assessment stimuli and response mode effects. International Journal of Listening,

12, 29-39.




Range of responses or degrees of competence: Messages to which subjects respond become increasingly more complex. Subjects must differentiate main ideas from supporting ones and identify relationships between ideas.

Primary strengths: This is a nationally recognized assessment that clearly identifi es various levels of listening abil-ity. It is quick to administer and since responses are evaluated by ACT, does not require time to evaluate.

Primary weaknesses: It is fairly costly. Developers note that while writing ability is not graded, it may be diffi cult for some individuals to express themselves in written form.

Overall adequacy: This is a viable option for assessing listening, particularly in work related settings.

Additional ReferencesACT White Paper: WorkKeys Validation. (2002) Iowa City, IA: ACT.A Comparison of Career-Related Assessment Tools/Models. Final [Report] (1999).WestEd, San Francisco, CA., 1999 ERIC

Document No ED445313Keller, Bess. (2005) Michigan chooses ACT as high school exam. Education Week, 25:6, 20.Stientjes, M.K. (1998). A study of listening assessment stimuli and response mode effects. International Journal of Listening,

12, 29-39.




WorkKeys for Education-Teamwork



Intended context or behavior: Group communication



Cost: $7.50

Description: This is one of eight WorkKeys assessments that can be used as a screening measure to determine if an individual possesses the skills for a particular job or as a diagnostic measure to ascertain training needs. “This assessment is administered by video and contains twelve teamwork scenarios, each accompanied by three multiple-choice items. The scenarios and items are based on actual demands of the workplace. Selections take the form of video presentations of teams in various workplace settings with a variety of problems or requirements. Examinees must identify the most appropriate teamwork responses to specifi c situations.” (www.act.org/workkeys)

Scoring description: “There are four levels of diffi culty. Level 3 is the least complex and Level 6 is the most com-plex. At Level 3, employees have a number of teamwork-related skills already in place. The levels build on each other, each incorporating the skills assessed at the preceding levels. For example, Level 5 includes the skills used at Levels 3, 4, and 5. As the skill levels move toward Level 6, the situations become more complicated. For example, at Level 3 there is a clear goal, team relationships are cordial, and resources are readily available. At Level 6, goals or consequences confl ict, relationships are volatile, and/or resources are limited. The tasks demand more skill as the complexity increases, ranging from goal recognition and problem identifi cation at Level 3 to goal revision and confl ict resolution at Level 6.”

(www.act.org/workkeys)

Culture and gender bias: The experts who develop items and scoring classifi cations were “a representative sample of the job incumbents in terms of gender, race/ethnicity, age and disability status.” WorkKeys was designed to meet the nondiscriminatory standards outlined in The Uniform Guidelines on Employee Selection Procedures adopted by the United States Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, and the Department of Justice.

Validity


Reliability

Test/retest: Exact classifi cation consistency of 46 percent was reported.



Practicality and usefulness: WorkKeys is widely used in workforce development programs in many states and lo-cal communities. High schools and community colleges in particular use these assessments to evaluate the listening abilities of graduates. Workkeys can also be used for self-assessment.




WorkKeys for Education-Teamwork



Intended context or behavior: Group communication



Cost: $7.50

Description: This is one of eight WorkKeys assessments that can be used as a screening measure to determine if an individual possesses the skills for a particular job or as a diagnostic measure to ascertain training needs. “This assessment is administered by video and contains twelve teamwork scenarios, each accompanied by three multiple-choice items. The scenarios and items are based on actual demands of the workplace. Selections take the form of video presentations of teams in various workplace settings with a variety of problems or requirements. Examinees must identify the most appropriate teamwork responses to specifi c situations.” (www.act.org/workkeys)

Scoring description: “There are four levels of diffi culty. Level 3 is the least complex and Level 6 is the most com-plex. At Level 3, employees have a number of teamwork-related skills already in place. The levels build on each other, each incorporating the skills assessed at the preceding levels. For example, Level 5 includes the skills used at Levels 3, 4, and 5. As the skill levels move toward Level 6, the situations become more complicated. For example, at Level 3 there is a clear goal, team relationships are cordial, and resources are readily available. At Level 6, goals or consequences confl ict, relationships are volatile, and/or resources are limited. The tasks demand more skill as the complexity increases, ranging from goal recognition and problem identifi cation at Level 3 to goal revision and confl ict resolution at Level 6.”

(www.act.org/workkeys)

Culture and gender bias: The experts who develop items and scoring classifi cations were “a representative sample of the job incumbents in terms of gender, race/ethnicity, age and disability status.” WorkKeys was designed to meet the nondiscriminatory standards outlined in The Uniform Guidelines on Employee Selection Procedures adopted by the United States Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, and the Department of Justice.

Validity


Reliability

Test/retest: Exact classifi cation consistency of 46 percent was reported.



Practicality and usefulness: WorkKeys is widely used in workforce development programs in many states and lo-cal communities. High schools and community colleges in particular use these assessments to evaluate the listening abilities of graduates. Workkeys can also be used for self-assessment.




Standardization in administration and evaluation: Administration and evaluation of the test are guided by ACT.

Range of responses or degrees of competence: Situations to which individuals respond range from straightfor-ward to complicated and complex.

Primary strengths: The scenarios are interesting, and clearly feature the task and relational behaviors that are needed to effectively communicate in group situations.

Primary weaknesses: This is costly to administer. Since the response is multiple choice, results may be skewed by the reading ability of subjects.

Overall adequacy: It is a good assessment of group behaviors in the work setting. Since there are few standardized measures of group communication abilities, this measure provides a foundation for assessing these abilities.

Additional ReferencesACT White Paper: Workkeys Validation. (2002) Iowa City, IA: ACT.A Comparison of Career-Related Assessment Tools/Models. Final [Report] (1999).WestEd, San Francisco, CA., 1999 ERIC

Document No ED445313Keller, Bess. (2005) Michigan chooses ACT as high school exam. Education Week, 25:6, 20.




Standardization in administration and evaluation: Administration and evaluation of the test are guided by ACT.

Range of responses or degrees of competence: Situations to which individuals respond range from straightfor-ward to complicated and complex.

Primary strengths: The scenarios are interesting, and clearly feature the task and relational behaviors that are needed to effectively communicate in group situations.

Primary weaknesses: This is costly to administer. Since the response is multiple choice, results may be skewed by the reading ability of subjects.

Overall adequacy: It is a good assessment of group behaviors in the work setting. Since there are few standardized measures of group communication abilities, this measure provides a foundation for assessing these abilities.

Additional ReferencesACT White Paper: Workkeys Validation. (2002) Iowa City, IA: ACT.A Comparison of Career-Related Assessment Tools/Models. Final [Report] (1999).WestEd, San Francisco, CA., 1999 ERIC

Document No ED445313Keller, Bess. (2005) Michigan chooses ACT as high school exam. Education Week, 25:6, 20.



NCA00020_Text.indd Sec1:138NCA00020_Text.indd Sec1:138 6/11/07 8:25:41 PM6/11/07 8:25:41 PM NCA00020_Text.indd Sec1:138NCA00020_Text.indd Sec1:138 6/11/07 8:25:41 PM6/11/07 8:25:41 PM

large scale assessment in oral communication

Documents