history, vocabulary, and ethics of assessment by emilio aponte, m.ed

History, Vocabulary, and Ethics of Administering, Scoring and

Interpreting Assessment

By Emilio Aponte, M.Ed. Artist & Guidance Counselor

Professor EDUC 583

EDUC 583:Instruments and Techniques for Diagnosis and Appraisal

Ana G. Mendez University System Feb 2014

History

Norm Referenced Tests Criterion Referenced Tests

Authentic Assessment

History, Vocabulary, and Ethics of Administering, Scoring and Interpreting Assessment

By Emilio Aponte, M.Ed.

Artist & Guidance Counselor | Professor EDUC 583| AGMUS| Feb 2014

Norm Referenced

1920

World War I - IQ Screening

Based on “the Bell Curve”

Use standardized tests

Comparing students to students

Want to create a spread

◦ Item analysis

◦ Distinguish items: High achievers get correct and low achievers get wrong

Used for screening people in and out




Many college entrance exams and nationally used school tests use norm-referenced tests. Their compare individual student performance to the performance of a normative sample. SAT Graduate Record Examination (GRE) Wechsler Intelligence Scale for Children (WISC)

Examples of Norm Referenced




http://en.wikipedia.org/wiki/SAT

http://en.wikipedia.org/wiki/Graduate_Record_Examination

http://en.wikipedia.org/wiki/Wechsler_Intelligence_Scale_for_Children

Criterion Referenced

1920 1970

Criterion Referenced

Testing Began

Specific standards established

Certain information/learning is necessary to continue the next steps of learning.

Students learning is compared to the criteria or standards (NOT to each other)

Assumption: If students do not reach standards, find other means of teaching students




Driving tests. Their goal is to see whether the test taker is skilled enough to be granted a driver's license, not to see whether one test taker is more skilled than another test taker. Citizenship tests. Their goal is to see whether the test taker is sufficiently familiar with the new country's history and government, not to see whether one test taker is more knowledgeable than another test taker.

Examples of Criterion Referenced




Authentic Assessment

1980s

Authentic

Assessment

1920 1970

Not all of what we teach can be assessed by paper and pencil tests nor by multiple choice items

Students need to demonstrate what they learned: performance based (based on constructivist learning theory)

Assessment is different than testing or grading (closer to diagnosis)




Examples of Authentic Assessment

Rubrics (specific criteria—teaching is planned around criteria)

Includes attention to non-academic or difficult to assess

◦ Cooperative learning

◦ Critical thinking skills

◦ Social learning

Differentiates summative vs. formative assessment

Portfolios

More examples here: http://jfmueller.faculty.noctrl.edu/toolbox/examples.htm




http://jfmueller.faculty.noctrl.edu/toolbox/examples.htm

Psychological Assessment

Process

Integration of information

Multiple sources- Psychological tests, personal and medical history, description of current symptoms and problems by either self or others

Collateral information -interviews with other persons about the person being assessed.

All psychologists do some level of assessment when providing services to clients or patients.

May use simple checklists to assess some traits or symptoms, but psychological assessment is a more complex, detailed, in-depth process.




Psychological Assessment

Continuation

Typical types of focus for psychological assessment are to provided a diagnosis, assess level of function or disability, help direct treatment, and assess treatment outcome.

A useful psychological measure must be both valid (i.e., actually measures what it claims to measure) and reliable (i.e., internally consistent or give consistent results over time).

Psychological testing

o Field characterized by the use of samples of behavior in order to infer generalizations about a given individual.

o The technical term for the science behind psychological testing is psychometrics.




• Rubric is a scoring tool for subjective assessments. It is a set of criteria and standards linked to learning objectives that is used to assess a student's performance on papers, projects, essays, and other assignments. Rubrics allow for standardized evaluation according to specified criteria, making grading simpler and more transparent.

Vocabulary

• Criteria is the plural form of Criterion. Criterion validity, in psychometrics, a measure of how well one variable or set of variables predicts an outcome. • Criterion-referenced test, translates a test score into a statement about the behavior to be expected of a person




• Reliability is the overall consistency of a measure. A measure is said to have a high reliability if it produces similar results under consistent conditions. For example, measurements of people’s height and weight are often extremely reliable.

Vocabulary

• Validity is important because it can help determine what types of tests to use, and help to make sure researchers are using methods that are not only ethical, and cost-effective, but also a method that truly measures the idea or construct in question.

• Psychological tests is one of the sources of data used within the process of assessment; usually more than one test is used.




Vocabulary

• By samples of behavior, one means observations over time of an individual performing tasks that have usually been prescribed beforehand. These responses are often compiled into statistical tables that allow the evaluator to compare the behavior of the individual being tested to the responses of a norm group.

• Norm group consists of a carefully constructed sample of the intended testing population which is intended to mirror the demographic composition of that population (e.g., sixth-graders in Massachusetts, or sixth-graders across the United States -- be specific). The performance of individuals in the norm group is the benchmark against which scores on a norm-referenced test are compared.




• Psychometrics is the field of study concerned with the theory and technique of psychological measurement, which includes the measurement of knowledge, abilities, attitudes, personality traits, and educational measurement. The field is primarily concerned with the construction and validation of measurement instruments such as questionnaires, tests, and personality assessments

• Norm-referenced test: An objective test that is standardized on a group of individuals whose performance is evaluated in relation to the performance of others; contrasted with criterion-referenced test. Most standardized achievement tests are referred to as norm-referenced.

Vocabulary




I learned that Assessment is not a test; it is a process that involves the integration of information from different sources. I also learned that World War I and World War II contributed in the development of Assessment. World War I serves as a boon to the testing movement, since thousands of recruits had to be quickly screened for intellectual functioning and emotional fitness. By the time World War II rolled around the psychologist Hathaway and the psychiatrist McKinley collaborated on the development of the Minnesota Multiphasic Personality Inventory (MMPI). I also learned the basic differences between the three types of assessments. The Norm Referenced (NR); the Criterion Referenced (CR); and Authentic Assessment (AA). NR is based on “the Bell curve” or IQ screening, using standardized tests and student to student comparison. CR has a specific standard established; students learning is compared to the standards and in AA students need to demonstrate what they learned e.g. the portfolio used by the Ana G. Mendez University.

What I have learned? Summary




*

http://faculty.mansfield.edu/mlaunius/Psy2201/Powerpoint/Psy%202201%20Ch%202%20Ethics.ppt

Ethics* Ethical, Legal, and Professional Issues in

Assessment

Ethical Issues in Assessment Ethical Codes Standards Ethical Decision Making

Ethical Codes: Highlights of APA and ACA • Choose appropriate instruments • Competence • Cross-cultural sensitivity • Obtain Informed consent • Invasion of privacy • Proper diagnosis • Release of test data • Test administration

o Test security o Test scoring and interpretation




http://faculty.mansfield.edu/mlaunius/Psy2201/Powerpoint/Psy 2201 Ch 2 Ethics.ppt

Ethics Ethical, Legal, and Professional Issues in

Assessment

Standards The following standards speak to the proper administration and interpretation of tests:

• The Standards for the Qualifications of Tests Users (ACA, 2003)

• Code of Fair Testing Practices in Education (JCTP, 2002) (Appendix B)

• Responsibilities of Users of Standardized Tests (RUST) (AACE, 2003)




Standards Standards of the Qualifications of Test Users

• Knowledge of theory relevant to the testing context • Understanding of testing theory, techniques of test construction, and test worthiness • Knowledge of test sampling and statistics • Ability to review and select appropriate tests • Skill in administration and interpretation • Knowledge of the impact of diversity on testing • Responsible use of assessment and evaluation practice.


Assessment





Assessment

Standards Code of Fair Testing Practices in Education

• For education in following areas: admissions, educational assessment, educational diagnosis, and student placement • Provides guidance for developing and selecting appropriate tests, administering and scoring tests, reporting and interpreting test results, and informing test taker (see Appendix B)




Standards Responsibilities of Users of Standardized Tests (RUST)

• Expands on the ACA’s Code of Ethics (1995) and on the Code of Fair Testing Practices in Education • Speaks to qualifications of examiners in following areas: qualifications needed, technical knowledge, test selection, test administration, test scoring, interpreting test results, and communicating test results.


Assessment




Making Ethical Decisions Kitchener’s Moral Model

• Autonomy • Beneficence • Nonmaleficence • Justice or fairness • Fidelity (loyalty)

Corey, Corey, and Callanan’s (2003) Model

1. Identify problem

2. Identify potential issues involved

3. Review relevant ethical guidelines

4. Know relevant laws and regulations

5. Obtain consultation

6. Consider possible courses of action

7. List consequences of various decisions

8. Decide best course of action.


Assessment




Making Ethical Decisions Ethical Decision-Making and Cognitive

Development.

• Cognitive development probably affects

how one makes a decision

• People on a continuum from:

o rigid, non-reflective manner of looking

at decision making, to

o open, deeply reflective decision-

making process

• People can become more relativistic (less

dualistic) over time


Assessment





Assessment

Legal Issues in Assessment Americans with Disabilities Act

Accommodations must be made for individuals with disabilities who take tests for employment Testing must be relevant to job in question

The Buckley Amendment (FERPA) People have right to educational records, including test records

Carl Perkins Act Assures rights of select disenfranchised groups to access vocational assessment, counseling, and placement




Legal Issues in Assessment Civil Rights Acts (1964 and amendments)

Tests used for employment or promotion must be shown to be suitable and valid for the job in question

Freedom of Information Act Allows access to federal records States have adopted similar laws.

Health Insurance Portability and Accountability Act (HIPAA)

Restricts info shared without client consent Allows clients access to their records except for process notes used in counseling


Assessment




Legal Issues in Assessment Privileged communication laws

Licensed professional has right to keep info confidential Jaffee v. Redmond

Section 504 of the Rehabilitation Act Testing for federally funded programs must be a reflection of individual’s ability, not disability.

Individuals with Disabilities Education Improvement Act (IDEIA) and PL94-142

Children 2 to 21 suspected of having disability must be tested at school’s expense Child Study Team: Individuals Education Plan (IEP)


Assessment




Professional Issues in Testing: Associations

Professional Associations ACA

Association for Assessment in Counseling and Education (AACE)

APA Division 5: Evaluation, Measurement, and Statistics

Professional Issues in Testing: Testing as a Holistic Process

Professional Associations A good assessment involves a number of different kinds of instruments Assessment is only a snapshot in time People change over their lifespan


Assessment




Codes and standards help us 1. understand the cultural bias

inherent in tests, 2. know when a test should not be

used because of bias, and 3. know what to do when test does

not predict well for minorities Laws have impacted on assessment

relative to cross-cultural issues There have been great changes made

on the construction of assessment instruments.

Professional Issues in Testing: Cross Cultural Issues in Assessment


Assessment




Department of Defense of U.S. (2005). Dictionary of Military and Associated Terms. Groth, G. (2003). Hand Book of Psychological Assessment. Fourth edition New Jersey Hashway, R. (1998. Assessment and evaluation of developmental learning:qualitative individual assessment and evaluation models. Westpor, CT: Greenwood publisher group. Hearne, J. (2001). Assessment as a Tool for Learning. Article retrieved on October 1, 2009. http://www.newhorizons.org/strategies/assess/hearne.htm The free online Medical dictionary. (2009). The Medical Dictionary. Thesaurus and encyclopedic. Retrieved Oct 1 2009. http://medical-dictionary.com Mansfield University. Ethical, Legal, and Professional Issues in testing. Retrieved on Feb 2014 from http://faculty.mansfield.edu/mlaunius/Psy2201/Powerpoint/Psy%202201%20Ch%202%20Ethics.ppt McDaniel, R. (2009). What is Assessment: Assessing Student Learning. Article retrieved on October 1, 2009 http://educationalissues.suite101.com/article.cfm/what_is_assessment#ixzz0SsRY0rDE Rossi, P; Lipsey, M; and Freedman, H. (2004). EVALUATION: A Systematic Approach. California, Sage Publications, Inc 2004.

References




http://www.newhorizons.org/strategies/assess/hearne.htm

http://medical-dictionary.com/





http://educationalissues.suite101.com/article.cfm/what_is_assessment

http://educationalissues.suite101.com/article.cfm/what_is_assessment

history, vocabulary, and ethics of assessment by emilio aponte, m.ed

Documents