learning objectives and classroom assessment

Western Michigan University, 18 September 2003, Kalamazoo, Michigan

Learning Objectives and Classroom Assessment

Jeff Froyd, Texas A&M University


Workshop Presenter

• Jeff Froyd, Director of Academic Development• Educational Achievement Division, College of

Engineering, Texas A&M University• Project Director, Foundation Coalition


Acknowledgement

• Russ Pimmel, NSF Program Officer• Former Professor of Electrical Engineering at the

University of Alabama• Assembled much of the material for similar

workshops


Objectives: Participants will gain experience in

• Describing the rationale for preparing and using learning objectives in an individual course

• Preparing specifications for high quality learning objectives

• Writing learning objectives for a single course• Preparing specifications for assessment

processes/tools• Generating alternative assessment

processes/tools for a single course• Selecting assessment processes/tools for a single

course


Agenda• Background for learning objectives

• Specifications for learning objectives

• Writing learning objectives

• Background for classroom assessment

• Specifications for assessment processes/tools

• Background on alternatives for assessment processes/tools

• Generating alternatives for assessment processes/tools

• Selecting assessment processes/tools

• Review workshop activities


Agenda• Background for learning objectives• Specifications for learning objectives

• Requirements for specifications• Team exercise: develop specifications for learning objectives• Workshop exercise: improve specifications for learning objectives

• Writing learning objectives• Individual exercise: write objectives for a course or a portion of a

course• Team exercise: review individual objectives• Workshop exercise: develop a list of suggestions for writing

learning objectives• Individual exercises: revise objectives Team exercise: review

individual objectives• Workshop exercise: reflection


Agenda• Background for classroom assessment • Specifications for assessment processes/tools

• Team exercise: develop a set of specifications for assessment processes/tools

• Workshop exercise: improve sets of specifications for assessment processes/tools

• Generating alternatives for assessment processes/tools• Team exercise: select some learning objectives and generate alternative

assessment processes/tools• Background on alternatives for assessment processes/tools• Selecting assessment processes/tools

• Individual exercise: select a set of learning objectives and generate alternative assessment processes/tools

• Individual exercise: select one or more assessment processes/tools that you would use in your course

• Team exercise: share and review choices of assessment processes/tools• Review workshop activities


Learning ObjectivesBackground for Learning Objectives

Session Objective: At the end of the session, participants will describe themselves as more confident in their ability to hold productive conversations with their colleagues regarding the place and importance of learning objectives in the teaching-learning process


Stakeholders for Learning Objectives

• Who are the stakeholders in conversations about preparing and applying learning objectives?• Faculty• Students• Employers• Accreditation organizations


EC Program Outcomes(a) an ability to apply knowledge of mathematics, science, and engineering (b) an ability to design and conduct experiments, as well as to analyze and

interpret data (c) an ability to design a system, component, or process to meet desired

needs (d) an ability to function on multi-disciplinary teams (e) an ability to identify, formulate, and solve engineering problems (f) an understanding of professional and ethical responsibility (g) an ability to communicate effectively (h) the broad education necessary to understand the impact of engineering

solutions in a global and societal context (i) a recognition of the need for, and an ability to engage in life-long learning (j) a knowledge of contemporary issues (k) an ability to use the techniques, skills, and modern engineering tools

necessary for engineering practice.


Bloom’s Taxonomy of Cognitive Learning

• Knowledge: defines, recalls, matches, reproduces

• Comprehension: explains, gives examples• Application: discovering, assessing,

computing• Analysis: breaking down, organizing,

inferring• Synthesis: creating, putting together• Evaluation: appraising, judging, selecting

Western Michigan University, 25 October 2002, Kalamazoo, Michigan


Purpose of Learning Objectives

• Communicate expectations for a course

• Provide a context for what will be learned

http://wings.buffalo.edu/vpaa/ctlr/events/brazeau/index.htm


Objectives and Students

• Objectives help students • Clarify their personal goals• Provide framework for measuring their success. • Reduce their anxiety • Improve their studying effectiveness

• Objectives help instructors• Guide preparation of classroom material • Make homework assignments• Aid in test designSource - http://wings.buffalo.edu/vpaa/ctlr/events/brazeau/index.htm


Strategies For Workshop Teams

• Be positive, supportive, and cooperative• Limit critical or negative comments• At least 5 positive comments for every negative

comment

• Be brief and concise in discussions• Avoid lengthy comments, stories, or arguments

• Stay focused


Team Roles• Assign team roles & follow through on

responsibilities• Coordinator -- Coordinates discussion & develops

consensus• Recorder -- Writes down the ideas & reports them• Gatekeeper -- Keeps the team on the subject • Timer -- Makes sure the team stays on schedule

• With smaller teams – combine gatekeeper & timer


Workshop Team Roles

• For first exercise• Coordinator – Individual with largest class last

semester • Recorder/Reporter – Individual on left of coordinator• Gatekeeper/timer -- Individual on left of recorder• Timer -- Individual on left of gatekeeper

• Roles rotate clockwise on subsequent exercise


Team Exercise

• Form teams of four people• Time: 5 minutes• Develop at least four (4) advantages and

four (4) disadvantages of preparing learning objectives for a course.


Team ExerciseAdvantages

• Communication with students expectations and what they need to learn

• Offers focus in preparing material to accomplish goals

• Evaluation tool• Tie a course to program or curriculum• Clarifying goals to students seeks buy-

in and self-assessment• ABET Assessment tool• Stating goals facilitates communicates

with colleagues for better coordination• Written goals facilitates

communications with employers and other external groups

• Makes course easier to transfer among faculty members because much work is already done

Disadvantages• Could be too narrow or

focused• Explicitly stating learning

objectives takes (too?) much time

• Busy work for faculty• Could turn into curriculum

nightmare• Poorly written objectives

don’t help• Could be cumbersome,

ambiguous• Could be more difficult to

measure


Learning ObjectivesPreparing Specifications for Learning

ObjectivesSession ObjectivesAt the end of the session, participants will Write specifications for learning objectives Describe themselves as more confident in their

ability to describe quality learning objectives.


Form of Learning Objectives

• Write objectives as student outcome statements

• Objectives should answer the questions• "What must students do to prove that they have

succeeded?" • "What should students be able to do as a

consequence of instruction?"

Source - http://wings.buffalo.edu/vpaa/ctlr/events/brazeau/index.htm


Elements of an Objective

• Objective must contain three basic elements: • Verb describing an observable action • Conditions of this action

• “When given x you will be able to..."

• Level of acceptable performance

Source - http://wings.buffalo.edu/vpaa/ctlr/events/brazeau/index.htm


Verbs for ObjectivesVerbs for constructing concrete objectives:

analyze compute classify collaborate compare appreciate contrast define demonstrate direct derive designatediscuss display evaluate knowidentify infer integrate interpretjustify list understand organizegrasp report respond solicitstate synthesize name explain

Modified from http://www.brown.edu/Administration/Sheridan_Center/pubs/syllabus.html#coura


Learning Objectives Verbs for Categories in Bloom’s Taxonomy

• Knowledge • Define, describe, list, reproduce, enumerate

• Comprehension• Classify, explain, discuss, give example, summarize

• Application• Determine, develop, compute, chart, utilize

• Analysis• Correlate, diagram, distinguish, outline, infer

• Evaluation• Compare & contrast, critique, justify, conclude

• Synthesis• Adapt, combine, compare, contrast, design, generate


Complex Versus Simple Objectives

• One complex objective versus several simple ones?• High-level versus low-level objectives

Example• One complex objective (4 or 5 weeks of classes)

“Given a verbal description of a digital module, develop an implementation using any of 7 different logic devices”

• 15 to 20 simpler objectives (1 or 2 per class)“Given a verbal description, draw the truth table”“Given a truth table, obtain a minimum-cost equation”…“Draw the the NAND-gate implementation for an equation”


Comparison Of Complex & Simple Objectives

• Multiple simple objectives• More manageable “chunks” for students• Explicit objective(s) for each class• Simple (more manageable) homework problems

and test questions• Single complex objective

• Student's attention directed to the overall process• May lead to higher level learning

• Students must deal with complexity • Students must subdivide problem on their own


Advantages Of Simple Objectives

• Advantages of simple objectives are more important in• Large classes rather than small classes• Introductory courses rather than advanced courses


How To Deal With “Understand” In Objectives

• How do you write objectives when you want students to “understand” a complex concept, system, or process

• Identify specific tasks that indicate “understanding” • Specify objectives for each task

• Similar comments apply to “know”, “appreciate”, “value”


How To Deal With “Understand” In Objectives -- Example

• In our computer architecture course we want students to “understand” a sample architecture made up of several modules

• What would students be able to do if they “understood”

• Objectives – • Students should be able to identify:

All the modules and interconnecting signalsModules involved in a given system-level operationOutput values for a given input values for each module Sub-module changes given a system level change…


Writing Objectives – Piecemeal Approach

• Writing low-level objectives for a whole course may be overwhelming

• Use a “piecemeal” approach• Write your lectures and define the homework as usual• After each class -- write down what you expect the students to be able

to do • These become a list of objectives

• Give them to the students before each exam • Use them to write the exam

• As semester progresses -- may become comfortable writing the objectives before you prepare your lecture


Evolving Objectives

• In a 3-credit semester course • Russ Pimmel (UA, now NSF) started with over 100

objectives• Four offerings later -- down to about 50

• Eliminated peripheral “stuff” that was not central • Broadening, informational, perspective material


Objectives and Homework Assignments

• Homework assignments should match objectives• Students need to practice and explore the skills, knowledge,

and attitudes defined in objectives

• Frequently require supplementary homework problems• In some of my courses

• 1/3 of homework is from textbook • Rest are supplementary problems

• With well defined objectives • Writing homework problems is straightforward • Same is true for exam questions


Student’s Use Of Objectives

• Survey in 400-level required course

• Did you find the objectives helpful? Yes --- 52 % No ---- 48%

• Did you read the objectives? Frequently ------ 22 % Occasionally -- 37 % Never ------------- 41 %


Team Exercise -- Guidelines For Learning Objectives

• Task• Write 3 or 5 guidelines for good learning objectives

• What are the common features?• What should objectives look like?

• Think of guidelines as specifications• Methodology

• Brain storm individually -- 2 minutes• Establish consensus as a team -- 5 minutes• Report team results -- 3 minutes• Revise guidelines as a team-- 2 minutes


Team Exercise• Must be testable and measurable• Achievable• Clearly and precisely articulated• Appropriate to course and audience• Relate to program objectives• Linked to course outcomes to allow for

assessment• Simple, one sentence, common format• Simple better than complex• Should be specific and unambiguous• Relate to topic coverage


Learning ObjectivesPreparing Learning Objectives

Session ObjectivesAt the end of the session, participants will Write learning objectives for one or more courses

that they teach Describe themselves as more confident in their

ability to describe quality learning objectives.


EC Program Outcomes(a) an ability to apply knowledge of mathematics, science, and engineering (b) an ability to design and conduct experiments, as well as to analyze and

interpret data (c) an ability to design a system, component, or process to meet desired

needs (d) an ability to function on multi-disciplinary teams (e) an ability to identify, formulate, and solve engineering problems (f) an understanding of professional and ethical responsibility (g) an ability to communicate effectively (h) the broad education necessary to understand the impact of engineering

solutions in a global and societal context (i) a recognition of the need for, and an ability to engage in life-long learning (j) a knowledge of contemporary issues (k) an ability to use the techniques, skills, and modern engineering tools

necessary for engineering practice.


Bloom’s Taxonomy of Cognitive Learning

• Knowledge: defines, recalls, matches, reproduces

• Comprehension: explains, gives examples• Application: discovering, assessing,

computing• Analysis: breaking down, organizing,

inferring• Synthesis: creating, putting together• Evaluation: appraising, judging, selecting

Western Michigan University, 25 October 2002, Kalamazoo, Michigan


Challenges in Engineering Education

• Challenges• Challenge of lifelong learning• Challenge of problem solving• Challenge of engineering design• Challenge of transfer

LASSI SCALE ENGR111(Mean)

CVEN349(Mean)

Significance*

Skill Component

Information Processing 60.68 60.29 0.930Test Strategies 64.33 63.27 0.794Selecting Main Ideas 55.18 59.29 0.342Will ComponentAnxiety 60.52 67.12 0.147Attitude 42.47 34.56 0.080Motivation 63.30 59.29 0.397Self-regulation ComponentConcentration 61.31 54.56 0.144Self-testing 52.47 37.92 0.006Study Aids 60.24 45.04 0.005Time Management 55.23 47.65 0.134

“Although the data suggest a slight upward upward trend, the trend proved not to be statistically significant based upon an analysis of variance (ANOVA). Thus the cross-sectional study did not find evidence of an increase in readiness for self-directed learning, even for students in the later semesters who are taking elective courses and their capstone courses.”

Litzinger, T., Wise, J., Lee, S., and Bjorklund, S. (2003) Assessing Readiness for Self-directed Learning, Proceedings, ASEE Annual Conference

Lifelong Learning at Penn StateSelf-Directed Learning Readiness Survey (SDLRS), Guglielmino & Associates, http://www.guglielmino734.com/prod01.htm, March 2003. 27


Challenge of Problem Solving

“Despite individual professors’ dedication and efforts to develop problem solving skill, “general problem solving skill” was not developed in the four years in our undergraduate program. Students graduated showing the same inability that they had when they started the program. Some could not create hypotheses; some misread problem statements. During the four-year undergraduate engineering program studied, 1974-1978, the students had worked over 3000 homework problems, they had observed about 1000 sample solutions being worked on the board by either the teacher or by peers, and they had worked many open-ended problems. In other words, they showed no improvement in problem solving skills despite the best intentions of their instructors.”

Woods, D. et al (1997) “Developing Problem Solving Skills: The McMaster Problem Solving Program,” Journal of Engineering Education,



• Ineffective approach #1. give the students open-ended problems to solve; This, we now see, is ineffective because the students get little feedback about the process steps, they tend to reinforce bad habits, they do not know what processes they should be using and they resort to trying to collect sample solutions and match past memorized sample solutions to new problem situations.



• Ineffective approach # 2: Show them how you solve problems by working many problems on the board and handing out many sample solutions• This, we now see, is ineffective because teachers know too

much. Teachers demonstrate "exercise solving". Teachers do not make mistakes; they do not struggle to figure out what the problem really is. They work forwards; not backwards from the goal. They do not demonstrate the "problem solving" process; they demonstrate the "exercise solving" process. If they did demonstrate "problem solving" with all its mistakes and trials, the students would brand the teacher as incompetent. We know; we tried!



• Ineffective approach #3: Have students solve problems on the board• Different students use different approaches to

solving problems; what works for one won't work for others. When we used this method as a research tool, the students reported "we learned nothing to help us solve problems by watching Jim, Sue and Brad solve those problems!"



• Through four research projects we identified why and how these and other teaching methods failed to develop process skills and which methods were successful in developing the skills

• Woods, D.R., J.D. Wright, T.W. Hoffman, R.K. Swartman and I.D. Doig (1975) "Teaching Problem Solving Skills," Annals of Engineering Education, 1, 1, 238-243.

• Woods, D.R. et al. (1979) "Major Challenges to Teaching Problem Solving" Annals of Engineering Education, 70, No. 3 p. 277 to 284, 1979 and "56 Challenges to Teaching Problem Solving" CHEM 13 News no. 155 (1985).

• Woods, D.R. (1993a) "Problem solving - where are we now?" J. College Science Teaching, 22, 312-314.

• Woods, D.R. (1993b) "Problem solving - what doesn't seem to work," J. College Science Teaching, 23, 57-58.

• Woods, D.R. (1993c) "New Approaches for developing problem solving skills," J. College Science Teaching, 23, 157-158.


Challenge of Engineering Design

The literature is filled with positive comments from students, instructors, and industrial sponsors who have participated in capstone design courses. The vast majority of participants feel that the course benefited all involved.The nature of capstone design courses, however, often leads to a purely subjective evaluation with little or no “hard evidence” of actual benefits. Born, for example, does not attempt to prove the value of senior level design courses. He simply states that he is convinced from his experiences that such courses are valuable. Other educators have similar “feelings” as to the relative costs and benefits of capstone design courses.

Dutson, A.J., Todd, R.H., Magleby, S.P., Sorensen, C.D., (1997) “A Review of Literature on Teaching Engineering Design Through Project-Oriented Capstone Courses.” Journal of Engineering Education


Challenge of TransferResearches posed this problem to people.

"Suppose you are a doctor faced with a patient who has a malignant tumor in his stomach. It is impossible to operate on the patient, but unless the tumor is destroyed the patient will die. There is a kind of ray that can be used to destroy the tumor. If the rays reach the tumor all at once at a sufficiently high intensity, the tumor will be destroyed. Unfortunately, at this intensity the healthy tissue that the rays pass through on the way to the tumor will also be destroyed. At lower intensities the rays are harmless to healthy tissue, but they will not affect the tumor either. What type of procedure might be used to destroy the tumor with the rays, and at the same time avoid destroying the health tissue?"


Challenge of TransferConsider the following story

"A small country was ruled from a strong fortress by a dictator. The fortress was situated in the middle of the country, surrounded by farms and villages. Many roads led to the fortress through the countryside. A rebel general vowed to capture the fortress. The general knew that an attack by his entire army would capture the fortress. He gathered his army at the head of one of the roads, ready to launch a full-scale direct attack. However, the general then learned that the dictator had planted mines on each of the roads. The mines were set so that small bodies of men could pass over them safely, since the dictator need to move his troops and workers to and from the fortress. However, any large force would detonate the mines. Not only would this blow up the road, but it would also destroy many neighboring villages. It therefore seemed impossible to capture the fortress. However, the general devised a simple plan. He divided his army into small groups and dispatched each group to the head of a different road. When all was ready he gave the signal and each group marched down a different road. Each group continued down it road to the fortress at the same time. In this way, the general captured the fortress and overthrew the dictator."


Challenge of Transfer• After the subjects read and summarized this story,

they were asked to solve the tumor problem under the guise of a separate experiment.

• Given the clear analogy, you might think that performance would be near ceiling. Surprisingly, only 30% of the subjects offered a convergence solution.

• Moreover, when these same subjects were given the suggestion that they should use the General story, 80% provided a convergence solution.

• This finding demonstrates that half the subjects could apply the General story to the tumor problem when they were instructed to but did not do so on their own.


Focusing Activity (8 minutes)

• INDIVIDUALLY – use 3 minutes to write your description of learning, what it is, what it looks like, how you might recognize when it has occurred, etc.

• AS A PAIR – use 5 minutes to discuss descriptions with someone sitting next to you. If you have additional time, develop a consensus description of learning.


Focusing Activity• ??


Individual Exercise -- Writing Learning Objectives

• Individually write a set of objectives for a topic representing a few classes • Something that you recently did in class

• Follow your team’s guidelines• Questions to consider about your objectives

• Do they define student behavior?• Are they observable, measurable?• Can you write homework & exam problems?• Are they consistent with the instructor’s intent?

• Time: 15 minutes


Team Exercise -- Reviewing Learning Objectives

• Review each other’s objectives

• Questions to consider in reviewing objectives• Do they follow your team’s guidelines

• Do they define student behavior?

• Are they observable, measurable?

• Can you write homework problems & exam questions?

• Are they consistent with the instructor’s intent?

• 15 minutes


Individual Exercise -- Revising Your Learning Objectives

• Rewrite your learning objectives based on your team’s review

• Report on biggest improvement

• 10 minutes


Biggest Improvement• Put quantifiable goals that students would achieve• Clarify my objectives• Classify learning objectives to go from basic to specific• Make a different level of objectives from whole course

to chapter to each lecture• Make few higher level objective to organize multiple

lower level objectives to support• Identify more complex goals and separate into simpler

components for teaching communications• Improve clarity• Make more understandable to the student• Use proper verbiage to clarify


Minute Paper

• Write a one-sentence answer to the following question:• What is the “muddiest point” about

learning objectives?• (What is the most confusing point?)


Muddiest Point• How to incorporate those that relate to attitude and enthusiasm for

lifelong learning• Why do we spend time writing things that we do and we do best?• Answer to my question about the level and number of learning

objectives that are appropriate for publication in my syllabus. Are there “intermediate” not published objectives?

• How to relate a given objective to Bloom’s Taxonomy level, and how important is it to do so?

• Ways of measuring each learning objective• How many objectives should be used – level of detail – then how do

we practically measure them?• “Being brief” vs. “Being precise and accurate”• Being able to link objectives to larger program outcomes or goals• Organizing subject matter to achieve learning objectives


Team Exercise -- Reflection on Learning Objectives

• Assume that you are a debate team

• Write the single best pro and con arguments for the statement• “Using learning objectives improves

student leaning.”


Classroom Assessment


Feedback on Learning Objectives

• ??


Agenda• Background for classroom assessment • Specifications for assessment processes/tools

• Team exercise: develop a set of specifications for assessment processes/tools

• Workshop exercise: improve sets of specifications for assessment processes/tools

• Generating alternatives for assessment processes/tools• Team exercises: select some learning objectives and generate alternative

assessment processes/tools• Background on alternatives for assessment processes/tools• Selecting assessment processes/tools

• Individual exercise: select a set of learning objectives and generate alternative assessment processes/tools

• Individual exercise: select one or more assessment processes/tools that you would use in your course

• Team exercise: share and review choices of assessment processes/tools• Review workshop activities


Classroom AssessmentBackground for Classroom Assessment

Session Objective: At the end of the session, participants will describe themselves as more confident in their ability to hold productive conversations with their colleagues regarding the place and importance of assessment instruments and processes in the teaching-learning process


Pre-Assessment• Exercise

• Classroom Assessment• Individual

• Write a one-sentence answer to the following question:• “What can you do in the last few minutes of class

to determine how well your class learned what you taught that day?”



• At end of session, participants will be able to:• Define several classroom assessment tools• Discuss the importance of using classroom tools in

the teaching-learning process• Write assessment tools for their classes


Resources on the Web• www.hcc.hawaii.edu/intranet/committees/

FacDevCom/guidebk/teachtip/assess-1.htm• www.hcc.hawaii.edu/intranet/committees/

FacDevCom/guidebk/teachtip/assess-2.htm• www.siue.edu/~deder/assess/catmain.html


Types Of Assessment• Assessment

• Used in many contexts• Used for several different foci

• Classroom• Teacher focus: concerns your performance• Student focus: concerns their individual performances

• Program• Alumni (as a group) focus: ABET Criterion 2• Graduates (as a group) focus: ABET Criterion 3• Individual graduate focus: Comprehensive exams


Types Of Assessment• Classroom, teacher focus

• Question -- How effective was a lecture, assignment, lab?• Tools -- One-minute paper, student survey

• Classroom, student focus• Question – Did a specific student achieve the learning

objectives?• Tools -- Exams, reports, presentation

• Program, graduates (as a group) focus• Question -- How well did a group of students achieve a set of

objectives (outcomes) in a program or course?• Tools -- Standardized tests, design project report analysis


• Two fundamental questions: • How well are learners learning? • How effectively are teachers teaching?

• Deals with better learning and more effective teaching• Provides feedback about effectiveness as teachers

• How students learn • How they respond to particular teaching approaches.

• Gives students a measure of their progress as learners


Modified from www.hcc.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-1.htm


Incorrect Assumptions About Teaching & Learning

• Instructors assume students learn what they teach • Tests, concept inventories, and term papers provide

disappointing evidence to the contrary• Students have not learned as much or as well as

expected • Gaps between what was taught and what was learned

• Sometimes considerable gaps

• Instructors notice gaps too late to remedy the problems

• Classroom assessment can uncover gaps earlier www.hcc.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-1.htm


Classroom AssessmentGetting Started

• Planning • Select one, and only one, of your classes • Choose a simple and quick technique

• Implementing• Make sure the students understand the procedure • Analyze student’s responses as soon as possible

• Responding -- “Close the feedback loop”• Tell students what you learned and what you will do

about it - motivates students to become actively involved www.hcc.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-1.htm


• Don’t use any technique that does not appeal to you

• Don't make it into a self-inflicted chore or burden. • Try it yourself before you use it with students • Allow more time than you think you will need

• To carry out the assessment • To respond to it

• Make sure to "close the loop" • Let students know

• What you learned from their feedback• How you and they can use that information to improve learning

Classroom Assessment – Five Suggestions

www.hcc.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-1.htm


Team Exercise

• Process• Brain storm individually -- 1 minutes• Establish consensus as a team -- 4 minutes

• Task: Write 2 to 4 guidelines for good classroom assessment tools (Think of guidelines as specifications)• What are the common features?• What should they look like?• Do all guidelines have to apply to all tools?


Team Exercise• Tool should be efficient use of time for instructor and

class• Easy for students to use• Can be used across different classes and situations• Built into teaching methodology and doesn’t look like an

add-on• Tied to learning objectives or limited number of topics• Flexible, not dependent on particular technology• Done repeatedly throughout semester• Easy for faculty to administer and evaluate• Self-documenting, easy to show results to others• Clear and precise instructions


Team Exercise

• Process• Brain storm individually -- 2 minutes• Generate different ideas as a team -- 3 minutes

• Task: Generate as many possible good classroom assessment tools that might satisfy your team guidelines• Generate different ideas


Team Exercise• Muddiest point• Quizzes on specific objectives• What is clear?/What is unclear?• What did you learn today?• Strengths/Improvements/Insights (SII)• Self or peer assessment• Who did you share X with today?• Weekly feedback on homework instruction – what

was clear? What was unclear?• Continuous Improvement – What was beneficial?

What would you change?


Team Exercise

• Process• Brain storm individually -- 2 minutes• Generate different ideas as a team -- 3 minutes

• Task: Generate as many possible good classroom assessment tools where the student work product is primarily graphical and the tools might satisfy your team guidelines• Generate different ideas


Team Exercise• List topics with smiling/frowning faces• Flow charts/flow sheets• Process flow of your problem solving process• Road map to get from A to B• Bar graph of confidence related to specific objective• Free body diagram• Web of association to a particular term• Story board of set up or explanation of particular concept• Fit concept into network or tree of concepts• Draw a concept


Examples of Assessment Techniques

• Background Knowledge Probe• Students respond to short-answers or multiple-

choice questions • General information on their level of preparation

• Minute Paper (most widely used) • Students write brief response to

• "What was the most important thing you learned during this class?"

• “What important question remains unanswered?“



Examples of Assessment Techniques (Cont.)

• Muddiest Point (simplest technique, remarkably efficient)• Students jot down a quick response to one

• "What was the muddiest point in ....... ?”• A lecture, a discussion, a homework assignment

• One-Sentence Summary• Students answer the questions "Who does what to whom,

when, where, how, and why?" (WDWWWWHW) • Synthesize answers into a simple, informative, grammatical

sentence.



Examples of Assessment Techniques (Cont.)

• What's the Principle?• Students state the principle that best applies to a

few problems



Concept Map

• A concept map is a set of nodes that represent concepts connected by a labeled links that describe a link between concepts.

Concept A

Concept BDescribe how concept A and

concept B are related?


• Start with a subset of the concepts on the following page and construct a concept map that shows the concepts you have selected and how they are related.

• Exchange concept maps and share insights

Team ExerciseBuilding a Concept Map

FeedbackDerivativeFinite Element AnalysisIntegralLinear MomentumAngular MomentumEnergyInterestMassIdeal Gas LawFick’s First LawFick’s Second LawVectors: Dot ProductVectors: Cross ProductOrdinary Differential EquationsKirchoff’s Voltage LawSecond Law of ThermodynamicsKirchoff’s Current LawModelingProblem-SolvingForceOhm’s LawResistanceComplex NumbersLogarithmic FunctionElectric FluxDecision TheoryDivergenceIndirect CostCapacitanceBending MomentFeedback

First Law of ThermodynamicsEntropyHeatElectric FieldMagnetic FieldPartial Differential EquationsDeterminantsReturn on InvestmentPhasorsBrainstormingExponential FunctionConductivityChemical KineticsSpecific HeatElasticityMalleabilityPlasticityResiliencyPermittivityCurrentElectric PotentialCurlPresentation SkillsDemocracyProfitDensityMoleculePhaseShearRheology Frequency ResponseEigenvalue, Eigenvector

Sinusoidal FunctionsWorkDisplacementVelocityAccelerationResistivityLeadershipHess’ LawZeroth Law of ThermodynamicsElectric PotentialMagnetic FluxDesignMaxwell’s EquationsPowerDuctilitySpring ConstantStressStrainPartial DerivativePermeabilityChargeMagnetic PotentialGradientParagraphRate of ReturnFrequencyAtomRoot LocusTorqueInductanceTorsionPolymerKinetic Theory of Gases


Cowan’s Teaching Examples

• Bridge design• Design and build two different bridges and grade

on the lower performance design• Problem-solving script

• Illustrate script for one type of problem, ask students to develop a script for another type of problem

Cowan, J. (1998) On Becoming an Innovative University Teacher: Reflection in Action. Buckingham: SRHE and Open University Press.


Some Personal Observations


Relating Student Performance On Exams To Objectives

• Write exam using objectives• Select objectives for exam questions from list

• Many objectives -- test questions represent a sample• Exam question may involve more than one objective• Use some “hard” and some “easy” questions

• Identify questions (& objectives) a high percentage missed• Review idea in class -- give additional work• Modify lecture, reading, or homework for future • Change the objective


Using Cooperative Learning In-class Exercises

• Examine students work during the in-class exercise• If all have a good approach -- may be wasting time• If all are lost -- may need more explanation• If one-half to two-thirds have a good approach -- level and

pace are right

• Collect and show a few solutions to in-class exercises• If all have correct approach -- may be wasting time• If all are wrong -- may need more explanation• If one or two are correct -- level and pace are right


“One-Minute” Papers

• Common questions• What one thing should be changed about ____?• What one thing should not be changed about ____?• What do you think about ____?• What is the “muddiest” point about ____ ?

• Ask about• Course or lecture• Text or chapter• Assignment or test• Teaching style or class activity


Some Colleagues’ Observations


Informal Assessment Techniques

• General Guidelines• Keep them anonymous• Use then frequently – better feedback• Close the loop

• Let students know results of the process

David Cordes


Informal Assessment Techniques - Daily Activities

• One-minute paper• At the end of the lecture, ask students for:

• The most important topic that we covered today• The one topic you are still confused about

• Single sheet of paper, no names• Can read on the way back to the office

• Look for “common problems”• Look for “did they understand my focus?”

David Cordes


ME 360 - Plus / Delta Assessment #1

• On one side of “sticky” pad• Put a “+” in upper left hand corner• What is something that worked well or made more sense in

lab this week?• On other side of “sticky” pad

• Put a “” in upper left hand corner• What is something that could have been done better in lab

this week?• “Stick” on the door on your way out

Joey Parker


ME 372 - 1st Day of Class

• What are a valid set of units for a mass moment of inertia? (Dynamics concept)

• What is the difference between a capacitor and a resistor? (Circuits concept)

• What is the equation of the straight line that passes through the points X=2, Y=7 and X=7, Y=2? (Math concept)

Joey Parker


Team Exercise• Process

• Brain storm individually -- 2 minutes• Establish consensus as a team -- 5 minutes• Report team results -- 3 minutes• Revise products as a team -- 2 minutes

• Task: Write 3 to 5 guidelines for good classroom assessment tools (Think of guidelines as specifications)• What are the common features?• What should they look like?• Do all guidelines have to apply to all tools?


Team Exercise• Simple, quick, easy to administer and evaluate• Provide immediate feedback to students – they

need to know you’re using the results• Tied to a specific topic or objective• Gives usable, valuable information• Identification of strengths and areas of

improvement• Give clear and precise instruction to the student• Using graphical and textual feedback• Result in action


Team Exercise

• Process• Brain storm individually -- 2 minutes• Establish consensus as a team -- 5 minutes• Report team results -- 3 minutes• Revise products as a team -- 2 minutes

• Task: Generate as many possible good classroom assessment tools that might satisfy your team guidelines• Generate different ideas that you have generated or

have seen


Team Exercise• Ask students to identify an application based on concept

presented in class• Person A teaches Person B, and then they write what was hard

to convey and what was hard to learn• Identify an objective for class and rate their comfort level with

number or bar graph• Ask student to give example to transfer example in class to

another context• Students examine 2-3 methods/opinions/processes for similar

purposes, ask groups to analyze similarities and differences, solicit results, and then invent their own that combines methods/opinions/processes

• Ask students to identify concept based on a diagram.• Ask students to write problem that could be solved by concept

presented in class


Individual Exercise

• Individually write a set of assessment tools for a class or a topic representing a few classes

• Follow your guidelines• Consider the following questions about your tool

• Can your students understand the task? • Can your students do the task quickly?• Can you analyze the results quickly?• Can you summarize and report the results easily?• Does it assess student learning?

• 5 minutes


Team Exercise

• Review each other’s objectives and assessment tools

• Consider the following questions• Does the tool follow your guidelines? • Can your students understand the task? • Can your students do the task quickly?• Can you analyze the results quickly?• Can you summarize and report the results easily?• Does it assess student learning?

• 15 minutes


Individual Exercise

• Rewrite your assessment tools based on your team’s review

• Identify the major improvement

• 5 minutes


Minute Paper• Write a one-sentence answer to the

following question:• What is the muddiest point about classroom

assessment?


Individual Exercise• How can students be more involved in assessing each

other, as well as the self-assessment we have largely concentrated on, and do this in reasonable time.

• How often is a reasonable amount of assessment? How much is too much?

• Going from creating an assessment tool to actually implementing the assessment.

• Incorporation of feedback will not be every time easy and straight forward

• Will students see the value of classroom assessment? Yes, with good approaches. Yes, with good feedback.

• Being able to design a clear, concise assessment tool that is both beneficial to students and professor


Team Exercise• Assume that you are a debate

team • Write the single best pro and con

arguments for the statement • “Using classroom assessment tools

improves student learning.”


Workshop Objectives & Action Items

• Recall objectives • At end of session, participants will be able to define, discuss,

& write• Learning objectives for their courses • Assessment tools for their classes

• Workshop provided a structure for & experience in writing • Learning objectives • Assessment tools

• Your charge – In one of your courses next semester use• Learning objectives• Classroom assessment tools


Questions?


Some Personal Observations

learning objectives and classroom assessment

Documents