An Industry Leader's View of Engineering Education

Steve Kirschstk@propel.com

Agenda

• My qualifications• Ideas on improving

- pre-school- design and process- feedback

Why should you listen to me?

• I have no credibility in the field of education• …but the EECS auditorium at MIT is named

after me • I represent the customer

- A fresh perspective

Engineering a “better education”

• 3 ways to make a better product- Better parts (“raw materials”)- Better product design and manufacturing process- Use feedback for quality assurance (QA) and for ideas on how to continuously improve

• Same is true of education- Better educated students coming into the system- Improve what we teach, how we teach it, and who teaches it- Ask students 0-15 years later for feedback and actually use it to change the previous 2 steps

Improving the quality of the incoming students

’95 TIMSS results42 countries

Texas had state/local control… imagine what a focus on testing can

do for the rest of the country!

The vision

All students in the USA receive the best K-12 education in the world

TIMSS

4th grade

12th grade

How do we do that?

• Establish clear, measurable goals- Example: For K-12, it could be NAEP or TIMSS improvements

• Change what we do- Copy what works - Stop doing stuff that doesn’t work

• Measure and adjust

This isn’t rocket science. It’s just that we don’t do it very well.

Changes required

• Understand why we have failed in the past• Avoid those mistakes• Realize that K-12 is a federal problem not a local problem:

the states have had their chance for over 200 years, yet no state “stands out” of the pack (NAEP). State/local control can make things dramatically worse, e.g., Texas’ results on TASP.

• Understand why other countries have succeeded and adopt “best practices” of top performing countries. Copying first to get to parity, innovating later.

• Have the leadership and courage to do things dramatically differently than we are today

• Create a vision, clear goals, and a believable strategy based on what has proven to work

Why isn’t there a checklist for education that we actually

use?• In aviation, a pilot uses an extensive proven

checklist to ensure a safe flight…Why not offer a substantial on-going cash

incentive to enable schools to pass a proven checklist of statistically proven replicable

requirements that ensures a quality education?

Why don’t we run K-12 schools like airlines?

A. Airlines governed by national safety standards (not set by airlines!)

B. All pilots are qualified and certified to fly the plane type (no unqualified substitutes allowed!)

C. On-going pilot training is requiredD. Pilots who don’t perform can be firedE. Planes that don’t meet code can’t be usedF. Pilots free to determine how to fly the plane, but not

the destinationG. Require pilots to go through a proven safety checklist

(that ensures a successful flight) before takeoff

How bad are things today?

• There are >50 state standards for what students are to learn. Everyone else has 1 national standard.

• Alignment (standards, curriculum, materials, assessments) is therefore virtually impossible

-Imagine if Bill Gates were forced to write Windows for each state?

• We are set up to fail.• There is no good reason we can’t have national

standards. That is step #1.- NAEP is perfect proof of “de facto” national standards. It can be done.

Changing how we teach

Why the sudden discontinuity after 12th grade?

K-12 College

Teaching style Very structured Very unstructured

Teacher training 2 years 0

Methods to improve the process

• Set clear, measurable goals-Understand very clearly what you are trying to achieve

• Copy what works-Copying is always the most efficient way to get to world class performance

• Innovate later- Innovation is required for progress AFTER you are #1

Metric ideas

• TIMSS-like standardized assessment tests• Self-perceived customer satisfaction• Peer/supervisor rated satisfaction• % employed after 2 years• % who were fired in first 10 years• Average salary improvement after 4 years• % who got jobs after graduation

Wrong way: NCLB

• # of schools that NCLB was tested on before being rolled out nationally

- ZERO

• Amount of improvement that can be expected from NCLB

- Nobody has a clue. Could be negative.- Even worse: we knew Bush’s model (TASS) failed

We should NEVER be experimenting on our kids at mass scale like this!!

Why not treat education policy like drugs?

• Efficacy-Require proof of efficacy, i.e., it has to work

• Safety-Require testing on a diverse population to look for unintended consequences (negative side effects)

• Scalability-Prove that it can be rolled out in scale and still work (e.g. GMP manufacturing standards)

Right way: NCEE’s America Choice

• Research best practices-Spend 11 years studying best practices in other countries

• Prototype-Create a prototype approach based on the learnings but adapted for the US market

• Test and validate-Test it at few schools- Get a third party to validate results by comparing to other schools of similar profile

• Scale -Roll out to more schools

• Re-test- Measure again to make sure it scaled with same results and without any unintended consequences

Changing what we teach

Give people what they need to be successful

The Kirsch dietIs it good for you?

ProteinFatCarbohydrates

• Genetically, we are all virtually identical. So shouldn’t this pie chart work for everyone? You can just pick the foods you want, but EVERYONE is required to stick to the %’s

MIT

• 360 units to graduate- 48 units humanities (pick from traditional subjects)- x units core subjects- y units elective- z units General Institute requirements

• They review this every 50 years, whether they need to or not• Why aren’t there multiple pie charts? Does one pie really fit

all?- executive- Manager- entrepreneur- researcher- engineer

MIT

• In 2004, they discovered that communication is more than technical writing

Why aren’t we teaching soft skills?

• Presentation skills• Teaching skills• Leadership• Teamwork• Negotiation skills• Nonverbal communication• Decision-making• Company politics• Giving feedback• Receiving feedback• Sales skills• Attire• Philanthropy

• Behavioral change• Hiring and firing people• Interviewing skills• Social skills, e.g., how to get a

date• Basic project management skills • Managing complex projects• Organization skills• Coaching/mentorship• Working a room• Running a meeting• Conflict resolution• Public policy

“I would love to take such a course” - Tony Eng, MIT

Soft skills

• If there is one change you can make right now to improve engineering education, this is it

• Few offerings are comprehensive• Even fewer universities (none?) require this

to graduate• Don’t try to integrate it into existing course;

copy what works- These skills have been taught very successfully in standalone single-topic seminars

MIT hasn’t “got it” yet (after 20 years of prodding)

• 6.UAT/6.ThT  Preparation for Undergraduate Advanced Project/Masters of Engineering Thesis

-Upon completion of the course, students will have learned how to:- propose and define research problems and think about solutions- critically evaluate technical presentations- architect technical presentations- present technical material in oral and written forms to different audiences at different levels of detail- give and receive constructive feedback- write progress reports

Why not also teach basic survival skills?

• Basic investment skills- Do you know why a stock goes up when they announce bad news?- Do you know why most people who trade stocks frequently lose money?

• Personal relationships

What else should we teach?

• The answer is in the feedback section!

Changing who is doing the teaching

Faculty changes

• Require a 6 week training course for new faculty

- West Point does that

• Require people who teach engineering to have “real world” experience in industry

- Example: take off a semester and get a full time job

• Change incentives to reward collaboration, teamwork, and teaching

- Not just individual research!

Measure and adjust

Creating a feedback system for quality assurance and continuous

improvement

Education is an open-loop op amp

Students in Students out

Jobexperience

Collegeeducation

Example of using feedback: Why executives fail

• Interpersonal skills (CFO, VP Engr)• Lack of teamwork

-Lack of confidence in the team to work together (VP Sales)

• Inability to execute-Use their knowledge and skills to lead a team to success (VP Marketing)

What’s common?

• They failed on the soft skills- Nobody failed due to a lack of understanding of the basic subject matter in their core area of expertise- Everyone failed due to an inability to translate their knowledge into action

The Paul Cook success model

• Q: How do you spend your time?• A: On people problems.

• So feedback should measure:- what skills are the most used- what skills they could use to be more effective- what skills were required for success- what skills led to failure

My experience at MIT was lopsided

• Technical skills- BEFORE<<AFTER

• People skills- BEFORE = AFTER

Why isn’t there a feedback system in place?

• What are our goals? • Do we have metrics aligned to those goals?• Do we know what they are?• Is it used as a basis for compensation?• Is it used as a basis for adjusting the process

and the design?

Methods for driving change

• Change the “change process”-Internally driven changes at universities are typically done on geological time scales- To change, copy what has worked at universities who have solved the “lack of change” problem- Start simply: Pick one or two metrics for driving improvement

• ABET can drive changes- Establish standardized customer satisfaction metrics- Phase in a few key checklist items: Are new faculty members required to learn how to teach? - Phase in a few key optional items: Do students demonstrate mastery of soft skills?- … etc.

• CASEE could create and market a single “J.D. Power” metric for rating engineering schools on customer satisfaction

- Could be coupled with a prize for “most improved”- A single combined metric is easier to focus on- Can be component metrics

Influence Public Policy

e.g., your vote on Nov. 2

Significant negative impacts from Bush decisions

• Cutting federal grants: NSF, etc.• Putting our kids at a permanent disadvantage

-Deregulating mercury emissions from power plants has led to the possibility of permanent brain damage in 15% of the children born in the US today- The requirement to remove 95% of the mercury emissions would have costed 1% of the cost of the plant

• Ignoring/distorting science• Cutting funding on his own education bill from what

Congress proposed• Believing unbelievable test results (TASS) then using that as

a basis for national policy• Unfunded mandates

Summary

• Improving education is a lot like building a better computer:

- Quality components: Demand the highest quality components from our suppliers; help them improve- Customer driven design: Adjust the design of the product to fit what people want to buy (a successful career)- Quality volume manufacturing: Improve the manufacturing process by copying best practices and making sure we have clear manufacturing specs (goals) and that the products meet the spec (testing) at high volume (scalability)- Feedback: Take a look at the product after it’s been in use for a few years to ensure quality and provide ideas for improvement

Summary

• Nothing in this talk is new• It’s just that we don’t do it (very well)

Leading indicators of success

• A few clear, measurable goals established and used• People skills being required to graduate• Changes are prototyped and proven to work before scaling

(no more NCLB)• Adoption of national standards (K-12)• Importing best practices from other countries• Federal incentives to adopt strategies that are proven to

work• New faculty are required to learn “how to teach”• Changes take a year instead of a century• Changes driven by customer satisfaction metric(s)

Latest copy of this talk

On my website at:www.skirsch.com