right things right: zef evaluation

Pekka SilvénDirectorProfessional Development Services

Evaluation, Quality and Feedback

Pekka’s ”cv”

– Evaluation, quality and feedback

– Use of Technology in Healthcare Services in Technology Health Centers

– International training programs and projects

– EU-projects

– Developmental neuropsychology

– Teacher

– Freelance radio journalist

–

We all agree on…

that feedback (queries) has an essential role in improving the teaching, learning, arrangements, management, well being and all the other important processes in any (educational) organisation

But…..

…do we really succeed well with this task in real life?

Feedback / Evaluation processes

1/2001

The common problems of evaluation / feedback processes

• expensive…

• lots of work and paper wasted…

• the whole process takes too much time…

• results are unclear and difficult to make conclusions…

lack of improvements based on the feedback… goal and purpose of the process / study remains unclear.. difficulties to see the effectiveness…

“Right Things Right”“the Zef-method”

ZEF Solutions Oy

ZEF Solutions Oy

Course feedback - Company visits- Study material- On-line support- Interactivity with the other students- - - - -

Zef-methodvs.Likert Scale

1 2 3 4 5

X

= 1,6

Averages and Gaussian curveOR

Priotising with Z-score

Reference material(X)

Reference material(Y)

Home Assignments(X)

Home Assignments(Y)

Interactivity with the teacher(X)

Interactivity with the teacher (Y)

Equipments (X)

Equipments (Y)

1,86 3,62 1,97 4,12 3,83 3,83 4,15 3,454,83 4,8 4,71 4,88 4,71 4,85 4,46 4,754,45 4,51 4,23 4,77 4,66 4,82 3,14 4,37

4,74 4,8 3,75 3,78 4,29 4,42 4,57 3,724,68 4,66 4,66 4,51 4,46 4,65 4,11 4,653,85 4,23 4,12 3,78 3,45 3,38 4,46 4,464,46 4,37 4,17 4,11 4,38 4,12 4,68 4,82,15 4,72 2,92 3,85 1,62 4,66 2,55 4,834,94 4,91 4,92 4,92 4,92 4,92 4,92 4,924,92 4,65 4,92 4,35 4,71 4,46 4,74 4,08

4,71 4,72 4,71 4,72 4,65 4,74 4,72 4,682,09 4,15 2,31 3,92 3,89 4,23 2,28 4,584,42 4,37 3,78 4,11 4,15 4,52 4,23 4,69

5 5 5 5 5 5 5 53,94 3,85 3,95 4,03 4,85 4,92 4,83 4,923,29 4,75 3,43 4,66 3,91 4,68 3,48 4,574,74 4,86 4,68 4,8 4,69 4,83 3,69 4,924,55 3,43 2,38 3,29 4,28 4,51 3,97 4,493,94 4,02 4,95 4,95 4,31 4,48 4,14 4,974,89 4,85 4,94 4,94 4,98 4,97 3,89 4,973,55 4,8 3,68 4,94 3,52 4,89 4,55 4,924,71 4,72 4,25 4,43 4,69 4,85 4,06 4,834,75 4,85 4,25 4,25 4,8 4,86 4,82 4,861,92 3,58 2,17 3,74 2,35 3,85 2,08 4,31

Excel/SPSS?

Index?

Ranking list?

Average or

the direction /

priorities

Key figures Present state Target Feedback Action plan

attractiveness of the course (the number of students applying for the school / courses)

2,5 3,8 Report –case1 action plan

how many students will complete the studies in scheduled time

67 % 85 % Report –case2 action plan

How many students will interrupt the studies

25 % 5 % Report –case3 action plan

Employment rate 74 % 92 % Report –case4 action plan

Euros Quality

Benchmarking

Customer satisfactionResponse rate 83%

How have we succeeded in serving you?

1. Project Management2. Project Planning3. Systematic Execution4. Communication5. Keeping our promises6. Experience and know-how7. Cooperation8. Dealing with errors and problems9. Flexibility10. Satisfactory results

PwC: Recession survey

Blue: SMEsRed: Large companies1. International networks2. Focusing on real customer needs3. Marketing of innovative products and services4. Role of suppliers in the value chain of your company5. Public and university research in your innovation activity

Employee well-being

1. I am valued and trusted2. Fairness in the workplace3. Giving and receiving feedback4. Communication in the workplace5. Compliance with rules and regulations6. Development opportunities7. Communication on difficult topics8. I support my manager in management9. My line manager values feedback10. Use of satisfaction surveys11.My work produces satisfaction

Hazard and risk assessments

Response rate: 73%Statutory hazard and risk assessmentsTo what extent do you see the following items as hazards/risk factors:1. Effect of the work environment on job satisfaction2. Threat of violence and substance abuse3. Cooperation between managers and employees4. Communication in the workplace5. Uncertainty regarding job security6. Ambiguity in work delegation and responsibilities7. Deficiencies in job orientation8. Lack of support from manager9. Lack of valuation of your work10. Inappropriate treatment in the workplace11. Substance abuse

Innovation evaluations

Which innovations are put into practice?Confidential ideas.

Organisation's self-evaluation

Section 4: Information and knowledge management:1. We monitor the fulfilment and development of our business objectives with appropriate indicators and set target values, in relation to the whole business and to individual processes.2. We monitor the impact and efficiency of our business processes.3. We can easily acquire analytical, up-to-date information on our company's performance and its development in relation to our strategic objectives and other points of comparison.4. Our real-time internal accounting is predictive.5. Our employees measure, analyse and evaluate the quality of their own work in order to make decisions and improvement measures related to their work in line with the whole company's development measures.6. Our employees and the company's most important stakeholders receive accurate information as to how the company operates as a whole and in relation to themselves, and how its objectives are fulfilled.7. We comprehensively manage the measurement data on our company and its functions, and the know-how of our people.8. In knowledge management we utilise effective, creativity-oriented employer-employee negotiations.

Strategy analysisResponse rate 88%Case: TEKES, $200M FinnWell programComponents of the Vision

1. Increase customer orientation in products and services

2. Innovate new diagnostic methods and products3. Innovate new patient care methods and products4. Innovate new biodiagnostics methods and products5. Innovate new biotech patient care products6. Increase user friendliness of the tech and products7. Innovate new preventive care technologies8. Digitalize patient transactions9. Digitalize professional transactions10. Create new services utilizing new technologies11. Increase the use of new technologies in healthcare

processes12. Innovate intelligent decision support systems13. Increase pilot cooperation between companies and

hospitals14. Facilitate the integration of health care systems15. Facilitate more flexible hospital environments16. Create new logistics systems and solutions

Values of the Centre Party

15 largest towns and cities vs. other towns1. Patriotism2. Family and friends3. Equality 4. Nature and environment

All Answers Explanations1. Blogs

2. Wikis

3. Podcasting

4. Forums

5. Instant messaging

6. Audio / video conferences

7. Emails

8. RSS feeds

9. Social bookmarking

10. Shared web based calendar services

11. Sharing of learning materials produced by teachers and students.

Greatest Potential

All Answers Explanations1. Blogs

2. Wikis

3. Podcasting

4. Forums

5. Instant messaging

6. Audio / video conferences

7. Emails

8. RSS feeds

9. Social bookmarking

10. Shared web based calendar services

11. Sharing of learning materials produced by teachers and students.

Start withThese

Services

Revised Competence might help to recognize the Potential

All Answers Explanations1. Blogs

2. Wikis

3. Podcasting

4. Forums

5. Instant messaging

6. Audio / video conferences

7. Emails

8. RSS feeds

9. Social bookmarking

10. Shared web based calendar services

11. Sharing of learning materials produced by teachers and students.

Revise yourCompetence

Revise your Competence

All Answers Explanations1. Blogs

2. Wikis

3. Podcasting

4. Forums

5. Instant messaging

6. Audio / video conferences

7. Emails

8. RSS feeds

9. Social bookmarking

10. Shared web based calendar services

11. Sharing of learning materials produced by teachers and students.

Re-evaluatethe Potentialfor Learning

• IT Specialists• Teachers with eLearning Experience

Mobile Technology Questionnaire: Clear Need For Communication

Evaluation Engine query excample: http://www.zef.fi/service/user/?q=425

Shall we try “Zefing”

EduZef example:http://www.zef.fi/arviointikone/player/?q=837-af98fk7e

How to use Zef-tool

Services / Solutions:

ZEF® Evaluation Engine

ZEF® Comparison Engine

Other solutions

Consultant services

1995 - Idea 1997 - First evaluation solution2001 - Founded2005 - Innofinland award2007 - Biggest survey ever2008 - Office founded in silicon valley

18 employees

5 offices in Finland, 1 in California

ZEF Solutions Ltd.

MTV3's European Election Engine (EU-vaalikone) helps you find the best MEP candidate to vote for. By answering the questions in the service, you can compare your opinions to those of candidates.

MTV3: European Election Engine

MTV3: Presidential Election Engine

More than half a million users.

Should fertility treatments be allowed for female couples?

Shown are the responses from the presidential candidates. Also shown are the averages for all users, and all male and female users

Talouselämä newspaper: Economic Election Engine

The Economic Election Engine (Talousvaalikone) focuses on the hard core of parliamentary power: economic administration. In deciding on the State budget, members of parliament also decide on the taxation and benefits of private individuals and organisations.

Question shown: Should corporate taxation be reduced?

Shown are the responses of the parties and the voters.

Pekka Silvén

DirectorProfessional Development Services

Oulu University of Applied Sciences

Pekka.Silven@oamk.fi+358 10 27 23195