tc0052knew tc websiteraw ma - rice · requires a data-gathering process. ... appropriate data...

IRRIInternational Rice Research Institute

Guidelinesfor the Disseminationand Impactof IRRI Information(GN1-05)

etting he essage utT M OG

March 2000IRRI

International Rice Research Institute

Guidelinesfor the Disseminationand Impactof IRRI Information(GN1-05)

etting he essage utT M OG

ACKNOWLEDGEMENTS

AUTHORS:

Revised version:

Paul Marcotte, IRRI Training Guidelines, March, 2000

Original Publication:

Luka 0. Abe, Paul Marcotte, Robert Raab: ISNAR Training Guidelines 1990

IRRI PRODUCTION STAFF

Editor: Madeline B. QuiamcoLayout and Graphic Artist: Irvin PanganibanProduction: Gina E. Zarsadias and Rina P. Coloquio

International Rice Research InstituteIRRI Training Guidelines: A Handbook for the TrainerDAPO Box 7777Metro ManilaLos Baños, Laguna, Philippines

TABLE OF CONTENTS

Introduction

Section 1: The Design and Delivery of a Training Event ........................1

Step 1: Assessment of Needs ..........................................2

Step 2: Prospectus/Agenda Development ........................3

Step 3: Module Construction ..........................................5

Step 4: Testing and Validation .........................................5

Step 5: Redesigning ........................................................6

Step 6: Production of Materials .......................................6

Step 7: Delivery ..............................................................7

a. The Creation of the Learning Environment .....8

b. Facilitation Skills ..........................................12

c. Presentation Skills .......................................15

Step 8. Evaluation .........................................................18

Section 2: Attachments

1. Prospectus/Agenda Example .......................................23

2. Event Evaluation Form ................................................31

INTRODUCTION:

Training is a process that enables individuals to acquire knowledge, skills/tools, andabilities that will allow them to fulfill the requirements of their job, achieve theircareer aspirations, and attain the goals of their organizations. For the organization,training is one of the processes by which it disseminates the products of its researchprograms, receives feedback from its partners and clients, and ultimately providesthe baseline to track the information and conduct evaluation. Thus, training ismultifunctional in that it plays a vital role in staff development and at the same timeis a conduit for information. Both functions are essential to the success and longevityof the organization.

IRRI has always considered training to be one of the key mechanisms in disseminatingits accumulated knowledge and techniques to a broad audience. Through its degreeand non-degree programs, it has developed human resource capacity in the riceresearch sector. Its distance learning program and materials are models for adult/distance learning. Its on-campus and regional training have literally changed theapproach and method of rice production in the world. Training has played a majorpart in the process historically, and remains an essential technique for multiplyingIRRI’s impact.

In recognition that training is one of the critical functions of an organization, wehave prepared this handbook for IRRI scientists, trainers, and our partners. Werecognize that our own science staff and NARS researchers change dramaticallyover time, that we now have new deliverers of information that include both privateand NGO/PO type organizations, that our information is accelerating rapidly withscientific and technological advancements. Given these developments, training servesan essential function in the development and continuation of the invisible college ofrice scholars and farmers.

This handbook contains:

Section 1: A generalized model of the design and delivery of a training event and

Section 2: Examples and forms

At the outset, it should also be noted that there are 3 overarching concepts thatapply to this handbook. The first is that this document has been prepared for all IRRIstaff at headquarters and in the field, not just the Training Center. It is recognized

that multiple programs, projects and individuals participate in and organize “training”events. These guidelines are intended to assist in the preparation and delivery ofthose events.

Secondly, “training” occurs in many forms such as consultations, conferences,presentations, seminars and workshops, and field demonstrations. In this handbookwhen the term “Training” is used, it is meant to be in the larger sense, ie. thedissemination of information where learning and application is the goal. The guidelinescontained in this handbook have been prepared to assist in any or all of those events,not just the Training Center.

Third, beginning in the year 2000, the Training Center is part of the IRRI wide effortto become proactive in its dissemination and evaluation of information. This includesall of the previous delivery mechanisms such as IPMO, TC, and CREMNET inconjunction with the Research Divisions. The goal is to cover the entire continuumfrom needs assessment including PRA, priority setting of the research agenda, researchexperimentation, to delivery, monitoring and impact evaluation. This also necessitatesthe identification of “new” delivery partners to efficiently reach the target group offield-based extension organizations and the farm communities. It is intended thatthe contents of this handbook become the standard operating procedure for allparticipants on the continuum when they are preparing and offering a “training”event.

IRRI Training Guidelines 1

Evaluation Assessment of Needs

Prospectus/Agenda

Testing and Validation

Redesign/Modification

MaterialProduction

ModuleConstruction

Delivery

Section 1: The Design and Delivery of a Training Event

The objective of this section is to describe a generalized model for training eventdesign and delivery. The model for training design and delivery of a training event, asdepicted in Figure 1, has at least 8 essential elements. These are: assessment ofneeds, prospectus/agenda development, module construction, testing and validation,redesigning/modification, material production, delivery/presentation, and evaluation.These elements are sequential and interdependent, beginning with the assessmentof needs. The model is also an interactive process, with evaluation validating individualelements and circumscribing the entire process. Each of these elements will bedescribed and illustrated in their development sequence.

Figure 1. The design and delivery of a training event

Validation/Evaluation Feedback

Development Sequence

k


STEP 1: Assessment of Needs

IRRI exists to achieve its goal, “… to improve the well being of present and futuregenerations of rice farmers and consumers.” In order to accomplish this, step one ofthe training event process is to identify relevant needs and wants of partners/targets/stakeholders; i.e., any party that has a substantial interest in the group’s work. Thisrequires a data-gathering process. Appropriate data collection instruments includefocus groups, participant observation, surveys, interviews, and maybe even intuitionbased on experience. The following are the recommended steps of the process.

1. Establishing Clear Goals: Prior to the development of a data collectioninstrument, tool or protocol, it is necessary to establish what is to beaccomplished by the data gathering process. Ask questions such as: why isthe data being collected and how will it be used?

2. Appropriate Data Gathering Instruments: There are a number of instrumentsfrom which to choose depending on the goal of the research. The followingare examples of instruments:

A. Focus group -- “… a facilitated group discussion in which open-endedquestions are asked in a way to trigger discussion amongst a panel ofparticipants.” ([email protected], 2000).

B. Participatory Learning/Action -- There are a number of approaches thatfall into this category. Some are: activist participatory research; agro-ecosystem analysis; applied anthropology; farming systems; and rapidrural appraisal. The common characteristics of these approaches are:multiple perspectives, group learning, context specificity, facilitativeprocess, stakeholder involvement they lead to change. Pretty et al (IIEDTrainer’s Guide, 1995).

C. Observation Method -- This is a primary technique for collecting dataon nonverbal behavior: it may be participatory, in which the researcheris a participant in the activities being observed, or non-participatory, inwhich the observer does not participate in the activity being observed.(Bailey, 1994)


D. Survey Research -- This includes questionnaire, mailed questionnaireand interview, each of which have specific purposes, advantages anddisadvantages, and the selection of which to use requires clarity ofpurpose and use (Bailey, 1994).

3. Clear/Relevant Questions: Once the method has been selected, then clear andrelevant questions must be constructed. Proper questions collect data thatwill answer the research question, and at the same time serve reliability andvalidity concerns.

4. Adequate Sampling: While it would be ideal to study the entire population ofthe universe of the stakeholders, a correctly selected sample or subset of thepopulation that most accurately represents the population may be chosen.Rules are dependent on the degree of accuracy of representation desired.

5. Meaningful Data Analysis: The data collected must be turned into informationthat can be related to decisions that need to be made.

6. Results Presented: Clear, precise, targeted to the audience. See the section onPresentation that follows for more detail.

7. Informed Decisions: The ultimate goal of the needs analysis is to make aninformed decision based on reliable and valid information.

Once the needs analysis for training has been accomplished in a professional, reliableand viable way, the “trainer” can proceed to match organizational needs to personalcapabilities, and begin the development of a prospectus.

STEP 2: Prospectus/Agenda Development

Once an informed decision has been made based on the “needs analysis,” step twoof the design and delivery process is to develop a prospectus and agenda for thetraining event. The purpose of the prospectus/agenda is multifold. For the trainer,the prospectus provides a framework to accomplish the training objectives bydocumenting step-by-step the items that need to be accomplished for a successfulevent. For the trainee, the prospectus provides information, builds expectations,and allows for preparation for the event. There are at least 11 items that need to be


systematically addressed in the prospectus and agenda. The items and a briefexplanation are as follows:

Introduction/Background -- this section provides information to the participant onwhy the workshop is being held, its importance, and where it fits into the generalframework of institutional or individual development.

Purpose -- a purpose statement provides information on the specific reason why theworkshop is being held.

Content -- a description of the general content of the workshop provides informationon topical areas to be covered, builds expectations, and allows individuals to beginpreparation or stimulates thinking about the topics.

Objects/Expected Outcomes/Outputs -- the objectives, expected outcomes, and outputsmust be stated prior to the workshop and must be verifiable so that evaluations canbe conducted subsequent to the event.

Target group -- the target group must be carefully identified so that materials andworkshop dynamics can be prepared specifically to suit the situation.

Resource Team and Materials -- the team should be identified for informationalpurposes for the participants. Identifying materials and distributing them prior tothe workshop if possible will facilitate and accelerate the learning process.

Methods/Dynamics -- Pedagogical methods should be specifically defined, particularlyfor interactive and participatory events.

Logistics -- these must be stated prior to the workshop and should include venue,accommodations, transportation, and expense information.

Date -- advance notice will allow people to block out the time required.

Contact -- a contact person to clarify information should be identified.

Agenda -- a daily agenda should be set. This includes time, materials, trainerresponsibilities, and dynamics such as small groups, plenary presentations, etc.

A sample prospectus/agenda is attached for your information (See Attachment 1).


Proper Materials

Print MaterialsLecturesCases/ExercisesAudio-visuals- Slide-tape- Video- Web-based

Goals/Objectives

Skill developmentBehaviorManagement

Proper Implementation

DynamicsEnvironmentStyle/PedogogyFitFacilitation

Knowledge enhancementt

n s

STEP 3: Module Construction

The third step in the process of developing a training event is the construction of thetraining module. As is the case for the entire process, a goal/objective must be setfor each module. Depending on the target audience that has been chosen, the goalsmay be general knowledge, awareness, skill development, attitudinal change, or bettermanagement. Once the goal has been established, then it may be attained with theproper combination of materials and implementation strategies.

Considerations for proper materials depend on the goal and the target group of thetraining. Thus print materials, lectures, cases, practicum, and audio-visuals areselected to suit the situation. Likewise the implementation mode, i.e., dynamics,delivery style, language, and techniques such as brainstorming and exercise, dependsupon the goal and materials. This process is illustrated in the following diagram.

Figure 2. Module construction

STEP 4: Testing and Validation

All materials should be tested prior to being put into their final form. There are anumber of ways to accomplish this objective:

1. Peer Review. This method can take various forms, such as a review or critiqueby professional peers. This may be formal as in a publication review procedure,or informal such as a working group review, brainstorming, and discussion.

2. Expert Consultation. An expert can be hired to review the materials. This personmust be knowledgeable about both the content and the process of the trainingbeing prepared.


3. Field testing of the materials in a workshop setting. Materials may be introducedinto a workshop setting, and then evaluated by the participants.

4. Review by either a project manager or regional specialist who may have fieldexperience or an intuitive feeling for a match with their local experience andpersonal needs assessment orientation.

STEP 5: Redesigning

The redesign phase is an assessment of the original analysis -- the “what is” of thenature of the given knowledge base according to our original analysis of needscompared to the design phase and “what ought to be” the body of knowledge.

The redesign itself focuses on the intervention, that is, whether or not what hasbeen designed and field tested enables the participant to move from the “what is”(i.e., existing knowledge) to the “what ought to be” (i.e., the required knowledge,skills, and attitudes). In other words, has what you have designed moved theparticipant from where s/he was to where s/he wants to be. Analysis of theinformation collected in the testing and validation phase will guide the redesign.

STEP 6: Production of Materials

For the production of materials, a number of items need to be considered.

Comprehension -- the materials should cover all aspects of the topic that is to bepresented. For short workshops, when all areas cannot be presented, counterpoints,different methodologies, and theory papers should be included in the training packageas informational, supplementary, and background reading.

Coverage -- each component must be carefully defined and broken down into itssmallest element so that there is a perspective developed and it is understood howpieces of the puzzle fit together.

Format -- it is preferable to have a notebook or a binder for a workshop. Miscellaneouspapers and sheets of paper will be lost, stacked on a shelf, or mixed with otheritems.


Accessibility -- a file must be accessible, understandable, and easy to reference. Thus,it should be tabbed and have an index to the context.

STEP 7: Delivery

The professional delivery of the event is essential to its success. When most peoplethink of a workshop and delivery, they automatically begin to think about preparationand presentation of lectures. While it is true that the proper preparation and practiceare essential to the success of a training event, presentation skills are not the onlyelement that needs to be considered. Other elements enable the success by allowingthe participants to actively learn, and thus make the information delivered in thetraining event part of their everyday work life. It must be remembered that trainersteach people not materials, and therefore the participant and the process by whichthey are enabled to learn is the most important aspect of training. Major componentsto consider in actualizing the trainee are:

- Creation of the learning environment

- Facilitation techniques

- Presentation skills

If any of these elements are missing, the training event and desired transfer ofinformation will be marginalized.


Figure 3. Delivery elements

Delivery Element 1: The creation of a learning environment

1. The interactive nature of the event/ownership

Participation may be enhanced with interactive techniques such as exercises,cases, problem solving, group and individual tasks. The trainer will have todecide, based on the technical nature of the materials and the level of thetrainees, how much of the event time will be dedicated to lecture, interactiveexercise, and hands-on participation. But it must not be forgotten that active,participatory process, self-directed, experiential learning is the most efficientand effective type of learning. Active participation will create a vested interestor ownership within the participant, thus enhancing the potential for successof the training.

In addition to the event specific techniques, there are generic meeting rolesthat also enhance the dynamics of the events. These are:

Chair/Convenor: The group leader who sets the meeting agenda, decides whenand how things will be done. Duties include keeping the group on task and on

PresentationLearning

Environment

Facilitation

PreparationExecution

- organization- - - -

- Interaction- Configuration- Ownership

Skills

group memoryproblem analysisconflict managementevaluation

nn

st

y


time, proposing and eliciting ideas from the group, creating new ideas whennecessary, and ensuring that the group members have access to informationgenerated.

Facilitator: The neutral role of making the group process flow as efficiently aspossible by providing just enough structure so that the group dynamics don’tinterfere with the agenda. S/he should not be a key participant in the contentdiscussion on agenda issues, but monitors and prompts the discussion onagenda issues, ensuring that some conclusion or consensus is reached oneach issue and making sure that all group members feel included in the process.

The Timekeeper: Pays attention to the passage of time and notifies the groupwhen to move on to the next task.

The Vibes master: S/he is given power by the members to moderate (lessen theintensity of) dysfunctional behavior within the group. S/he steers the flow ofemotion away from dangerous waters. When inappropriate behavior or seriousattacks surface, it is the vibes master who is responsible for cutting it off bycalling attention to it. Since it is the “role” that calls attention to the behaviorrather than a member of the group, the notice is depersonalized and decisioncan be accepted more readily.

Rapporteur: This is potentially the most powerful position in the group sincethe person in this role controls information (and therefore decisions madeupon it) by recording and interpreting what takes place in the group. Duties ofthe rapporteur are to produce a written summary of the conclusions, actions,and recommendations of the group and to report to the plenary sessions whenspecified.

2. The setting

The setting and room configuration are important factors in the successfulcreation of the learning environment. With respect to the room configuration,there are a number of possible designs depending on what is to be accomplished.Some of the standard configurations are as follows:

Small group: Semi-circle, no table vs. semi-circle with table. Generally peoplewill talk to those they are facing, so if you want two people to engage each


other in a conversation, have them face one another. In a small group, it isdifficult to identify the leadership position if there is no table or if there is around table. If a rectangular table is placed in the same group, then theleadership position will be associated with the end of the table. In either ofthese 2 configurations, participants will associate the end that has audio-visual aids as the business end of the design and those wishing to be activelyinvolved will gravitate to that end.

With respect to classroom/training room designs, there are some fairly commonones that are effective depending on what the trainer wants to accomplish.(Training: Special Report: Off-site Training Meetings/November, 1990). Severalare as follow:

Traditional Classroom: The classic arrangement for dissemination of informationis the classroom. This arrangement is appropriate for information downloadin the form of lecture and computer-based training. This is very effective forcontrol of the class and information, but allows for little interaction with thetrainer or fellow participants. There may also be a psychological barrier createdas it may remind participants of power relationships from school days.

Conference Style/Rectangle: The conference style configuration enables groupdiscussions with relatively limited presentation input. Lecturing is disenabledas people will have to move chairs or sit sideways, thereby losing their writing


surface. For effective small group discussions, the optimum number is 6-8, upto 12, but the larger the group the less effective the discussion.

Horseshoe: The horseshoe configuration is suited to small group interactionand at the same time allows for presentation. It is less formal, allows thetrainer to move around, and is suited for cross-table and side-to-sidediscussions. Control remains high in this configuration and sight andparticipation is medium.

Fan-type/Half-round/Chevron: The final common configuration is the Fan type/Half-round/Half-Chevron. These configurations have 4 primary benefits:

All participants have a clear view of the trainer, visual aids, and the participants.It is easy to switch from listening, to working, to discussions. Cross-roomdiscussion is facilitated as there is little visual obstruction. Trainers can roamto assist individuals and groups working on exercises.


A summary of the various designs and their effectiveness follows:

Room Configuration Effectiveness

Semi-circle Classroom Rectangle Horseshoe Half-round

Control Med High Med High Med

Sightlines High Low Med Med High

Participation High Low Low Med High

(Adapted from Training: Special Report Nov. 1990)

Delivery Element 2: The selection and the proper useof facilitation skills

Facilitation quite simply is ”… enabling groups to succeed. More specifically,facilitation is the design and management of structures and processes that help agroup do its work and minimize the common problems people have working together.”(Justice and Jamieson, 1999:5). According to these authors/practitioners, facilitationis a neutral process that focuses on accomplishment involvement, flow,communication, participation, energy, and environment. Those of particular interestfor the IRRI trainer are described here:

1. Organization

In the delivery of a training event, the primary considerations for organizationare to ensure that the best approach/fit has been selected for the needs of theparticipants and that the design of the event will enable the group to succeed.Depending on the needs and structure, the event and activities may take oneor several forms. Examples are: workshop, conference, seminar, colloquium,and/or consultation.


2. Creating and using group memory

Capturing the record of the group is an essential task for success of an event.This capturing of information serves as a record for the event, keeps theindividuals and group on track, and encourages participation. Having a recordkeeper or rapporteur creates the story that is told later.

3. Building the database of ideas

Building a database of ideas for training events may come from multiple sources.One source is the participants themselves and there are a number of techniquesto get the ideas out on the table for discussion. Some of these are: (Summarizedand/or adapted from Justice and Jamieson, 1999):

Leading Questions: One of the necessary tasks of a facilitator is to get everyonein the training event involved and learning. One of the facilitation skills is toask questions that lead participants to develop solutions of their own. Thisrequires the facilitator to seek out ideas by asking open-ended questions,encouraging discussions and dialogue, being an active listener, playing backinformation, openly accepting ideas, and recognizing the audience feeling/acceptance. (Training and Development Journal, 1991).

One type of leading question is the “problem probe”. The purpose of a problemprobe is to identify and agree upon the problem so that the solution constructedis the correct one for the problem. In order to understand the dimensions,boundaries, and nature of the problem, the group is led through a series ofcues or question that help to clarify and define the problem. There are multiplesteps to the method:

Step 1: Each participant writes what s/he thinks the problem is. Participantsare led through a series of probes such as: what happens; wheredoes the problem occur; when does the problem occur; how longhas it been going on; what resources are involved, etc. (The probescan be adapted to the particular problem based on the knowledgeand experience of the group and trainer.)

Step 2: Prepare flip charts with each question on the top. Lead group throughquestions and record data without comment or disagreement. Gothrough the entire series again.


Step 3: Discussion

Step 4: Group forms problem statement ending with “The primary cause(s)of the problem appear(s) to be… ”

Step 5: Check/Feedback

A second tried and true method of soliciting ideas is brainstorming. The purposeof brainstorming is to create a quantity of ideas. While a great number ofarticles and how-to advice has been written about brainstorming, thefundamentals have changed little since Brainstorming, by Charles Clark, 1953.His simple rules were:

Criticism is ruled out.

The wilder the ideas, the better.

Quantity is wanted.

Combination and improvement are sought.

These rules have been refined to include the advice that each idea should berecorded verbatim to ensure that the idea is not altered or interpreted by thefacilitator. There are a number of options that may be used such as: suspendingassumptions, segmenting the process, round robin, anonymous alternatives,being someone else or changing hats (De Bono), idea derby, and guided imagery.

4. Conflict management

While conflict management/resolution is a primary management tool, it isalso a facilitator skill. The first step in conflict management is to recognizethat conflict exists and is part of human communicative behavior. Once it isrecognized that it exists, it should not be repressed. Rather it should bechanneled into positive behavior. When seen as a normal social phenomenon,conflict is positively functional in the following ways:

- It can prevent stagnation.

- It can stimulate interest and curiosity.


- It can be the medium through which problems can arise and be discussed.

- It can be the root of personal and organizational change.

- It can foster cohesiveness.

According to Justice and Jamieson, there are several simple strategies torecognize and manage conflict in a functional manner:

Pinpoint conflict

Identify conflict

Identify resolution options

Point out positive aspects

Differentiate position from interest

Link 2 positions with “and” to determine compatibility/incompatibility

Point out negative consequences of failure to resolve

Surface value of differences

Evaluate (Workshop, Follow-up, Impact) See Step 8

Delivery Element 3: Presentation

While we often hear that some speakers are naturally gifted or that a competentprofessional is a good speaker, neither of these statements is completely true.Communication requires not only a message, but also a skilled sender and a perceptive/engaged receiver.

The skills necessary to accomplish this are acquired through practice and hard work.A separate handbook of guidelines for presentation skills is currently being producedby the Training Center. The following highlights some of the materials from train-the-trainer workshops.


There are two phases to presentation: preparation and execution. In the preparationphase, a number of items need to be considered:

Audience -- Make sure that materials are appropriate for the target audience.

Relevance -- Continually question relevance of material, level and complexity, necessity,importance, etc.

Clarity -- Ask: Are statements clear, precise, understandable?

Simplicity -- Start with simplicity rather than complexity. Materials can always be mademore complex but audiences will be lost if material is too complex for them at thebeginning.

Planning/Preparation -- Speaking, teaching, presenting, are acquired skills. It is essentialthat the presentation be thoroughly planned, prepared, and practiced. Know exactlywhat you are going to say, when to say it, and why. PRACTICE, PRACTICE, PRACTICE.

If phase 1 (the preparation phase) is carried out properly, phase 2, the executionphase, will take care of itself.

Two areas should be considered:

Interest - keep the audience interested by voice modulation, mannerisms, appropriatevisual aids that speak for themselves, and eye contact.

Sensitivity - clarify if the audience does not understand, answer questions, be flexibleand good humored.

1. Personal Effectiveness

For personal effectiveness, there are several items to consider:

Don’t distance yourself -- beware of physical and psychological barriers.

Make the audience comfortable.

Maintain coherence -- be simple, structured, concrete, and visual.

Maintain connection -- clarify, change direction as needed.


2. Visual Aids

If they are properly done, visual aids will assist the education process. It isestimated by adult learning specialists that 40% more of the materials will beretained if proper visual aids are used in the training process. If they areimproper, they are distracting, confusing, and disruptive. Some key things tonote about the content of visual aids:

Simplify -- one topic, key words, phrases

Clarity -- visible, no more than 2 types of lettering: can be multiple colors butyellow, brown, and purple are not good; provide a key if necessary; highlightsignificant figure, words or numbers;

Visibility -- use symbols, diagrams or key words -- no text.

With respect to the use of visual aids, the following are some do’s and don’ts.

Do’s Don’ts

Remove them when they are not needed Speak to the machine

Stand to the side Stand in front of the light oraudience,

Let them speak for themselves Speak while you are changing visualaids

Point to the screen, not the projector

Have one key message per visual aid

Distribute handouts at the end

Questions

The following is a list of tips on how to handle questions: be alert, listen carefully,ask for clarification; don’t let yourself be interrupted; correct false options ormisinformation; speak to one of your friends; avoid “um” and “em”, don’t let yourself


Event/Intervention

Training Event Evaluation

Assess Satisfaction of Trainees

Event AssessmentQuestionaire

Completion of Training

Personal Acquisition

Skills/KnowledgeAcquisition

Assess Change inKnowledge, Skills,Attitudes

Pre-Test/Post Test

Completion of Training

Follow-up

Skill/KnowledgeTransfer

Assess Extentof Applicationof Skill/Knowledgeto Job Related Activities

Survey: Interviewand/or Questionaire

3 months to 6 months

Impact

Organizational Performance Change

Assess OrganizationalChange as a Resultof Skill/KnowledgeTransfer and Incorporation

Baseline Comparison

1-3 years

be taken out of your topic; know your subject but admit it if you don’t havean answer; stay calm; pay attention to messages from the person asking thequestion--overt or hidden.

STEP 8: Evaluation

Since evaluation is the final step in the process and confirms (or denies) your approach,the following section is covered in more detail.

“Millions for training, but not one cent for evaluation. By design or by happenstance,this is an all-too-common occurrence. In many instances it is assumed that trainingprograms have been effective because participants enjoyed the presentations” (Cascioand Awad, 1981:307).

In our recent past, it was said that our clients were smiling, and therefore we aredoing an excellent job. That, of course, was said in jest; however, there are alarminggrains of truth in that, in the past, training events have not been documented in astandard fashion, assessment often has been oriented towards acceptance orappreciation of the event, and little follow-up has been done to date. Given thattraining is one of IRRI’s primary programs, and that it is one of the vehicles orconduits for working group and research materials, it is essential that we instituteand implement a continuous self-evaluation of our program.

To accomplish this task, a conceptual model that measures training outcomes bylevels, is proposed.

Figure 4. Evaluation type/sequence


The conceptual model for evaluating training begins with the need to improveorganizational performance and then sequentially progresses through the provisionof training (level 1), the completion of training (level 2), the resumption of job-related activities (level 3), and the changed organizational performance (level 4).

Improved Organization Performance

The need to improve organizational performance is a given with many organizations.As discussed in “Step 1: Needs Analysis,” the type of improvement required and thespecific intervention prepared is based on a training diagnosis or ex-ante evaluationperformed to identify training gaps or needs for information in the organization.This diagnosis or identification of gaps or needs comes from a variety of sources andinteractions with NARS and their partners, including extension organizations andfarmers.

In our analysis, assessment, and prioritization of training needs we must make thatour planned events are consistent with our strategy, in line with the levels of ourmaterials, planned for appropriately targeted audiences, and within the capability,time, and professional constraints of our staff.

Provision of Training (Level 1)

The evaluation of level 1 focuses on the training event itself as it assesses thesatisfaction of trainees with the trainers, the facilities, and the content of the program.These are simply reaction criteria that measure the trainees’ impressions and/orfeelings about the program. This is commonly accomplished with an evaluation formadministered at the end of a training event.

While we have used a variety of formats for this in the past, we have been workingon a standard data collection format so that the information collected on events iscomprehensive and comparable (See Attachment II)

Data collection is a necessary step in assessing the success of the event and, evenmore important, it is absolutely essential ultimately for the training programevaluation. The recommended reporting format for the level 1 is as follows:


Part 1: Review of the workshop

A. Background

B. General Assessment

C. Preparation

D. Organization

E. Content

i. Agenda

ii. Materials

iii. Presentations

iv. Discussions

v. Synthesis

vi. Report

F. Participants

G. Facilities

H. Interpretation

I. Administrative support

J. Lodging/Entertainment/Transport

K. Resources

Part 2. Summary of Participants’ Evaluation (Recommended Format Attached)

Part 3. Conclusions/Recommendations

Part 4. Annexes


Completion of Training (Level 2)

Level 2 evaluation, done at completion of training, evaluates the output of training interms of increased knowledge, change skills, and attitudes of trainees. This refers to thelearning criteria, which are more rigorous measures of training outputs such as examsor performance tests. The assessment of this level will not necessarily be required orappropriate for all the events. However, for certain types of targeted skill training, pre-tests and post-tests on the skills offered in the courses would provide an assessment ofchanges that occurred as a result of the course. Since our courses are normally ofshort duration, one to two weeks, factors such as history, maturation, testing, andinstrumentation which often contaminate measurement would not be relevant. Forthose events which level two evaluations are appropriate, we should begin to developpre-event or pre-test questionnaires for use as soon as possible.

Resumption of Job-Related Activities (Level 3)

Level 3 evaluation, done at resumption of job-related activities, evaluates the applicationof the newly acquired skills, knowledge, and attitudes as they are related to job activities.These are behavioral criteria that indicate a positive transfer from training to the jobhas occurred. This is an ex-post evaluation that should be conducted three to sixmonths after the training event. This evaluation requires a systematic observation ofbehavior that should be accomplished by a supervisor and supplemented by self-reports. Reports on behavioral changes should be submitted to the trainers, the trainingunits, appropriate National Agricultural Research and Extension Systems (NARES)leaders, and funding and coordinating organizations.

Changed Organizational Performance (Level 4)

Level 4 evaluation, that of changed organizational performance, evaluates change inthe organization that is the result of the training received. While the extent towhich training has improved organizational performance is the ultimate measure oftraining success, it is also the most difficult to access because it is difficult to attributechange. It is essential at the outset that the expected result is clearly defined, and atangible product is possible, so that empirical evidence collected can be measuredagainst some anticipated or expected outcome.


BIBLIOGRAPHYLuka A, Marcotte P. 1989. Proceedings of the SACCAR/ISNAR regional workshop on

human resource management in National Agricultural Research Systems. TheHague: ISNAR.

Bailey K D. 1994. Methods of Social Research, The Free Press.

Cascio W F, Awad. E M. 1981. Human resources management. Reston, Virginia:Reston Publishing Company Inc.

Casse P, Declerck R, Deol S, Rena O.1984. Management training: Guidelines fortrainers. EDI Training Materials 670/022 International Bank for Reconstructionand Development.

Charles C. 1953. Brainstorming.

ISNAR. 1987. Communications techniques trainers manual. The Hague: ISNAR.

Justice T, Jamieson DW. 1999. The facilitator’s fieldbook, HRD Press.

Kerrigan J E, Luke JS. 1987. Management training strategies for developingcountries. Boulder, Colorado: Lynne Rennier Publishers.

Samuel P. 1983. Training for public administration and management in developingcountries: A review. World Bank Staff Working Paper Number 584. WashingtonD.C.: The World Bank.

Pretty J N, Guijt I, Scoones I, Thompson J. 1995. A trainer’s guide for participatoryleaming and action, IIED.

Sachdeva P. 1989. Implementing ISNAR’s training strategy and program. TheHague: ISNAR.

Stanley L A. 1987. Guide to training needs assessment. Ljubljana, Yugoslavia:International Center for Public Enterprises in Developing Countries.

Training: Special report, November 1990.

Training and development journal, 1991.


Section 2: Attachment 1. Prospectus/Agenda Sample

Problem-based Technology Generation for Rice Environments

(an on-the-job training for Indonesian scientists)Training Center, IRRI, Los Banos, Laguna, Philippines

21 February-14 March 2000

COURSE DESCRIPTION

INTRODUCTION

Despite technology advances in agriculture in the last three decades, productivity amongthe greater proportion of the world’s farmers have remained low. It is for this reasonthat equity issues have been raised against agricultural technology development andtransfer; the benefits of technology advances do seem to exclude resource-poor farmers,and attempts at correcting this inequitable situation have met with limited success.Forerunners of a recent trend in agricultural development advocate the need to considerthe farmer’s problems and conditions even at research planning stage, especially theproblems of the majority who farm in resource-poor areas where resource-intensivetechnologies have failed to make a beneficial impact.

Through collaborative research and human resource development efforts, NationalAgricultural Research Systems (NARS) have expanded and strengthened their researchbase to systematically tackle emerging problems in the various rice ecosystems.Nevertheless, there is a need for rice scientists in the NARS to recognize the continuousprocess from problem identification to research planning and implementation to researchresults utilization, and acquire the skills to get involved actively in that process.

This 1-month on-the-job training course intends to develop such understanding andskills in Indonesian rice scientists and R & D professionals to enable them to designresearch that responds directly to farmers’ field problems. It will help them improvetheir skills in problem identification/analysis, research planning, and researchinformation/technology utilization, and enhance their understanding of their role inmaking agricultural technology useful for the greater proportion of Indonesian farmers,including those in the marginal areas.

IRRI Training Guideline s 24

OBJECTIVES

At the end of the course, participants will have enhanced their capability to

• understand the problem identification-research-technology/information utilizationprocess;

• identify and analyze the effects of bio-physical and socioeconomic factors onproduction systems and production problems;

• use various diagnostic tools to identify key production constraints;

• translate diagnostic results to strategic research problems and prioritize theseproblems; and

• design and evaluate technologies to address key production constraints.

METHODOLOGY

Mini-lecturesBrainstorming sessionsGroup discussion/Sharing of experiencesHands-on activitiesField practicumGroup projectPresentation and critique

COURSE REQUIREMENT

Participants are expected to use available data and information on rice production inthe area to be studied that may be useful in cross-checking their own findings. Thesecould be existing base maps, climate data (on rainfall, temperature, wind velocity, etc.),soils data, current crop calendar, and current agronomic data (on yield, seeding rate,crop establishment practices, etc.) documented in reports about the locale.


PROGRAM SCHEDULE

Venue: Breakup Room Time: 8:30 a.m.

Week 1

Day 1

Opening Session

• Welcome remarks and P L Marcotte (Paul)introduction of training team Head, Training Center

• Course overview and M B Quiamco (Madz)leveling of expectations Senior Associate Scientist,

Training Center

COFFEE BREAK

• Filling the world’s rice bowl (a video presentation)

• Tour of TC facilities and introduction to TC staff

Session II: Methods, Tools, and Techniquesfor Problem Diagnosis and Prioritization

• Problem-based research – an overview

• Activity 1: Brainstorming sessionWho are the Indonesian farmers?

Day 2

• Debriefing on Activity 1

• Agricultural development approaches- Evolution of approaches- Farmer participatory approach


• Concepts and approaches to problem diagnosis

• Participatory methods and skillsA. Agroeconomic characterization

− Observation skills− Inventory skills− Mapping skills− Measurement skills

• Activity 2: Developing observation, inventory,mapping, and measurement skills

Day 3

B. Problem identification and diagnosis− Selecting key informants− Formulating guide/key questions− Using different types of questions− Facilitating a brainstorming session− Group management− Listening− Appreciating farmers’ opinion− Handling neutrality− Using appropriate language− Probing techniques

• Activity 3: Preparing for and conducting aninterview

Day 4

• Visits to− the Philippine Council for Agriculture and Natural

Resources Research and Development (PCARRD) BPI Economic Garden, Los Banos, Laguna

− Masajo Farm


Day 5

• Activity 3 continued

• (3 p.m.) Presentation of results to training team

Week 2

Days 1-3

• An overview of the problem V P Singh (Pal)identification-research planning Coordinator,technology/information South Asiadissemination continuum Project and Systems

Agronomist, IRRI

• Participatory problem identificationand diagnosis: the process

• Diagnosis and design interface: Translatingfield problems into research questions

• Activity 4: Group exercises in problemidentification and research question formulation

Days 4-5

Session III: Methods, Tools, and Techniques forExperimentation and Technology Evaluation

• Types of experiments and their features(on-station, on-farm, participatory experiments)

• Experimentation and evaluation- Evaluation of long-term trials- Criteria for assessing technology

• technical feasibility


• biological feasibility• socioeconomic feasibility• sustainability• criteria fo farmers’ evaluation

Week 3

Day 1

• PRA (participatory resource appraisal) tools M B Quiamcoand outputs- maps- resource flows- charts and calendars- diagrams- timelines and trends

• Planning, briefing on the fieldpracticum

Day 2

• Travel to Nueva Ecija

Days 3-5

• Activity 5: Field practicum— Participatory Training Teamagro-ecosystems characterization

1. Agroeconomic characterization andproblem identification with farmers

2. Brainstorming session and group discussion:Translating diagnostic data to strategicresearch problems

3. Feasibility assessment and evaluation4. Preparation of visual presentation tools for

verification meeting with farmers5. Verification meeting6. Revision of visuals to incorporate farmers’ inputs


Days 6-7 (Weekend trip to Pangasinanor Baguio City)

Week 4

Day 1

• Travel back to Los Banos

Days 2-4

Session IV: Beyond Problem Diagnosis:Problem-based Research Planning

• Techniques and guidelines for V P Singhresearch planning

• Activity 6: Formulating a researchplan for practicum site

Day 5

• Presentation of research plan

• Debriefing with training team

• Closing program


Course TitleDate

Venue

Please list what you consider to be 3 strengths of the course:

Please list what you consider to be weaknesses of the course:

Your cooperation in completing this questionnaire will be greatly appreciated. The informationyou provide will be useful in planning future events and will help resource persons and courseorganizers to improve their materials and presentations

In general, I would rate the course as:

On balance, would you say that the course objectives were achieved?

ExcellentGoodAverageFairPoor

YesPartiallyNo

A.General

Reactions

1.

2.

3.

1.

2.

3.

Section 2: Attachment 2. Event Evaluation Form


In th

is se

ction

we w

ould

like f

or yo

u to

rate

the c

onte

nt, t

he u

sefu

lness

, the

pres

enta

tion,

and

the t

ime a

llotm

ent

for ea

ch o

f the

topic

s inc

luded

in th

e cou

rse. W

hen

ratin

g , y

ou sh

ould

cons

ider s

uch

facto

rs as

rigo

r of

mat

erial

(theo

ry, so

undn

ess,

met

hodo

logy)

. In

rega

rd to

, r

ate t

he to

pic i

n te

rms o

f its a

pplic

abilit

y/rel

evan

ce in

your

day-

to-d

ay w

ork a

ctivi

ties.

Fact

ors t

o co

nside

r in

asse

ssing

in

clude

clea

rnes

s, log

ical s

truct

ure,

good

use

of v

isuals

, etc

. Plea

se pl

ace a

chec

k in

the b

ox th

at m

ost a

ccur

ately

repr

esen

ts yo

ur

opini

ons o

f the

se fa

ctors.

cont

ent

usef

ulne

sspr

esen

tatio

n

B.To

pics

1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

11.

12.

13.

Exce

l- len

t

Good

Ave

r- Fa

ir Po

or

age

Cont

ent

U

sefu

lnes

s to

Wor

k

Pr

esen

tatio

n

T

ime

Allo

tmen

t

Exce

l- len

t

Good

Ave

r- Fa

ir Po

or

age

Exce

l- len

t

Good

Ave

r- Fa

ir Po

or

age

Too

Ju

st T

ooSh

ort R

ight L

ong


Very Good Good Fair Poor

Please use the space below to write down any additional comments or suggestions you might have.

Accommodation.........................................Meals..........................................................Stipend.......................................................Lectures......................................................Discussions................................................Papers/Handouts........................................Organization and Management...................Quality of visual aids..................................Quantity of visual aids................................

C.Features

F.Comments

D.What additional topics would you

have likedincluding in this

training activity?

C.On what othermanagement

topics would youlike to receive

training infuture events?

tc0052knew tc websiteraw ma - rice · requires a data-gathering process. ... appropriate data...

Documents